Exploring the Self-Organizing Mechanisms for Maintaining Stability in Multi-Agent Systems within Dynamic Environments

Abstract

In the absence of external interference, independent multi-agent systems evolve over time, leading to a transition from an ordered to a chaotic state within the internal dynamics of the scene. To assist in maintaining the stability of dynamic systems, this paper explores the design and composition of the scenario. A category of altruistic agents with selfless attributes is introduced into the multi-agent environment, and the decentralized algorithm I-DDPG is employed to train these altruistic agents to perform tasks with altruistic tendencies. Weakly connected self-organization is used to enable cooperation among altruistic agents in addressing internal issues. The study demonstrates a positive correlation between the overall system stability and individual information entropy, as well as the method of obtaining individual altruistic rewards. To investigate the connection between overall scene stability and self-organization among internal individuals, a traffic scenario is constructed with both human-driven and autonomous vehicles. Autonomous vehicles (AVs) represent a type of altruistic agent, and experiments are conducted to explore the relationship between scene stability and internal self-organization among individuals. The study concludes that using I-DDPG improves the learning efficiency of agents compared to DDPG and demonstrates a higher safety level in stress tests. Post-training, agents utilizing weakly connected self-organization in scenarios involving merging and exiting obtain higher altruistic rewards. This approach aids human-driven vehicles in achieving their intended goals, contributing to an overall improvement in the stability of dynamic systems.