A Novel Action-Aware Multi-Agent Soft Actor–Critic Algorithm for Tight Formation Control in USV Swarm

Tight-formation control is a key technology for unmanned surface vehicle (USV) swarms in harbor navigation, cooperative berthing, and operations in hazardous environments, yet achieving reliable obstacle avoidance while maintaining formation stability remains highly challenging. Although multi-agent reinforcement learning has shown strong potential in cooperative systems, parallel policy structures in many existing methods still struggle to achieve synchronized coordination in tight formations, leading to behavioral inconsistencies and unstable formation keeping. To address these challenges, an action-aware multi-agent soft actor–critic (AAMASAC) algorithm is proposed that introduces a hierarchical, action-aware decision mechanism. Within each time step, upper-layer actions are propagated as prior signals to lower-layer policies, establishing an ordered, intent-aligned decision flow that mitigates temporal inconsistency and enhances coordination efficiency. The architecture explicitly encodes inter-layer dependencies via a decision priority hierarchy and real-time behavioral information channels, enabling more accurate credit assignment and more stable value estimation and policy optimization. Across three representative validation scenarios, the AAMASAC algorithm significantly outperforms baseline methods in average reward, path-tracking accuracy, formation stability, and obstacle-avoidance performance. These results indicate that introducing a hierarchical model and action awareness effectively improves control accuracy and coordination in a USV swarm.