Wake Homing Torpedo Guidance Using a Hierarchical Deep Reinforcement Learning Framework

This paper proposes a novel Hierarchical Deep Reinforcement Learning (HRL) framework for wake homing torpedo guidance, applying the Discrete Event System Specification (DEVS) formalism to design high-level policies and reward shaping functions. Wake homing torpedo guidance generates course commands to enable the torpedo to follow the wake trajectory of a target ship. When the target ship evades the incoming torpedo, the wake trajectory becomes curved, often causing the torpedo to lose track due to the narrow detection range of the wake detection sensor. This necessitates a sophisticated algorithm to consistently track the target ship, particularly in scenarios where the torpedo exits and re-enters the wake trajectory in noisy environments. While heuristic algorithms can handle typical wake trajectories, developing a robust solution for unknown trajectories remains a significant challenge. To address this, we apply a novel reinforcement learning approach to develop the guidance logic and compare its performance with a conventional algorithm-based method. The performance and effectiveness of the proposed approach are demonstrated through numerical simulations.