Two-person interactive action recognition based on hypergraph convolutional networks

Abstract Two-person interactive action recognition has significant potential in applications such as security monitoring and educational assistance. Currently, methods based on the combination of joint data and graph convolutional networks have shown promising results. However, these approaches still face challenges in adequately capturing the interaction features of two-person actions, and the models are often complex, limiting practical applications. To address these issues, a two-person interactive action recognition algorithm based on hypergraph convolutional networks is proposed. Firstly, the input data are symmetrically processed and enhanced. Then, the two-person interaction hypergraph is used to model the two-person interaction hypergraph, consisting of a two-person hypergraph and interaction relation matrices. Finally, the two-person interaction hypergraph is deployed into a multi-stream graph convolutional network, enhanced with Spatial-Temporal Part Attention to improve recognition accuracy. This method requires fewer parameters, while effectively capturing both local and long-range spatiotemporal information for each individual, as well as better leveraging the interaction between the two people. Experiments on the NTU RGB+D 60 and NTU RGB+D 120 interaction datasets show that the algorithm achieves an accuracy of 98.24%, with a smaller model size and faster inference speed.