Deep learning framework for crowd congestion detection in smart cities via encoding motion irregularities using recurrence plots

Abstract With the rapid growth of urbanization, large public gatherings in the form of religious festivals, marathons, political rallies are commonly observed in modern cities. Ensuring public safety and security is one the main responsibilities of law enforcement agencies during such events. Despite safety measures, crowd disaster often occurs during such events which leads to the fatalities and injuries. Therefore, effective crowd management is important to ensure public safety and support resilient urban life in smart cities. Traditional crowd management systems rely on manual monitoring and analysis of video streams by human analyst. These methods are slow and labor-intensive and are not suitable for real-time decision making. To address this challenge, we propose a framework based on advanced deep learning approach that detects and localizes congested areas in crowded scenes. Our framework utilizes recurrence plots (RP) images to encode irregularities in crowd motion. These RP images are then classified using a specialized convolutional neural network tailored for congestion detection. We use both synthetic and real-world datasets to evaluate the performance of proposed framework. From the extensive experimental results, we conclude that proposed framework achieves better performance in detecting congested areas compared to state-of-the-art methods.