AI and post-quantum cryptography powered cybersecurity approaches for IoT systems: a literature review

The rapid expansion of the Internet of Things (IoT) has introduced significant cybersecurity challenges across healthcare, industrial systems, and smart cities. Existing security models provide limited protection due to weak authentication, heterogeneous device architectures, and constrained computational resources. Although Artificial Intelligence (AI) methods, particularly Long Short-Term Memory (LSTM) networks for anomaly detection and Reinforcement Learning (RL) for adaptive defense, show strong potential, most studies emphasize performance metrics while overlooking scalability, energy consumption, and deployability. Concurrently, classical cryptography approaches such as Rivest–Shamir–Adleman (RSA) and Elliptic Curve Cryptography (ECC) are increasingly vulnerable to quantum computing, creating an urgent need for Post-Quantum Cryptography (PQC). This study addresses these gaps by exploring how AI techniques and PQC can jointly strengthen IoT systems in a manner that is adaptive, resource-efficient, and resistant to quantum-level attacks. A systematic literature review (SLR) was conducted following PRISMA 2020 guidelines and Kitchenham’s software engineering methodology. The review examined 150 peer-reviewed articles published between 2014 and 2025 from IEEE, ACM, Springer, ScienceDirect, Scopus, and Web of Science. Findings show that LSTM models consistently achieve high anomaly detection accuracy (91–99%) but impose computational burdens that limit real-time scalability. RL-based defense mechanisms provide strong adaptability for threat response, yet require careful balancing of resource efficiency and response flexibility and currently lack standardization. PQC methods, particularly lattice-based schemes such as Kyber, LWE, and NTRU, demonstrate promise for securing constrained IoT environments against quantum threats, but introduce challenges related to key sizes, hardware demands, and deployment feasibility. Integrated AI-PQC models remain in early development, with limited practical implementations and several unresolved trade-offs. Key challenges include aligning AI’s computational requirements with PQC’s increased resource consumption, ensuring interoperability under strict hardware constraints, and minimizing system-wide latency while maintaining reliability. This review establishes an initial roadmap for developing practical AI-PQC security solutions for IoT, highlighting essential trade-offs among security strength.