Self-DSNet: A Novel Self-ONNs Based Deep Learning Framework for Multimodal Driving Distraction Detection

Distraction can cause delayed decision-making and slower awareness, posing significant risks in driving. For reliable driving systems, continuous monitoring of driver behavior is essential to mitigate the impact of distractions. Current strategies for distraction detection widely rely on machine learning models, but the non-linear relationships among various data modalities complicate the identification of optimal combinations. To address these challenges, we propose a novel model, Self-DSNet, for efficient driving distraction detection. The proposed Self-DSNet model utilizes Self-Organizing Neural Network (Self-ONN) layers to enhance complex pattern learning within the data. We employed a publicly available multimodal dataset encompassing three data categories: physiological, vehicle dynamics, and vision-based data. Our approach aims to identify the normal state and three types of distractions: cognitive, emotional, and sensorimotor. The model was evaluated using both single-modality and combined-modality data, focusing on binary classification to distinguish between distracted and non-distracted driving states. The Self-DSNet model demonstrated an impressive accuracy of 94.23% when using vision-based data alone. Incorporating additional physiological data, such as heart rate and breathing rate, alongside vehicle dynamics data, such as steering behavior, further enhanced the model’s performance. The combined data approach achieved a 95.13% accuracy in detecting driving distractions. Specifically, the binary classification yielded a 96.58% accuracy with vision-based data, which increased to 97.31% when steering, breathing rate, and heart rate data were included. Our approach significantly outperformed state-of-the-art methods in terms of classification accuracy. The proposed Self-DSNet model offers a robust solution for driving distraction detection by effectively leveraging multimodal data and enhancing complex pattern recognition through the Self-ONN layers. The model’s high accuracy rates underscore its potential for improving driving safety by providing reliable and continuous monitoring of driver behavior. Future research may focus on real-time implementation and the integration of additional data sources to further refine and validate the model’s effectiveness in diverse driving scenarios.