High Performance GPU Strategies for Real-Time In-Cabin Monitoring

With the expeditious advancement in multicarrier automotive in-vehicle monitoring for a bus conforming environment with computer vision, there is an exigency for a setup subsisting on inspection cameras and an onboard embedded computing device. This paper offers a novel method for in-cabin occupant recognition with an integration of hardware subsystem comprising NVIDIA AGX Orin Development Kit, camera sensors, and camera interface serdes card accompanying software components embracing Pytorch, Open Neural Network Exchange (ONNX), TensorRT (TRT), CuPy, CV-CUDA, and NVIDIA Nsight Profiler tool. This work presents an optimized model construction picked up from a pool of state-of-the-art deep learning models and deployment, achieving high accuracy and low latency, furthermore permitting real-time video streaming and inference. The inference performance of an embedded device hiring a Graphic Processing Unit (GPU) and Compute Unified Device Architecture (CUDA) is embellished on TRT with CuPy and CV-CUDA to escalate GPU performance. Each GPU optimization strategy is thoroughly analyzed in the context of design space exploration or on-the-fly tuning. The GPU performance is monitored with the NVIDIA Nsight Profiler tool to knock off the GPU coldspots, GPU starvation, and asynchronous CUDA memory copy operations. The Roofline model analysis in Nsight Compute is used to pinpoint the exact causes of GPU underutilization. The experimental performance showcase to achieve a test accuracy of over 80% with the boost in GPU utilization by a factor of 2.24.