Det-Recon-Reg: An Intelligent Framework Toward Automated UAV-Based Large-Scale Infrastructure Inspection

Visual inspection remains essential for inspecting infrastructure surfaces. While there are cornerstones in developing intelligent inspection systems, most existing solutions are limited to small-scale infrastructures and components, making them challenging to scale up for real-world applications. Leveraging deep learning and unmanned aerial vehicles (UAVs), this article proposes Det-Recon-Reg, an intelligent framework born for large-scale infrastructure inspection by decomposing it into three complementary stages: detect for defect detection, reconstruct for infrastructure reconstruction, and register for defect registration. In the detect stage, we introduce the first high-resolution dataset designed for defect detection on large-scale infrastructure surfaces. State-of-the-art real-time object detectors are evaluated on this dataset, and the CUBIT-Net is proposed to strike a better balance between accuracy and efficiency. In the reconstruct stage, we present a scalable multi-view stereo (MVS) network to reconstruct dense point cloud representation of the infrastructure from multi-view images. Extensive experiments on benchmark datasets, including DTU, Tanks and Temples (TNT), and BlendedMVS, demonstrate the superior performance of our method over existing approaches. In the register stage, we propose a novel defect registration method that leverages the geographic information system (GIS) to accurately map the detected defects onto the infrastructure model while preserving their geometric and visual properties, thereby enabling global defect localization and more informed maintenance decision-making. The proposed framework can serve as a reference for effective and efficient infrastructure maintenance as consolidated in real-world experiments. Codes, datasets, and pretrained models for each stage will be released at https://github.com/YANG-SOBER/Det-Recon-Reg. The supplementary video is available at: https://youtu.be/MVMp7k9qB84