Application of multimodal deep learning in the auxiliary diagnosis and treatment of dermatological diseases

Skin diseases are important factors affecting health and quality of life, especially in rural areas where medical resources are limited. Early and accurate diagnosis can reduce unnecessary health and economic losses. However, traditional visual diagnosis poses a high demand on both doctors’ experience and the examination equipment, and there is a risk of missed diagnosis and misdiagnosis. Recently, advances in artificial intelligence technology, particularly deep learning, have resulted in the use of unimodal computer-aided diagnosis and treatment technologies based on skin images in dermatology. However, due to the small amount of information contained in unimodality, this technology cannot fully demonstrate the advantages of multimodal data in the real-world medical environment. Multimodal data fusion can fully integrate various types of data to help doctors make more accurate clinical decisions. This review aimed to provide a comprehensive overview of multimodal data and deep learning methods that could help dermatologists diagnose and treat skin diseases.