Enabling Competitive Performance of Medical Imaging with Diffusion Model-generated Images without Privacy Leakage

Deep learning methods have impacted almost every research field, demonstrating notable successes in medical imaging tasks such as denoising and super-resolution. However, the prerequisite for deep learning is data at scale, but data sharing is expensive yet at risk of privacy leakage. As cutting-edge AI generative models, diffusion models have now become dominant because of their rigorous foundation and unprecedented outcomes. Here we propose a latent diffusion approach for data synthesis without compromising patient privacy. In our exemplary case studies, we develop a latent diffusion model to generate medical CT, MRI and PET images using publicly available datasets. We demonstrate that state-of-the-art deep learning-based denoising/super-resolution networks can be trained on our synthetic data to achieve image quality equivalent to what the same network can achieve after being trained on the original data (the p values well exceeding the threshold of 0.05). In our advanced diffusion model, we specifically embed a safeguard mechanism to protect patient privacy effectively and efficiently. Consequently, every synthetic image is guaranteed to be different by a pre-specified threshold from the closest counterpart in the original patient dataset. Our approach allows privacy-proof public sharing of diverse big datasets for development of deep models, potentially enabling federated learning at the level of input data instead of local network weights.