Exploring Multi-View Pixel Contrast for General and Robust Image Forgery Localization

Image forgery localization, which aims to segment tampered regions in an image, is a fundamental yet challenging digital forensic task. While some deep learning-based forensic methods have achieved impressive results, they directly learn pixel-to-label mappings without fully exploiting the relationship between pixels in the feature space. To address such deficiency, we propose a Multi-view Pixel-wise Contrastive algorithm (MPC) for image forgery localization. Specifically, we first pre-train the feature extraction backbone network with a supervised contrastive loss to model pixel relationships in view of within-image, cross-scale and cross-modality. That is aimed at increasing intra-class compactness and inter-class separability. Then the localization head is fine-tuned using cross-entropy loss, resulting in a better forged pixel localizer. The MPC is trained on three different scale training datasets to make a comprehensive and fair comparison with existing image forgery localization algorithms. Extensive test results on over ten public datasets show that the proposed MPC achieves higher generalization performance and robustness than the state-of-the-arts. It is particularly noteworthy that our approach maintains a high level of localization accuracy under various post-processing combinations that approximate real-world scenarios, as well as when confronted with novel intelligent editing techniques. Finally, comprehensive and detailed ablation experiments demonstrate the reasonableness of MPC.