Multi‐Scale Transformer for Image Restoration

ABSTRACT Although Transformer‐based image restoration methods have demonstrated impressive performance, existing Transformers still insufficiently exploit multiscale information. Previous non‐Transformer‐based studies have shown that incorporating multiscale features is crucial for improving restoration results. In this paper, we propose a multiscale Transformer (MST) that captures cross‐scale attention among tokens, thereby effectively leveraging the multiscale patch recurrence prior of natural images. Furthermore, we introduce a channel‐gate feed‐forward network (CGFN) to enhance inter‐channel information aggregation and reduce channel redundancy. To simultaneously utilise global, local and multiscale features, we design a multitype feature integration block (MFIB). Extensive experiments on both image super‐resolution and HEVC compressed video artefact reduction demonstrate that the proposed MST achieves state‐of‐the‐art performance. Ablation studies further verify the effectiveness of each proposed module.