IterNet++: An improved model for retinal image segmentation by curvelet enhancing, guided filtering, offline hard‐sample mining, and test‐time augmenting

Abstract In clinical medicine, the segmentation of blood vessels in retinal images is essential for subsequent analysis in clinical diagnosis. However, retinal images are often noisy and their vascular structure is relatively tiny, which poses significant challenges for vessel segmentation. To improve the performance of vessel segmentation, an improved model IterNet++ based on the architecture of IterNet is proposed. First, curvelet signal analysis is applied to enhance retinal images. Second, residual convolution (ResConv) blocks and guided filters are introduced to utilise the encoder features of previous iterations in the model to reduce overfitting. Third, offline hard‐sample mining is used to improve segmentation performance by utilising training samples with low segmentation accuracy as many possible on a few‐sample training set. In addition, a test‐time augmentation method is applied to testing samples in test dataset during inference. Extensive experiments show that this model achieves Dice scores of 0.8313, 0.8277, and 0.8372 on DRIVE, CHASE‐DB1, and STARE datasets, respectively, demonstrating the best performance compared with IterNet and other baseline models.