HMPFormer: Hierarchical vision transformer with multi-perspective feature learning for precise polyp segmentation

초록

Accurate polyp segmentation (PS) in endoscopic images is critical for early diagnosis and treatment of colorectal cancer, yet remains challenging due to significant variation in polyp morphology and complex boundaries. Existing approaches are limited by their reliance on uniform feature extraction mechanisms and basic multi-scale fusion strategies. Furthermore, single-stream attention mechanisms struggle to handle the diverse morphological characteristics and subtle boundary details of polyps in colonoscopy images. To address these limitations, we introduce a novel Hierarchical Vision Transformer with Multi-Perspective Feature Learning (MPFL) framework called HMPFormer for precise PS. Our framework integrates effective feature learning with attention-guided refinement, allowing robust feature representation and accurate boundary delineation across multiple scales. The proposed architecture leverages a Pyramid Vision Transformer v2 (PVTv2) backbone enhanced with a MPFL module, which enriches feature diversity through the impactful package of parallel dynamic, deformable, and standard convolutions. A Cross-Scale Context Fusion (CSCF) module establishes bidirectional cross-resolution interactions, enabling adaptive aggregation of multi-level semantic and spatial information. For precise mask generation, the progressive decoder equipped with Hierarchical Complementary Attention Module (HCAM) employs reverse refinement with residual attention, emphasizing overlooked regions through complementary masking. Extensive experiments on five benchmark datasets (Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, ETIS-LaribPolypDB, and Endoscene) demonstrate HMPFormer's superior performance over state-of-the-art approaches. Code, predcictions and trained models will be made publicly available at: https://github.com/TalhaUsman-ZERO/HMPFormer.

키워드