Multimodality-based framework for enhanced skin lesion recognition over federated learning

초록

Skin cancer remains one of the most common and serious health conditions worldwide, driven by the uncontrolled proliferation of skin cells. Early detection is critical, as diagnostic delays strongly correlate with increased mortality, making timely identification essential for positive treatment outcomes. Although artificial intelligence (AI) offers transformative potential in medical diagnostics, current approaches face notable limitations, including the scarcity of high-quality data, privacy challenges in handling patient information, and suboptimal performance in accurately recognizing skin lesions. To address these challenges, this study proposes a novel Multi-modal Adaptive Federated Learning Network (MAFL-Net) that systematically integrates multimodal dermoscopic images and structured clinical metadata within a Federated Learning (FL) environment to improve skin lesion diagnosis while ensuring patient privacy. The framework fuses visual and clinical metadata streams by employing an optimized ConvNeXt-Large backbone with a Channel Attention Module (CAM) for advanced dermoscopic feature extraction, while a one-dimensional Convolutional Neural Network (1D-CNN) with a skip connection and a Self-Attention Mechanism (SAM) interprets clinical metadata. Distinct feature representations from each modality are combined through a late-fusion strategy that preserves high-level modality-specific features and reduces noise from premature cross-modal interactions. In addition, this study introduces an Adaptive Federated Averaging (AdaFedAvg) algorithm that enables efficient multimodal collaborative learning while maintaining patient privacy. The proposed MAFL-Net framework was evaluated on the HAM10000, ISIC 2019, and ISIC 2024 datasets, achieving superior accuracies of 99.65 %, 98.87 %, and 95.46 %, respectively, and outperforming State-of-the-Art (SOTA) methods. Overall, the results demonstrate the efficiency, scalability, and effectiveness of MAFL-Net for skin lesion recognition through the systematic integration of multimodal data enhanced by adaptive FL techniques.

키워드