Assessment of Foundation Models' Applicability to Visualization

초록

Direct Volume Rendering (DVR) and Cinematic Rendering (CR) are representative visualization techniques that transform Computed Tomography (CT) and Magnetic Resonance Imaging (MRI)-based 3D volume data into intuitive images using color, lighting, and shading information. These visualization images allow efficient understanding of complex anatomical structures, but the results vary greatly depending on user-defined parameters such as transfer functions and lighting. Existing Region of Interest (ROI)-based automatic enhancement methods primarily rely on scribble inputs, leading to limited accuracy and consistency. Recently, foundation models capable of performing segmentation using simple prompts such as points or boxes have opened new possibilities. SAM has demonstrated strong segmentation performance on natural images, while MedSAM has shown high accuracy on medical images, suggesting the potential to apply intuitive ROI specification to visualization images as well. However, visualization images form a hybrid domain combining characteristics of natural images and medical images, making it unclear which model is more suitable. This study compares and evaluates the segmentation performance of SAM and MedSAM on DVR and CR-based visualization images to identify the model best suited for the hybrid domain and analyze its applicability.

키워드