End-to-end 자율주행을 위한 경량 VLM: Q-Former 학습 유무에 따른 성능 분석

초록

Recent advances in vision-language models (VLMs) have enabled integrated visual perception and language understanding for autonomous driving. However, the high computational cost of large-scale VLMs limits their practical deployment in real-time systems. In this study, we revisit the common assumption that jointly training the Q-Former improves adaptability in End-to-End autonomous driving. Through empirical analysis, we demonstrate that training the Q-Former can, in some cases, hinder performance by introducing representational bottlenecks or disrupting pre-trained visual semantics. Instead, we show that applying Low-Rank Adaptation (LoRA) to the language model alone achieves better accuracy with significantly fewer trainable parameters. Our findings highlight the importance of careful architectural tuning and suggest that simpler adaptation strategies may be more effective for real-world VLM deployment in autonomous driving.

키워드