Research on enhancing model performance by merging with Korean language models

초록

This study proposes a novel and straightforward approach to enhancing the capabilities of top-ranking Large Language Models (LLMs) from the Open LLM Leaderboard, leveraging the Drop and Rescale (DARE) technique. DARE facilitates efficient model merging by minimizing delta parameter redundancy from fine-tuned models. We integrate a top-performing multilingual LLM with a specialized Korean language model using DARE. The merged model is evaluated on six benchmark tasks and a multi-turn question set (MT-Bench), focusing on reasoning capabilities. Results show that incorporating the Korean language model achieves a significant performance improvement of 1.69% on average across the six benchmark tasks, and notably demonstrates over 20% higher performance on Grade School Math 8K (GSM8K), which requires complex reasoning skills. This suggests that the inherent complexity and rich linguistic features of the Korean language contribute to enhancing LLM reasoning abilities. Moreover, the model exhibits superior performance on MT-Bench, demonstrating its effectiveness in real-world reasoning tasks. This study highlights DARE's potential as an effective method for integrating specialized language models, demonstrating the ability to unlock existing language models for advanced tasks.1 © 2025 The Authors

키워드