An Empirical Evaluation of Out-of-Distribution Detection Using Pretrained Language Models

초록

In Natural Language Processing (NLP) tasks, detecting out-of-distribution (OOD) samples is essential to safely deploy a language model in real-world problems. Recently, several studies report that pre-trained language models (PLMs) accurately detect OOD data compared to LSTM, but we empirically find that PLMs show sub-par OOD detection performance when (1) OOD samples have similar semantic representation to in-distribution (IND) samples and (2) PLMs are finetuned under data scarcity settings. To alleviate above issues, state-of-the-art uncertainty quantification (UQ) methods can be used, but the comprehensive analysis of UQ methods with PLMs has received little consideration. In this work, we investigate seven UQ methods with PLMs and show their effectiveness in the text classification task.

키워드