None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering
Published in arXiv preprint, 2025
This study examines how “None of the Above” (NA) options affect LLM performance on multiple-choice questions. Through systematic experiments with 28 LLMs on the MMLU benchmark, results reveal a consistent 30-50% performance drop when NA is the correct answer regardless of model scale, suggesting LLMs lack the meta-cognitive ability to systematically evaluate and reject all options when none are correct. This degradation shows strong domain dependence, with minimal impact on mathematical reasoning (14.6% drop) but severe effects on tasks requiring uncertainty handling like business ethics (48.1% drop). The research highlights important implications for benchmark design and raises questions about LLMs’ ability to handle uncertainty in real-world applications.
Recommended citation: Tam, Z.R., Wu, C.K., & Chen, Y.N. (2025). “None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering.” arXiv preprint arXiv:2503.01550.
Recommended citation: Tam, Z.R., Wu, C.K., & Chen, Y.N. (2025). "None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering." arXiv preprint arXiv:2503.01550.
Download Paper