I Need Help! Evaluating LLM’s Ability to Ask for Users’ Support: A Case Study on Text-to-SQL Generation
Published in arXiv, 2024
This study explores the proactive ability of LLMs to seek user support. We propose metrics to evaluate the trade-off between performance improvements and user burden, and investigate whether LLMs can determine when to request help under varying information availability.
Recommended citation: Wu, C.K., Tam, Z.R., Wu, C.C., Lin, C.Y., Lee, H., & Chen, Y.N. (2024). “I Need Help! Evaluating LLM’s Ability to Ask for Users’ Support: A Case Study on Text-to-SQL Generation.” arXiv preprint arXiv:2407.14767.
Recommended citation: Wu, C.K., Tam, Z.R., Wu, C.C., Lin, C.Y., Lee, H., & Chen, Y.N. (2024). "I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generation." arXiv preprint arXiv:2407.14767.
Download Paper