[Text Machine Lab] Question Answering for Artificial Intelligence (QuAIL)

less than 1 minute read

This is a post I wrote during my time in Text Machine Lab: https://text-machine-lab.github.io/blog/2020/quail/. It presents an English resource for machine reading comprehension, balanced across 4 domains and 9 question types - and our experience of getting the crowdworkers to generate diverse types of data. There are also some follow-up experiments on instability of benchmark results due to variation in fine-tuning runs and data splits.

Rogers, A., Kovaleva, O., Downey, M., & Rumshisky, A. (2020). Getting Closer to AI Complete Question Answering: A Set of Prerequisite Real Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 8722–8731. https://aaai.org/ojs/index.php/AAAI/article/view/6398