Anna Rogers

Anna Rogers

[Text Machine Lab] Question Answering for Artificial Intelligence (QuAIL)

less than 1 minute read

This is a post I wrote during my time in Text Machine Lab: https://text-machine-lab.github.io/blog/2020/quail/. It presents an English resource for machine reading comprehension, balanced across 4 domains and 9 question types - and our experience of getting the crowdworkers to generate diverse types of data. There are also some follow-up experiments on instability of benchmark results due to variation in fine-tuning runs and data splits.

Rogers, A., Kovaleva, O., Downey, M., & Rumshisky, A. (2020). Getting Closer to AI Complete Question Answering: A Set of Prerequisite Real Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 8722–8731. https://aaai.org/ojs/index.php/AAAI/article/view/6398

You May Also Enjoy

Closed AI Models Make Bad Baselines

24 minute read

Will GPT-4 become a universally expected baseline in NLP research, like BERT in its time? Basic scientific methodology demands otherwise.

The attribution problem with generative AI

15 minute read

Some argue that any publicly available text/art data is fair game for commercial models because human text/art also has sources. But unlike models, we know w...