[The Gradient] When BERT Plays The Lottery, All Tickets Are Winning

less than 1 minute read

This is a post I wrote for The Gradient: https://thegradient.pub/when-bert-plays-the-lottery-all-tickets-are-winning/. It reports on our investigation of the lottery ticket hypothesis for pre-trained Transformers, in particular BERT:

Sai Prasanna, Anna Rogers, and Anna Rumshisky. 2020. When BERT Plays the Lottery, All Tickets Are Winning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3208–3229, Online. Association for Computational Linguistics. https://aclanthology.org/2020.emnlp-main.259/