[The Gradient] When BERT Plays The Lottery, All Tickets Are Winning

less than 1 minute read

This is a post I wrote for The Gradient: https://thegradient.pub/when-bert-plays-the-lottery-all-tickets-are-winning/. It reports on our investigation of the lottery ticket hypothesis for pre-trained Transformers, in particular BERT:

Sai Prasanna, Anna Rogers, and Anna Rumshisky. 2020. When BERT Plays the Lottery, All Tickets Are Winning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3208–3229, Online. Association for Computational Linguistics. https://aclanthology.org/2020.emnlp-main.259/

You May Also Enjoy

[Text Machine Lab] BERT Busters: Outlier Dimensions that Disrupt Transformers

less than 1 minute read

This is a post I wrote during my time in Text Machine Lab: https://text-machine-lab.github.io/blog/2020/bert-secrets/. It reports on an influential paper on ...

[Text Machine Lab] BERT Busters: Outlier Dimensions that Disrupt Transformers

less than 1 minute read

This is a post I wrote during my time in Text Machine Lab: https://text-machine-lab.github.io/blog/2021/busters/. It reports on one of the first studies on t...

[The Gradient] How Can We Improve Peer Review in NLP?

less than 1 minute read

This is a post I wrote for The Gradient: https://thegradient.pub/how-can-we-improve-peer-review-in-nlp/. It reports on a position paper that discusses the h...