[Text Machine Lab] BERT Busters: Outlier Dimensions that Disrupt Transformers

This is a post I wrote during my time in Text Machine Lab: https://text-machine-lab.github.io/blog/2020/bert-secrets/. It reports on an influential paper on the analysis of self-attention mechanism in BERT, which by 2023 had over 500 citations:

Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the Dark Secrets of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4365–4374, Hong Kong, China. Association for Computational Linguistics. https://aclanthology.org/D19-1445/

See also our follow-up 2022 study that used the types of BERT self-attention patterns, proposed in the above paper, in the analysis of ‘good’ and ‘bad’ subnetworks in BERT!

Sai Prasanna, Anna Rogers, and Anna Rumshisky. 2020. When BERT Plays the Lottery, All Tickets Are Winning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3208–3229, Online. Association for Computational Linguistics. https://aclanthology.org/2020.emnlp-main.259/