I am joining ACL Rolling Review
It’s official: I joined the ACL Rolling Review team as an editor-in-chief, and I’d like to share some brief thoughts on this.
Posts by Tag
- peer-review 9
- academia 7
- methodology 6
- transformers 5
- LLMs 5
- conferences 4
- ethics 4
- society 2
- teaching 2
- debate 2
- negative-results 1
- review 1
- machine-reasoning 1
- productivity 1
peer-review
[ACL 2023] Peer Review Report
This post (at ACL conference website) summarizes the analysis of ACL’23 peer review process: https://2023.aclweb.org/blog/review-report/. The full analysis i...
Closed AI Models Make Bad Baselines
Will GPT-4 become a universally expected baseline in NLP research, like BERT in its time? Basic scientific methodology demands otherwise.
[ACL 2023] Paper-Reviewer Matching
As a program chair of ACL’23, I was the lead author for this blog post on the conference website that summarized our approach to peer-review matching: https:...
[ACL 2023] Generative AI Policy
This blog post (on the conference website) summarized our approach to the use of generative AI in ACL conference submissions and reviewing: https://2023.aclw...
[The Gradient] How Can We Improve Peer Review in NLP?
This is a post I wrote for The Gradient: https://thegradient.pub/how-can-we-improve-peer-review-in-nlp/. It reports on a position paper that discusses the h...
Should the reviewers know who the authors are?
Why fully anonymous peer-review is important, and how we can achieve that in ACL rolling review reform.
Peer review in NLP: resource papers
Resource papers strike back! How the authors and the reviewers can stop talking past each other.
Peer review in NLP: reject-if-not-SOTA
Many reviewers at major NLP conferences tend to reject models that fail to beat state-of-the-art. It is a heuristic that is simple, convenient, and wrong.
academia
On AI-assisted writing in graduate school
What is the proper role of AI ‘assistance’ in graduate student writing? It depends on what you mean by ‘graduate’.
I am joining ACL Rolling Review
It’s official: I joined the ACL Rolling Review team as an editor-in-chief, and I’d like to share some brief thoughts on this.
Closed AI Models Make Bad Baselines
Will GPT-4 become a universally expected baseline in NLP research, like BERT in its time? Basic scientific methodology demands otherwise.
[ACL 2023] Generative AI Policy
This blog post (on the conference website) summarized our approach to the use of generative AI in ACL conference submissions and reviewing: https://2023.aclw...
Should the reviewers know who the authors are?
Why fully anonymous peer-review is important, and how we can achieve that in ACL rolling review reform.
How the Transformers broke NLP leaderboards
With the huge Transformer-based models such as BERT, GPT-2, and XLNet, are we losing track of how the state-of-the-art performance is achieved?
Why blog about NLP in 2019?
Benefits of blogging for the academic souls.
methodology
A Sanity Check on ‘Emergent Properties’ in Large Language Models
One of the often-repeated claims about LLMs is that they have ‘emergent properties’. Unfortunately, in most cases the speaker/writer does not clarify what th...
Closed AI Models Make Bad Baselines
Will GPT-4 become a universally expected baseline in NLP research, like BERT in its time? Basic scientific methodology demands otherwise.
Peer review in NLP: resource papers
Resource papers strike back! How the authors and the reviewers can stop talking past each other.
Peer review in NLP: reject-if-not-SOTA
Many reviewers at major NLP conferences tend to reject models that fail to beat state-of-the-art. It is a heuristic that is simple, convenient, and wrong.
On word analogies and negative results in NLP
Negative results are hard to publish, and even harder to make well-known. Even when the disproved result is something as pervasive as Mikolov’s word analogies.
How the Transformers broke NLP leaderboards
With the huge Transformer-based models such as BERT, GPT-2, and XLNet, are we losing track of how the state-of-the-art performance is achieved?
transformers
A Sanity Check on ‘Emergent Properties’ in Large Language Models
One of the often-repeated claims about LLMs is that they have ‘emergent properties’. Unfortunately, in most cases the speaker/writer does not clarify what th...
[Text Machine Lab] BERT Busters: Outlier Dimensions that Disrupt Transformers
This is a post I wrote during my time in Text Machine Lab: https://text-machine-lab.github.io/blog/2021/busters/. It reports on one of the first studies on t...
[The Gradient] When BERT Plays The Lottery, All Tickets Are Winning
This is a post I wrote for The Gradient: https://thegradient.pub/when-bert-plays-the-lottery-all-tickets-are-winning/. It reports on our investigation of th...
[Text Machine Lab] BERT Busters: Outlier Dimensions that Disrupt Transformers
This is a post I wrote during my time in Text Machine Lab: https://text-machine-lab.github.io/blog/2020/bert-secrets/. It reports on an influential paper on ...
How the Transformers broke NLP leaderboards
With the huge Transformer-based models such as BERT, GPT-2, and XLNet, are we losing track of how the state-of-the-art performance is achieved?
LLMs
On AI-assisted writing in graduate school
What is the proper role of AI ‘assistance’ in graduate student writing? It depends on what you mean by ‘graduate’.
AI ‘News’ Content Farms Are Easy to Make and Hard to Detect
AI ‘news’ produced by LLM-powered content farms are flooding the web. They are currently very cheap and easy to produce, even beyond English, and currently w...
A Sanity Check on ‘Emergent Properties’ in Large Language Models
One of the often-repeated claims about LLMs is that they have ‘emergent properties’. Unfortunately, in most cases the speaker/writer does not clarify what th...
Closed AI Models Make Bad Baselines
Will GPT-4 become a universally expected baseline in NLP research, like BERT in its time? Basic scientific methodology demands otherwise.
The attribution problem with generative AI
Some argue that any publicly available text/art data is fair game for commercial models because human text/art also has sources. But unlike models, we know w...
conferences
[ACL 2023] Peer Review Report
This post (at ACL conference website) summarizes the analysis of ACL’23 peer review process: https://2023.aclweb.org/blog/review-report/. The full analysis i...
[ACL 2023] Paper-Reviewer Matching
As a program chair of ACL’23, I was the lead author for this blog post on the conference website that summarized our approach to peer-review matching: https:...
[ACL 2023] Generative AI Policy
This blog post (on the conference website) summarized our approach to the use of generative AI in ACL conference submissions and reviewing: https://2023.aclw...
Field Notes on Hybrid Conferences (EMNLP 2021)
Field notes from EMNLP 2021, the first hybrid *ACL conference.
ethics
AI ‘News’ Content Farms Are Easy to Make and Hard to Detect
AI ‘news’ produced by LLM-powered content farms are flooding the web. They are currently very cheap and easy to produce, even beyond English, and currently w...
I am joining ACL Rolling Review
It’s official: I joined the ACL Rolling Review team as an editor-in-chief, and I’d like to share some brief thoughts on this.
Closed AI Models Make Bad Baselines
Will GPT-4 become a universally expected baseline in NLP research, like BERT in its time? Basic scientific methodology demands otherwise.
The attribution problem with generative AI
Some argue that any publicly available text/art data is fair game for commercial models because human text/art also has sources. But unlike models, we know w...
society
AI ‘News’ Content Farms Are Easy to Make and Hard to Detect
AI ‘news’ produced by LLM-powered content farms are flooding the web. They are currently very cheap and easy to produce, even beyond English, and currently w...
Talking to people outside your echo chamber: SocNLP challenges
A post inspired by an Uber ride with a Trump supporter.
teaching
On AI-assisted writing in graduate school
What is the proper role of AI ‘assistance’ in graduate student writing? It depends on what you mean by ‘graduate’.
How to teach NLP to non-CS-majors in 2 weeks?
What I learned from organizing an introductory course on NLP for linguists at ESSLLI 2019.
debate
A Sanity Check on ‘Emergent Properties’ in Large Language Models
One of the often-repeated claims about LLMs is that they have ‘emergent properties’. Unfortunately, in most cases the speaker/writer does not clarify what th...
The attribution problem with generative AI
Some argue that any publicly available text/art data is fair game for commercial models because human text/art also has sources. But unlike models, we know w...
negative-results
On word analogies and negative results in NLP
Negative results are hard to publish, and even harder to make well-known. Even when the disproved result is something as pervasive as Mikolov’s word analogies.
review
On word analogies and negative results in NLP
Negative results are hard to publish, and even harder to make well-known. Even when the disproved result is something as pervasive as Mikolov’s word analogies.
machine-reasoning
[Text Machine Lab] Question Answering for Artificial Intelligence (QuAIL)
This is a post I wrote during my time in Text Machine Lab: https://text-machine-lab.github.io/blog/2020/quail/. It presents an English resource for machine r...
productivity
How to Record a Virtual Conference Talk
Yes, it is possible to record and edit a conference talk at home, with open-source tools in reasonable time.