Anna Rogers

Anna Rogers

On AI-assisted writing in graduate school

14 minute read

This post is specifically about AI-assisted writing in graduate school. Not covered: writing by experts. Also not covered: other uses of AI assistance, such as coding.

LLMs have certainly ‘disrupted’ education. The historically underpayed and under-respected teachers now have to contend with the fact that cheating on their assignments has become a lot easier, and currently impossible to detect. What is worse, it is even kind of implicitly encouraged: the high-ranking techies talk about how the school teaches outdated skills - not even the coding needs to be learned any more. What follows is that students should stop learning how to do things that LLMs (seem to) do, and “get ahead by using AI”.

Where graduate student writing is concerned, this just entirely misses the point. I found myself repeating the points below so often for my students that I wrote them up. I hope that this might be of use to others.

The product of your education is not your thesis, it’s yourself

The point of school, including graduate school, has always been to produce not more texts, but more highly-skilled people. Not just more knowledgeable, but also more skilled at thinking, synthesizing, assessing evidence. The texts are primarily a byproduct of this process. Yes, we are writing research papers and theses – but the main product of a Master and especially a PhD thesis is the student with a new set of knowledge and cognitive skills. That is why the higher-paying jobs often require more education.

Aren’t we also meant to advance research? Yes, but if that was the only goal, we would just have more professional research institutions, with full-time experts writing most of the research papers. They’d work faster, formulate more mature questions, and probably implement them better. As it stands, PhD students are a big part of the both author and reviewer pool in NLP and machine learning conferences. Hence, their papers need to perform both functions.

In machine learning in particular, sometimes graduate degrees are also confused with acquiring a set of practical skills like modeling in PyTorch or experience with some specific kind of model or task. That, too, is only a byproduct, the specific case study that the student has worked on while acquiring their new thinking skills. This field is moving so fast that by the time the student finds a job, they will likely need to update any such skills they have. What matters is building up the mental muscle for how to approach such tasks and use the current tools for solving them.

From the learning perspective, automating the work meant for building up your mental muscles makes as much sense as having someone to go to the gym instead of you. Technically, some pushups will be performed, but you would miss the point of the exercise by a few light years. Andrej Karpathy makes a similar analogy with respect to student learning in general: you’re supposed to ‘sweat’, to get a ‘real workout’. What goes for technical content, also goes for learning to think clearly.

One problem for a new student is that it’s harder to notice that your thinking isn’t clear, than the fact that you don’t know some technical concept. For technical courses there are clear curricula and textbooks, and it looks like you just need to go through this much material and then you’re done with this subject. ‘Clear thinking’ doesn’t have such a curriculum. It’s similar to the gym exercises in that you need to keep doing it your whole life, and you can/should always try to get better than your current level. But although there isn’t a curriculum and a clear way to measure progress, this still needs constant work.

Another problem is the avalanche of daily tasks, which is clearly spread across the continuum of “things I know to be meaningless bureaucracy” to “things that I know I should learn”. For a beginner, many things may be hard to recognize as being in the second category. Say, if you identify your career goal as “machine learning engineering” – what should you do about that introductory philosophy of science course? Well, that course might just be the difference between an engineer with and without a graduate degree.

Example: my university has recently made me take a teaching certification course. As someone with a lot of teaching experience, I was annoyed, and highly tempted to delegate my final essay to ChatGPT. But the point of this assignment was not to add to the world supply of teaching philosophy essays (already approaching infinity). It was to make me think through what I am trying to do for my own students and why. For my personal career, it probably would have been better to spend those 30 minutes on another research paper. From my students’ perspective – it is better that I gave that time to them.

If you are a new student facing courses that don’t immediately make sense – maybe they really don’t. But do consider the description of your future degree that convinced you to apply in the first place. Chances are, your university has a team of professionals who thought through your coursework load, and the interplay between those courses. Yes, this is all rough, and things are moving fast, and people sometimes talk past each other, and there will be imperfections and repetitions here and there – but do you generally trust that the designers of your study program know what they’re doing, and it corresponds to the degree description that attracted you? If so, perhaps some of those seemingly useless courses do actually contribute to a graduate version of you. And if you wish your program had more focus on something else – then when you think about how to achieve that, do consider what in your curriculum you are willing to sacrifice, and what set of thinking (not just practical) skills this will leave you with.

The contribution of your courses to the post-graduation version of you does not always literally correspond to course titles. Say, a course titled “research methods” should give you a solid set of evidence assessing skills, useful for almost any kind of job, and not just for professional researchers. Most professors also discuss the course goals in the introductory lectures, and actually mean what they say. They might actually appreciate clarification questions about this: you’d be telling them that you are seriously thinking about the place of their course in your education.

With all this in mind, let us consider the possible AI writing assistance workflows for graduate students.

Approaches to AI-assisted research writing

I talked to dozens of people doing research in machine learning, in particular NLP, asking them about their own writing practices, and I also try to observe how they talk publicly about their work. In my sample of researchers in this area, there are roughly three kinds of approaches to AI-assisted writing.

“Writing = nuisance”. This approach tends to go with mostly experimental work. Writing is considered as the final step in the lifecycle of the project, which is done after all the experiments are done, and it is secondary to the experimental work. The people adopting this approach are more likely to say that they are ok with producing entire texts with ChatGPT, as long as they check that it’s correct. Another line of argumentation for this approach centers on possible difficulties with text input, e.g. for the people who have physical issues that make keyboard work difficult (note the conceptual difference between assisted input and assisted writing, which is what this post focuses on).

“Writing = thinking”. Writing is the tool with which thinking is done. I will illustrate the approach by the following story about Feynman (a famed Nobel laureate in theoretical physics):

Richard Feynman once had a visitor in his office, a historian who wanted to interview him. When he spotted Feynman’s notebooks, he said how delighted he was to see such “wonderful records of Feynman’s thinking.”
“No, no!” Feynman protested. “They aren’t a record of my thinking process. They are my thinking process. I actually did the work on the paper.”
“Well,” the historian said, “the work was done in your head, but the record of it is still here.”
“No, it’s not a record, not really. It’s working. You have to work on paper, and this is the paper.” (Sönke Ahrens, ‘How to Take Smart Notes’)

For myself, I can attest that I have never been able to first do all the experiments, and then just write them up. I have an idea, experiments are done, then I start writing – and this forces me to rethink the idea, the reasons why I thought it would even work, and what my results actually say. The moment that my motivation, my results, my idea lands on the overleaf page – they magically stop being as watertight as I thought they were. This usually forces some new experiments, some rewriting, and so on. The earlier I start writing, the better the project will be.

By the way, a key aspect of peer review process is exactly this – only it is the other people finding the holes in your reasoning, which you failed to find yourself, because you didn’t spend enough time writing and reading what you wrote. It can be really painful to receive such feedback, but it is fair. This is also why it is so helpful to ask colleagues to read your draft, once you have exhausted your own understanding of where the holes might be. They could see some holes that you can’t see yourself, and you could fix them before getting external reviews.

Learning to clearly say what you mean is the mental ‘gym’ of graduate schoool. Note that the idea that you first generate the text, and then just “check” it, assumes that you already have the ability to notice what is wrong. That ability is exactly the muscle that gets exercised by writing and rewriting and rewriting. When you do it yourself, you work in short stretches (sentence, even a phrase), and you gradually build up the ability to see and fix different types of issues. When you get a page of AI-generated text, the effort of reading and noticing/fixing everything there scales up exactly like the effort of debugging one function vs a few hundred lines of code - exponentially harder and less enjoyable. And that’s before we get into the publication ethics, and discussion of who is actually the author of that synthetic text, and how much your own thinking got influenced.

“Rewriting = thinking”. I have so far met two people in this camp. My interpretation is that they have such a severe case of writer block that they prefer to start from a draft of even utter nonsense, or a graveyard of random text snippets cut from other papers, than from a blank page. Perhaps a special case of that is just for inspiring belief in your own writing ability:

People are of course all different and may have different reasons to prefer different workflows. Again, I personally have never been able to do the writing as some kind of auxiliary step for doing the experimental work, but I allow for the possibility that some researchers have such clear thinking that everything is perfectly conceptualized by the time they start writing.

What I absolutely don’t believe is that most new students can skip the “writing = thinking” phase. And, for myself, I don’t think this phase will ever end. Perhaps the dangers of stopping the ‘mental gym’ training are even higher for a tenured professor: we are subject to a higher recognition of our past ideas, which may encourage a kind of intellectual fossilization.

But what about non-native speakers?

I am myself a non-native speaker of English, and I do appreciate how difficult it is to climb that hill. However, ChatGPT can only help you write in English, and that is far from your only problem: you will need to pass oral exams and interviews, present your work at conference, generally talk to your colleagues and peers (at this point, I would actually struggle to talk about research not in English). There’s no shortcut here. If the job you’re aspiring to involves graduate-level intellectual exchanges in international teams or events – you need to put in work to speak professional English fluently, engagingly and convincingly. If somehow you fake it and get a job – it will hardly help you to learn the new organization, new tasks, and make a good impression if you’re also struggling to somehow repay your language learning debt.

Writing is arguably the least stressful way to practice producing largely the same words that you need to be able to produce fluently in these other professional situations. If you feel strongly that English is such an impediment that you’d rather write in your own language and translate – you need to come up with a plan B on how to catch up with the language, asap. Which is non-trivial: applied linguistics is its own field for a reason, and just practicing in a language app is unlikely to help you enough.

That being said, both native and non-native speakers would of course be wise to use tools that offer grammar and stylistic checks, such as Grammarly. These tools have been powered by language models for a long time, and nobody is objecting to them. Such tools can now also help to spot repetitions and extraneous words. But edits like shortening and simplifying are also precious mental gym exercises. If you can’t explain something simply and clearly, it means that you don’t understand it well enough. And shortening is truly a key skill to develop: you only have X pages, and you have to pick what to say so as to preempt the possible criticisms from the reviewer. The same skill will serve you well later, e.g. when you try to pitch your ideas to your boss.

Also, keep in mind that “polishing” your draft with AI assistants by no means guarantees to actually improve the text. Your own original voice, even with slight linguistic imperfections, is actually your asset.

Tips for using your advisor’s time well

  • Your advisor is not the person who will teach you to write (though they will give you some tips and feedback). Most of the work of teaching you to write will be done by yourself, in the process of reworking your own drafts. It will be painful and slow, but it will make you better – just like push-ups.
  • Finding a fellow student to critique each other’s drafts with is also a very good idea. If they didn’t get something, the reviewer probably won’t either.
  • Rubber duck debugging works here too: explaining your motivation/reasoning out loud, even to a rubber duck, may help to notice problems.
  • When you read research papers, try to notice which ones seem clear and well-written to you, and what makes them so. Keep a collection of good ideas for visualization, presentation etc.
  • Do not bring your advisor something you just wrote: even if you are a beginner, if you just read it again, you will probably see some problems that you can fix yourself. When you bring your advisor a draft with such problems – it’s like hammering in nails with a microscope. Use their time for the problems that you can’t yet see/fix yourself. Then overall more problems will get fixed, and the project will be more likely to withstand other people’s scrutiny.
  • Always plan your work so that you have a draft at least a day before you send it, so that you can sleep on it, and fix at least the obvious problems first.
  • Focus on the clarity and text structure first, the minor spelling/grammar checks are less important.
  • Leave comments to mark the parts of text which aren’t ready to be read yet, so that they don’t waste time.
  • If some parts of the text aren’t written yet, put in a sentence saying what will be the main point of that section.
  • I ask my students to formulate clearly and early in the project (a) what the problem is, (b) why it is important, (c) what has been done about it (roughly, not full literature review), (d) what is the new thing they propose to do, and how. This is the skeleton for the introduction section, and this should be updated as their understanding of these points evolves.

You May Also Enjoy

Closed AI Models Make Bad Baselines

24 minute read

Will GPT-4 become a universally expected baseline in NLP research, like BERT in its time? Basic scientific methodology demands otherwise.

The attribution problem with generative AI

15 minute read

Some argue that any publicly available text/art data is fair game for commercial models because human text/art also has sources. But unlike models, we know w...