Writing a Research Paper in the Age of AI

In the late spring of 2026, I suddenly found myself with some spare time on my hands, so I decided to take an idea that I had had in the back of my mind for a while and turn it into an academic research paper. In our current age, this of course involved a lot of AI, in my case, primarily Claude Code with Opus. This post collects some reflections on this experience, partly as a snapshot of this moment in the history of the technology and partly just to get them out of my head.

(A couple of caveats up front: I am not a professional academic, it's an occasional sideline. My experience might be quite different from someone doing this regularly. And the paper has only just been submitted, so who knows if this will all turn out to have been worthwhile.)

Overall, the AI added a lot of value. It let me get the paper done in about two months at about half-time effort, with a level of quality that would have taken probably 5x longer without it. The tools still require a lot of oversight and skepticism, but on balance they're still very helpful. Who knows if the paper will be accepted, but I ended up with a work product that I'm willing to stand behind.

What am I even going to write about?

Research ideas often start out pretty vague, but a good paper is focused. My idea was even more challenging, because the idea was to combine Thing A which I know really well with with Thing B that I don't really know well at all. I'm now pretty sold on the utility of LLMs for coming up to speed in a new area.

I spent a lot of time having claude.ai do literature review: Summarizing the state of the art in various domains and basically acting as an oracle to answer my questions about specific papers, as a prelude and complement to reading the paper. Even understanding how terminology maps across subfields can be difficult. With the LLM, I could say, "The thing I'm thinking about has this shape..." and it could say "ah, in the literature it's called this", and provide citations.

And citations are important. Hallucination and fixation are a problem. Pretty often, Claude would read the title and abstract of a paper and guess at answers to questions based on that. Once we had discussed a paper, further questions would stay in its orbit, even if there were more relevant things elsewhere in the literature. As I became aware of these tendencies, they could be overcome with some nudging and clearing context, but that still added overhead.

As I learned enough to be able to focus the research idea, back-and-forth with the AI helped narrow things down. Here's an abstract, does this seem novel? How does it overlap with what exists? How should we adapt it? Not getting whole big ideas from the AI, but using the context it had built up through the discussion to refine my direction.

Let's make an experiment

A big part of this paper was implementing the ideas in software to see how it performed. Naturally, vibe coding. I needed to implement my new thing as well as a few comparison points from other papers, and I have had good experiences using AI coding tools for turning clear specifications into reference implementations. In the end, I ended up generating about 4,000 lines of code, of which around 2,000 were implementing things from the literature, 1,000 lines were my new stuff, and 1,000 lines were the code to measure everything.

Things got up and running smoothly. We got initial results; we ran into challenges scaling up; we discussed and implemented optimizations to resolve the scaling challenges. What was interesting was that in addition to code optimizations (better memory models, etc.), we could also discuss what I'll call "interpretation optimizations": Ways that we could adapt the algorithms from the literature in ways that would work better in our scenario, while still being faithful to the original definition. Modern agents are also very good debugging, especially finding root causes for things, and mostly good at identifying ways to address them.

As usual, the problem is review. Here, in addition to "is this good code?", there were critical questions of fidelity, "is this code actually implementing the algorithm?" For my algorithms, it was mainly a matter of cleaning up the code to be clearer and more understandable. For the algorithms from the literature, I had to build confidence that the code was a faithful representation of an algorithm that I was not all that familiar with. Ultimately this came down to my review and a judgement call, but as a warm up, more AI: I had independent agents review the code and the corresponding paper and make an assessment about alignment. In addition to being good prep for my review, it improved the section of the paper where I described the experiment, because I could be precise about how we interpreted the literature.

My main takeaway is that while AI is good at making functional code, it is not good at making clear and understandable code. Ideally, someone should be able to look at this code and say "Yes, this correctly implements what's in this paper". Giving that goal to the AI mainly resulted in a bunch of comments on code that didn't structurally align, as opposed to a deeper refactoring.

When it came time to actually start understanding data, Claude was exceptionally useful. Rather than learning matplotlib incantations, I could ask for the data to be presented in a certain way and get an SVG pretty instantly — and then iterate to get a usable figure or ask for an explanation of a seeming anomaly. Looking at data also led me to learn about some unfortunate choices that Claude had made in the coding process, which required a few correct-and-collect-new-data cycles.

Writing the actual paper

I despise AI-written prose, especially now that I have read a lot of it and understand its load-bearing seams. So I decided early on that I would write the actual prose of the paper.

But writing a research paper is not just about writing words, one must also be a master of 1980s arcana such as LaTeX and BibTeX. Magic incantations abound. Here as elsewhere, AI let me skip a bunch of mucking around in documentation. Beyond just "How do I do X in LaTeX?", things like "Turn this screenshot into a figure that can go in the paper" works remarkably well.

"Please go through and replace all of my \TODO{cite Maxwell and Ding} notes with real citations" also worked pretty well. Between my giving enough detail in the notes and context clues, the resulting citations were pretty much correct. I was determined that this would not be one of those papers with hallucinated citations, though, so one of my last preflight tasks was to check every single citation. And indeed I did find inaccuracies. Mostly just differences in how journals were described, but in one case, completely hallucinated authors. Yikes!

The Murder Board

I have historically not been great at anticipating reviewers' objections to my work, and this case was especially fraught because I was deliberately writing about stuff where I was thin on expertise. So I convened a committee.

I had Claude make a committee of five review agents with varied expertise. Some were experts in Thing A, either in theory or in deployment; some were experts in Thing B and totally ignorant of Thing A. The reviewers were provided with the paper and the conference acceptance criteria, and asked to provide their view on the strengths and deficiencies of the paper, as well as an ultimate recommendation to accept or reject.

The committee was not unanimous! The best I got was Accept with Major Revisions. The worst was Weak Reject, from the reviewer with expertise most different from mine. Ultimately, the reviews identified several things that I agreed required rework, some of which substantially improved the paper. Since I could call on the committee any time, I asked them to re-review after I had updated the paper. It was enough to get all of them to at least Accept with Major Revisions, and some better. For good measure, I ran it quickly past Gemini and ChatGPT as well, and got a similar vaguely positive vibe. We'll see if the humans agree.

From a technology point of view, this highlights the importance of fresh context. I deliberately isolated this committee from all the previous discussions (modulo anything Claude leaks between sessions behind the scenes). You could tell in the reviewers' feedback that there were some common ideas, but different reviewers did pick up on different things, and the things they picked up on were things that had not come up in earlier conversations. When you're dealing with a stochastic system, a fresh roll of the dice can be effective.

Would I do it again?

Definitely. For all the frustrations above — hallucinations, fixations, misinterpretations, backtracking and redoing — I could not have created something of similar quality in a similar time frame without these tools.

The frustrations are not without antecedent. Who hasn't had the experience of a miscommunication with a collaborator resulting in a bunch of work that had to be re-done? Or gotten feedback on an early draft that required a lot of rethinking? The dynamics with agents are similar, just faster, and you can enlist more (though less useful) "collaborators".

The thing I appreciate most, here as with AI coding, is the ability to operate at a higher level. Someone once told me that better musicians experience music in bigger units, phrases, lines, or movements instead of individual notes. With these tools, I can articulate the important things, the overall shape of the line, and the machine has enough understanding to see where I'm going and work out the rest.