Writing is thinking, but what happens when AI does it for you?

An exploration of what AI is doing to our writing, our brains, and our voices.

​​”Nowadays, the hand-made article hasn’t a hope. It can’t possibly compete with mass-production. Carpets…chairs…shoes…bricks…crockery…anything you like to mention— they’re all made by machinery now. The quality may be inferior, but that doesn’t matter. It’s the cost of production that counts. And stories—well—they’re just another product, like carpets and chairs, and no one cares how you produce them so long as you deliver the goods. We’ll sell them wholesale, Mr Bohlen! We’ll undercut every writer in the country! We’ll corner the market!”

You might think this quote is from some hoodie-wearing, disruption-obsessed tech founder convincing their investors that they can ‘scale’ storytelling.

It’s not. It’s actually from Roald Dahl’s 1953 short story The Great Automatic Grammatizator.

The story goes like this. Adolph Knipe is an engineer who secretly wants to be a writer. He has written hundreds of stories, but nobody will publish them. So instead of giving up, he builds a machine that writes stories. His machine could write a five-thousand-word story in thirty seconds. A novel in fifteen minutes.

You press a button for the genre, another for the theme, another for the literary style, and out it comes with perfect grammar and structure. The stories and novels are good enough. Good enough to sell.

It might sound eerily familiar, with AI tools being able to generate our essays and stories from a single prompt in seconds. But Dahl, in 1953, wrote it as satire.

—

A few years back, when AI had just started going mainstream, I remember having a conversation with a colleague. We were talking about how relying on AI for our everyday writing, like emails and social captions, might be crippling our thinking. How we were already finding it harder to form even simple sentences without AI’s help.

But as two naive AI enthusiasts, we quickly talked ourselves out of it. Maybe this was a good thing. If AI was taking care of the routine writing, surely that freed up our minds for deeper, more strategic work.

And maybe that’s true for functional writing. But what about the other kinds of writing? The kind where you’re building a world, crafting characters, writing a story or a novel. Or an essay like this one where writing IS thinking, where the writer is figuring out what he believes as he writes. What actually happens to our brains when we let AI do the heavy lifting in our writing?

‘Your Brain on ChatGPT’

Researchers at MIT decided to find it out. They split 54 participants into three groups. One group wrote essays using ChatGPT. Another used a search engine. The third used nothing at all, just their brain. Throughout the process, the researchers monitored brain activity using EEG.

The results were quite uncomfortable. The group that wrote with ChatGPT showed the weakest brain connectivity of all three. The brain-only group showed the strongest, most widespread neural activity, especially in areas associated with creativity, memory, and deep thinking. The ChatGPT group showed significantly less of all of that.

When asked to quote from the essay they had just written, 83% of the ChatGPT group couldn’t do it. Their brain was barely involved in writing it in the first place. The essay existed. They just had no relationship with it.

The researchers called this cognitive debt. The more you rely on AI to do your thinking, the less capable your brain becomes of doing it on its own. Like a muscle you’ve stopped using.

A caveat worth mentioning. This study is still a preprint, meaning it hasn’t been peer reviewed yet. And with only 54 participants, mostly young American adults, it’s preliminary. But the direction it points in is uncomfortable enough to take seriously.

We’re All Starting to Sound the Same

But it’s not just what AI does to your individual brain. It’s what it does to all of us, collectively. (No, the irony is not lost on me – “it’s not X, it’s Y” is a very AI sentence structure).

Researchers at Cornell gave two groups of people, one American and one Indian, simple writing prompts. What’s your favourite food? What’s your favourite holiday? And so on. One subset used a ChatGPT-powered autocomplete tool while writing. The other wrote unaided.

The writing of the Indian and American participants who used AI became more similar to each other, gravitating towards “Western norms”. The AI users were most likely to say their favourite food was pizza and their favourite holiday was Christmas.

What’s more, the specific details disappeared. The nutmeg and lemon pickle in biryani became just “rich flavors and spices.”

This is how AI works. It’s trained on everything humans have written, and it produces the most statistically probable version of whatever you ask for. Not the most interesting or original. The most average. And average, as Vauhini Vara, journalist and author, pointed out, is not as innocent as it sounds.

According to her, the mediocrity of AI text gives it an illusion of safety and harmlessness. But what is actually happening, she says, is a reinforcing of “cultural hegemony” – the values, tastes, and worldview of those who build these systems. AI companies have every incentive to keep things acceptable, because the more people find the output agreeable, the more subscribers they get.

Averageness is efficient. And when we write with AI, we drift towards that average too.

—

This is where I’ll stop for now. These aren’t my complete thoughts on the matter. It would be hypocritical of me to leave it here without telling you that I have Claude open in another tab as I write this. What that means for me as a writer is something I’ve been thinking about for a long time.

In Part 2, I’ll talk about how I use AI to help me write better, how I don’t outsource my thinking to it, and how this piece, despite having Claude open the whole time, took me more hours than any other edition I’ve written. I don’t have all the answers. I’m still figuring this out, and honestly, my opinions on it might look very different a year from now.