• by Steven Rosenbaum , Featured Contributor, 3 hours ago

For the past five months, I've been in a kind of self-imposed exile. My book “The Future of Truth: How AI Reshapes Reality” had just been published when The New York Times found a handful of quotations that didn't hold up. I had used AI extensively while researching and writing the book, something I had disclosed, and suddenly I found myself at the center of a very public argument about AI, authorship, hallucinations and trust. For a couple of weeks, I got pummeled.

I could have tried to fight every story, every tweet, every accusation. But I was badly outgunned and, frankly, exhausted. Rather than keep swinging, I decided to shut up and go back to work. I spent the summer taking the book apart. I hired smart fact checkers and proofreaders. We went back through the sourcing and citations. I corrected what needed correcting, wrote a new author's note explaining what happened, and added a chapter about AI hallucinations and what I had learned from becoming one of its more public casualties.

The new edition comes out tomorrow, and after months of mostly keeping my head down, I decided it was time to talk.

One of the people I talked to was Will Oremus of The Atlantic.

I like Oremus. He's a good writer, a careful reporter, and he's been thinking about technology for a long time. His new piece has a wonderfully provocative headline: "Don't Write With AI, Say People Who Write With AI."

I'm one of the people in it.

Except there's a small problem. I've never told people not to write with AI. Oremus says so himself: "Unlike some of the others, Rosenbaum disclosed his use of AI in the writing and editing process, and he has never tried to tell other people not to use it.” 

(Thank you, Will.)

What I did tell him was that AI is "incredibly seductive." I told him I had begun using it because I wanted to understand the technology I was writing about, and gradually it became a writing partner. It made writing easier and, perhaps more important, less lonely. Then I said something  I suspect will be the central question writers are going to wrestle with for a long time: "The line between thinking and researching and reading and outlining and drafting and writing is really blurry."

Oremus doesn't entirely agree. He writes that The Atlantic has a strict policy against AI-generated prose, and he has his own bright line. He'll ask an AI questions. He'll ask it about a word choice. He'll ask it to poke holes in an argument. But he won't ask it to write for him, and he won't put AI-generated prose into his drafts.

That's a perfectly reasonable rule. But then something interesting happens.

While writing the article about people using AI to write about AI, Oremus asks Claude for help. (You knew this was coming.) He tells Claude the thrust of his argument and asks for examples of AI-writing hypocrisy he might have missed. Neither Google nor Claude turns up anything he didn't already know. So Oremus asks it another question: Why might AI skeptics fail to follow their own advice?

Claude comes back with a formulation Oremus likes: "Writing is thinking is a rule for other people." He doesn't copy the sentence. But he acknowledges something more interesting. Claude's formulation, he writes, "informed how I thought about this story's framing."

And then Oremus asks the obvious question: "Am I an AI-writing hypocrite now too?"

I don't think he is. I think he's discovering the problem.

Where exactly did the writing happen? Was it when Claude generated the sentence, which Oremus didn't use? Was it when he  read the sentence and thought, yes, that's closer to what I've been trying to say? Was it when that thought changed how he framed the article? Or was it only later, when Oremus put his own words on the page?

This isn't a gotcha. Quite the opposite. I think Oremus' disclosure is one of the most interesting parts of the article because he's doing exactly what writers ought to be doing right now: experimenting, setting boundaries, disclosing what he did, and then questioning whether his own boundaries make sense.

And The Atlantic is an especially interesting place for this experiment.

Its CEO, Nicholas Thompson, doesn't run editorial. Jeffrey Goldberg does, and Thompson has been quite explicit about that divide. But Thompson has said that he and Goldberg agree on a remarkably bright line: AI should never be used to write "a single word of a single story."

On the business side, where Thompson does have authority, the approach is very different. He wants The Atlantic experimenting aggressively with AI. The company has a licensing and product partnership with OpenAI. Thompson has said that people searching with AI models will become one of the fundamental ways they navigate the web and that publishers need to participate in shaping that future.

Fair enough. But here's where it gets even more interesting.

Thompson is an author, too.

He recently wrote a memoir, "The Running Ground," and he has been open about using AI during the process. He  described uploading sections of his manuscript to AI to look for repetition. He used it to identify contradictions, help him understand complicated processes and serve as what he called an "editor thought partner." He draws essentially the same distinction Will does: AI helped him with the book, but AI didn't write the book.

So now we have three writers standing in roughly the same room, trying to draw the line in slightly different places.

Oremus uses AI for research, questions, word choices and to challenge his thinking. Claude helps sharpen an idea that influences the framing of his story, but Oremus writes the words.

Thompson uses AI to question his manuscript, identify problems, and act as an editorial thought partner, but Thompson writes the words.

And I used AI as a research, thinking, editing and writing partner, while taking responsibility for the book that ultimately carried my name.

There are important differences among those uses. I'm not trying to erase them. I know better than anyone that the consequences can be very different. But I'm increasingly skeptical that the old binary question -- "Did you use AI to write this?" --  tells us very much.

When my book blew up this spring, much of the conversation around AI and writing was remarkably binary. Did you use AI or didn't you? Human or machine? Authentic or fake? Writing or cheating?

Now those categories look increasingly inadequate.

Writers use AI to find sources, transcribe interviews, challenge arguments, summarize documents, suggest words, organize research, outline ideas, edit sentences, find contradictions, test hypotheses and, yes, sometimes generate prose. Those things aren't all the same. Pretending they are doesn't help us establish useful rules about any of them.

And even the phrase "AI-generated prose" gets complicated quickly. Suppose I ask AI to give me 10 ways to express an idea and don't use any of them, but the eleventh version I write myself is better because I've read the 10. Did AI write it? Of course not. Did AI affect the writing? Of course it did. That's essentially what Oremus describes happening to him.

There is another distinction that I understand much better today than I did in the spring: Using AI and trusting AI are not the same thing.

I still use AI. I'm not going to pretend otherwise. It remains astonishingly useful. It is also astonishingly capable of being wrong, sometimes with such confidence and fluency that the mistake can sail right past you. I learned that lesson the expensive way.

I'm increasingly skeptical that the binary question -- "Did you use AI to write this?" --  tells us very much. 

Oremus comes back to that problem near the end of his piece. After looking at the growing list of people who understood AI's risks and still managed to get burned by them, he writes: "But the fact that even professed AI experts blunder into overrelying on the technology suggests that Rosenbaum is onto something: Chatbots might come with fine-print disclaimers that they can make mistakes, but they bend over backwards in their interactions with users to come across as honest and trustworthy."

I'll take that. I also told Oremus readers don't care how the sausage is made. I'd add one distinction. They care about truth.

But what I'm "onto" isn't an excuse for what happened. It doesn't absolve me, or anyone else who puts their name on something that turns out to be wrong. It does tell us something important about the technology. Knowing that an AI can hallucinate is not the same as being protected from its hallucinations. These machines don't behave like an unreliable source. They behave like the most confident, patient and agreeable research assistant you've ever had.

Oremus shares a wonderful analogy from a journalist who had gotten into trouble with AI-generated errors. AI, the journalist told him, is like a power tool. You can warn everyone that a table saw can cut off your fingers and still lose a finger yourself. The danger is built into the machine, and the more you use it, the greater the exposure.

I understand that analogy better than I'd like to.

The people who understand these machines and their flaws aren't necessarily walking away from them. Neither are the institutions publishing our work. We're all trying to figure out what happens when a tool stops being something sitting next to the creative process and starts becoming part of the environment in which the creative process takes place.

Five months ago, I thought I had crossed a line. Now I'm not sure there is one line.

There are many choices about when to use AI, what to use it for, when to disclose it, what needs to be checked, and where the human needs to remain firmly in control.

And whatever tools we use along the way, responsibility for those choices still belongs to us.