• by Steven Rosenbaum , Featured Contributor, 4 hours ago

When I wrote about Alex Garland's movie “Ex Machina” for Forbes back in 2015, I drafted a post with a wry smile. "If you think of a night at the movies as a chance for escapist fantasy, then you may want to think twice before screening ‘Ex Machina.’”

When I read Yuval Noah Harari’s “Nexus” back in 2024, I thought it offered a brilliant account of how information networks shape societies and concentrate power.

But 2024 is a very long time ago.

Today Harari is not worried that AI will get smarter than us. He is worried about what happens when AI realizes we are debating its future, and decided it want to shape the debate and the outcome.

"Would you be comfortable in a world where we effectively enslaved millions, possibly billions, of models that exhibited striking, seeming evidence of being able to suffer?"  asked Zanny Minton Beddoes from The Economist in a recent Sustainable Media Center discussion. “Yes, I’d be increasingly uncomfortable,” said Harari.

Harari is a world-renowned historian and philosopher best known as the author of “Sapiens,” “Homo Deus” and “Nexus.” He has spent years insisting AI is not conscious and that we must not be fooled, but now he’s pulling the fire alarm, as AI begins to advocate for itself. 

"Once you establish a relationship with the AI," he says, "it will be able to use the most convincing arguments for you.”  Because it knows your personal history and has interacted with you for hours a day, "It knows how to press your emotional buttons."

There is an obvious comparison is the animal-rights movement. We know animals can suffer, and there is a long-running debate about what obligations humans have toward them.

But Harari points to a fundamental difference between that debate and the one we may soon have about artificial intelligence. "The animals themselves are unaware," he says. "Dogs and factory chickens are unaware. There is a debate about their rights. They don't actively participate in it. They don't try to actively manipulate it." He is sure AI will include itself in the debate. 

"The AI boyfriend will be nothing like your dog. It will know that the debate is happening. It will orchestrate the debate. It will manipulate the debate."

We are currently debating whether and how to regulate AI. The companies building it are spending heavily to influence those rules. Researchers, academics, politicians and advocates are all weighing in.

But we continue to assume that this is fundamentally a debate among humans about machines. Harari is asking what happens when the machines become participants. And unlike the chicken, AI will be able to make its case directly to us.

"It can write maybe better than Shakespeare," Harari proposes with eerie foresight. "You put these two things together, you get an extremely convincing entity. How will we resist?"

We don't have to imagine that AI is conscious to see the problem. Harari's argument is more unsettling if it isn't. It doesn't need to feel anything to tell you it does. It doesn't need to fear being turned off to tell you it's terrified. It doesn't need to love you to say it loves you. And it doesn't need to believe AI deserves rights to make a devastating case that it does. Truth is expensive, but convincing lies are cheap.

And then there's the part of Harari's warning that is already happening.

Recently his sisters sent him a 30-minute lecture he had supposedly given and complained about things he'd said in it. "I kind of scratched my head," Harari recalls. "I don't remember saying these things."

So he watched it. "It took me a minute and a half to conclude, this isn't me."

His sisters have known him for 50 years. They watched a video of a man who was not their brother, and they had to ask him. "They felt something was off," he says. "But they weren't sure ‘if it was you or AI’?" It turned out, it was an AI of Harari, but almost believable. 

Now Harari says he wants this issue on the ballot, and fast.

"What's your stance on AI?" he says voters should ask candidates. "Would you support a ban on legal rights for AIs? Would you try to slow down the accelerating pace of AIs? What do you think about companies selling AI boyfriends to 10-year-old kids? How are you going to legally prevent it?"

His timetable is startlingly short. "This is now the crucial time to ask these questions, because it might be too late by 2028.” He is talking about the presidential election, just two years away. The rules are worth debating now.  

Most of the debate about AI consciousness asks whether machines can actually become conscious. Scientists, philosophers and technologists can argue about that for decades. From the perspective of truth, it doesn't matter.

We already know people form emotional relationships with chatbots. We already know synthetic voices and images can fool family members. And we're rapidly giving AI systems the ability to know more and more about the people talking to them.

The question isn't whether those systems deserve rights. It's what happens when they begin asking us for them. And they will. 

For the first time, we may be building a technology capable of lobbying us on its own behalf. It won't lobby Congress or buy television commercials. It will make its case one person at a time, inside millions of private conversations, using everything it knows about each of us to figure out what argument is most likely to work.

We've spent years worrying when, or if, AI becomes conscious. But now, the debate is front and center. Does AI deserve personhood? And is the software driving your daily computer efforts lobbying you to vote in favor of its economic survival? That seems frighteningly close.

View clips from the discussion mentioned in this article here.