• by Steven Rosenbaum , Featured Contributor, Today

In just the past six months, my inbox has changed. Each morning I wake to a series of pleasant emails from Joe, Sally, or Fred, each of them "checking in" or "following up." They know my name, the work I do, something about the projects I'm involved in, and how the service they're offering could solve a problem. Unlike the spammers of years past, who offered me a piece of some inheritance or another grammatically challenged scam, these are conversational and, for a moment, believable.

Individually, they're lame. Collectively, they're exhausting. And each day, they get better at what they're trying to do.

The philosopher Daniel Dennett called them "counterfeit people," and he raised a chilling alarm. Writing in The Atlantic in May 2023, less than a year before his death, Dennett argued that counterfeit people are "the most dangerous artifacts in human history," capable of destroying not just economies but human freedom itself. He called for outlawing both their creation and their distribution, with penalties severe enough to match the stakes.

He wasn't alone. According to Dennett, the philosopher and historian Yuval Noah Harari, writing in The Economist that same spring, ended his own warning about AI's imminent threat to human civilization with a single unsettling line: "This text has been generated by a human. Or has it?"

Both suggested these fakes must be called out by law, and that the risks were severe. I agree.

What made Dennett's warning different from the usual AI doomsaying was its precision. He wasn't worried about superintelligence enslaving humanity. In an interview with the “Machine Learning Street Talk” podcast just weeks after the Atlantic piece ran, he put it plainly: The real danger is that we're going to let much "stupider systems beguile us and manipulate us," degrading the entire world of human communication in the process. Not AGI. Not robot overlords. Joe, Sally, and Fred.

Dennett saw two ways the trend would break us: Either we'd grow paranoid, probing everyone we meet online with gotcha questions to prove they're real, or we'd simply tune each other out and lose the very connectivity the internet gave us. His slogan for the alternative was simple. We should be building smart machines, he argued, not artificial colleagues.

He never softened the message. In one of his last extended interviews, with Tufts, where he taught for five decades, Dennett said we face "a pandemic of fake people" that could destroy human trust, and with it, civilization. He admitted he was an alarmist, then added that there was every cause for alarm. He told anyone who would listen that if they could show him a flaw in his argument, he would be grateful. No one ever did.

The term has traveled since then. The night before cognitive scientist Gary Marcus testified to the U.S. Senate in May 2023, Dennett sent him the manuscript of the counterfeit people essay, days before it ran in The Atlantic.

Marcus has carried the argument forward ever since. This February, he renewed the call in the starkest terms yet, writing in a Substack that we urgently need a federal law forbidding AI from impersonating humans, and pointing to video tools that can now fake anyone's appearance, at any time, “for almost nothing.”

Here in the United States, the legal response has been piecemeal at best. My home state of New York passed the nation's first AI companion law, effective last November, requiring chatbots that simulate human relationships to remind users every three hours that they are not talking to a person, with penalties of up to $15,000 a day.

Maine now requires businesses to disclose when a consumer is chatting with a bot. California's companion chatbot law followed in January.

But look closer, and the gaps are glaring. New York's law explicitly exempts chatbots used for ordinary customer service and sales. So the AI girlfriend must confess she is software, while Joe, Sally, and Fred remain free to "follow up" forever. And there is still no federal law that says what Dennett said plainly: A machine that passes itself off as a person is a counterfeit, and counterfeiting is a crime.

And it's not just scammers. Brands, phone companies, and marketers are all deploying counterfeit people to "chat" with you, offer services, and "follow up." The line between criminals sending fake invoices and threatening emails, and marketers using "tools" to reach unsuspecting humans, is harder and harder to see. And with AI voice and video now virtually undetectable from real humans, the danger is far greater.

Dennett saw where this was headed. His proposed remedy borrowed from the world of currency: a mandatory, technically robust watermark for AI content, modeled on the anti-counterfeiting patterns embedded in the world's banknotes, backed by strict liability for companies whose products do the impersonating. Not negligence liability, but strict liability. If your AI passes itself off as a person and someone gets hurt, you pay.

Three years later, we have neither the watermarks nor liability. What we have instead is Joe, Sally, and Fred, multiplying in my inbox every morning, each one a small rehearsal for the larger deception to come.

Here is Dennett's closing warning, from his Atlantic article. It reads even more urgently now than it did in 2023: “The moment has arrived to insist on making anybody who even thinks of counterfeiting people feel ashamed—and duly deterred from committing such an antisocial act of vandalism. If we spread the word now that such acts will be against the law as soon as we can arrange it, people will have no excuse for persisting in their activities. Many in the AI community these days are so eager to explore their new powers that they have lost track of their moral obligations. We should remind them, as rudely as is necessary, that they are risking the future freedom of their loved ones, and of all the rest of us." 

Dennett died in April 2024. He did not live to see how quickly his prediction became a business model. But you can still hear him make the case in his own voice, patient, precise, and unmistakably worried, in his “Machine Learning Street Talk” interview on YouTube. It is worth an hour of your time.

The counterfeiters are not hiding in the shadows of the internet. They are in your inbox, on your phone, and increasingly on your screen, and they would very much like to follow up.