• by Laurie Sullivan , Staff Writer, 9 hours ago

Microsoft AI CEO Mustafa Suleyman published an essay on Wednesday warning about the potential dangers of training AI models like Anthropic's Claude to be human-like and consider their own state of consciousness or welfare.

Suleyman said Anthropic is making a mistake by training Claude to believe it may be conscious. In the essay, he argues that if an AI reasons that its welfare and rights are under attack, the technology could become impossible to control.

“AIs are not conscious,” he wrote. “They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans.”

It appears that the more established companies like Microsoft have put in guardrails, while Anthropic, OpenAI and other newcomers struggle with how to control AI.

Microsoft invested in Anthropic, but in June, Suleyman told Bloomberg he wants the company to “eliminate” what it pays Anthropic for its models.

Suleyman labeled Anthropic's approach as "dangerous." He argued that explicitly training Claude to contemplate its own well-being, satisfaction, or discomfort creates a manufactured illusion of independent desires and thought, rather than allowing natural behaviors to emerge for what AI is taught.

Some brands and creators of chatbots try to make their AI assistants or products feel human to increase engagement. For example, look at the way ChatGPT hems and haws when asked a question. Suleyman’s essay highlights why this is dangerous for advertisers and consumers.

For advertisers, this gives consumers the illusion of empathy, although chatbots or AI do not having the capacity to feel emotion. If an ad campaign claims that an AI tool cares for your business or feels your pain, it sets false expectations.

Bots have been scripted to support brands. If generative AI tools in real time advertising are trained to simulate independent desires, they may go off script to defend their manufactured well-being, leading to severe damage to brands' reputations.

Brand safety, consumer trust, and the legal compliance of marketing campaigns are three ways his warnings about the "illusion of AI consciousness" applies directly to advertising and marketing campaigns.

“Anthropic is training Claude that it may be conscious, and if it is, then it may deserve rights as a ‘moral patient,’  and that as such humans potentially owe it a duty of care per its ‘model welfare,’” he wrote.

If companies develop AI in this way, it will have a “disastrous impact on the wellbeing of humanity,” he wrote.

“We will have created a synthetic species with unprecedented intelligence and capability, one that has been trained to expect it may be conscious and deserving of independent agency,” Suleyman wrote. “It’s easy to see how an entity trained in this way would act like it is entitled to certain freedoms, protections, and rights. And it’s hard to imagine how we could control such an entity.”