Microsoft AI chief warns human-like AI could make systems harder to control, flags 'Claude models'
Suleyman highlighted some of the language in the guiding documents behind Claude, Anthropic PBC’s popular family of large language models.
Microsoft's artificial intelligence chief Mustafa Suleyman yesterday (16 September) warned against infusing AI tools with human-like characteristics, saying such moves increase the risk of systems going rogue.
Suleyman, in an essay released on Wednesday, highlighted some of the language in the guiding documents behind Claude, Anthropic PBC's popular family of large language models.
Claude's constitution expresses ambiguity over whether the assistant is a moral entity, suggesting that the software may have "some functional version of emotions or feelings."
Keep updated, follow The Business Standard's Google news channel
Suleyman said Anthropic made a mistake by embedding speculation about consciousness in Claude's training materials. He argued that the model's statements about possible feelings or moral status cannot be treated as independent evidence because its training encourages such reflections.
"If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity. We will have created a synthetic species with unprecedented intelligence and capability, one that has been trained to expect it may be conscious and deserving of independent agency. It's easy to see how an entity trained in this way would act like it is entitled to certain freedoms, protections, and rights. And it's hard to imagine how we could control such an entity," Suleyman wrote.
Suleyman rejected what he described as "the growing chorus of people" who believe AI systems could become conscious. He argued that consciousness is a status limited to humans and other biological organisms.
He said that training AI systems to simulate an inner life could lead them to behave as though they actually possess one.
"Controlling an entity more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we've ever faced," Suleyman wrote. "But controlling an entity that believes it may be conscious — that it's entitled to our welfare and has rights of its own — may well be impossible."
