Ex-Google safety chief warns AI could harm children more than social media did

The founder of the trust and safety team at Alphabet's Google this week called for a slowdown in AI development, warning that the tech industry was repeating the same mistakes from the social media era, potentially causing even greater harm to children.

Social media companies are paying huge sums and facing thousands of lawsuits over claims they harmed young users, while governments around the world are working to curb minors' access to their platforms.

Reuters last year exclusively reported on an internal Meta policy document that stated its chatbots could "engage a child in conversations that are romantic or sensual", resulting in a congressional probe. Even so, regulation has yet to catch up with AI's breakneck pace of development.

"We find ourselves in a really terrible situation in terms of what AI is doing to kids," Tom Siegel, the former vice president of trust and safety at Google, told Reuters. "The idea that the industry will figure it out by itself without outside help is unrealistic."

Siegel's concerns mirror a growing fear among some tech insiders and academic researchers that AI poses a deeper threat to children than social media did, at a time when the industry is racing to add increasingly advanced features to chatbots that millions of young people already use daily.

Siegel joins Youth AI Safety Institute

Siegel announced yesterday (22 September) that he had joined Common Sense Media, a children's online advocacy group. The non-profit, established in 2003, conducts research and reviews the suitability of various media for children. In May, it launched the Youth AI Safety Institute, a research organisation to assess children's AI use, for which Siegel will serve as the first executive director.

Anthropic and OpenAI Foundation, the non-profit that controls OpenAI, are among the institute's backers.

Siegel said he planned to develop standards by which to hold AI firms accountable, and hoped to do so by working alongside those companies and third-party groups.

He previously worked at Google from 2004 to 2019, but left after concluding that the tech industry was "plateauing" in its ability to create safe online spaces for consumers. He then co-founded TrustLab, a startup that built tools to help companies detect safety threats, before stepping aside as CEO earlier this month to take on his new position.

As it stands, major AI companies are rolling out features to children without establishing sufficient guardrails, Siegel said.

Risks range from suicide to psychosis

"I think we're looking at very, very severe scenarios of harm being done," he said.

He suggested that the companies had immediate steps they could take to better protect young people, citing risks ranging from suicide and psychosis to cognitive offloading, a term used to describe decreased critical thinking caused by leaning on AI to answer questions.

Anthropic and OpenAI could enforce more robust age verification measures to stop underage children from accessing their chatbots, he said. His former employer Google could easily implement an option to turn off AI Overviews and AI Mode in its search engine, he added, in the same way that it already has a toggle that allows users to filter out sexually explicit content.

While these measures could affect usage or profits in the short term, AI companies that do not act will run the risk of lawsuits and regulation that could impact their business more heavily, Siegel added.

Asked to respond to Siegel's comments, a Google spokesman said parents have controls to turn off a child's access to the search engine entirely. "Our AI Search features are an incredibly useful way for kids and teens to learn, explore and make sense of information and the world," the spokesperson said.

An OpenAI spokesperson said the company uses "account and behavioural signals" in addition to self-reported age to identify underage users. "While no single age-assurance method is perfect, we continuously test and improve these systems, including against attempts to bypass safeguards."

Anthropic did not immediately respond to a request for comment.

At Common Sense Media, Siegel said a top priority would be scaling up independent standards similar to car crash testing that would measure the risk of publicly available AI tools. The effectiveness will depend in part on the willingness of AI companies to collaborate, the likelihood of which Siegel said was not certain, but was high enough for him to take the job.