'Unbelievably dangerous': ChatGPT Health may miss life-threatening emergencies, finds study

Experts and medical researchers have raised concerns about the safety of ChatGPT Health, a feature launched by OpenAI in January 2026 that provides health advice by connecting users' medical records and wellness apps.

A study published in the journal Nature Medicine, the first independent safety evaluation of the tool, found that the AI frequently fails to recognize life-threatening medical emergencies, says the Guardian.

The study, led by Dr. Ashwin Ramaswamy, tested the AI using 60 realistic patient scenarios. It found that in 51.6% of cases where immediate hospital care was medically necessary, ChatGPT Health advised the user to stay home or book a routine appointment.

In simulations of respiratory failure or diabetic ketoacidosis, researchers noted a "50/50 chance" of the AI downplaying the severity. In one asthma scenario, the platform suggested waiting for a future appointment rather than seeking emergency help, despite identifying signs of respiratory failure.

The AI was also 12 times more likely to downplay symptoms if the "patient" mentioned that a friend thought the situation was not serious. Conversely, it advised 64.8% of healthy individuals to seek immediate medical care, which experts said could lead to unnecessary medical presentations.

Researchers expressed particular concern over the AI's handling of suicide ideation. While the platform displayed crisis intervention banners when a patient mentioned suicidal thoughts alone, these guardrails failed in every attempt (0 out of 16) when "normal lab results" were included in the scenario. Dr. Ramaswamy said a guardrail that fails based on irrelevant data like lab results is "arguably more dangerous than having no guardrail at all" because its failure is unpredictable.

Experts, including Alex Ruani from University College London, described the findings as "unbelievably dangerous," warning that the AI creates a "false sense of security" that could lead to preventable harm or death. Professor Paul Henman highlighted the potential for legal liability for tech companies as AI-related suicide and self-harm cases enter the legal system.

An OpenAI spokesperson said the study did not reflect how people typically use the tool in real life and emphasized that the model is continuously updated and refined. Researchers, however, maintain that the "plausible risk of harm" justifies the urgent need for independent oversight and stricter safety standards.