AI in healthcare needs strong human oversight, says India's science minister

While a pathologist examining a cancer patient's biopsy slide with the naked eye may inadvertently miss a minute but crucial cluster of affected cells, an AI-enabled system can guide him directly to the precise location, minimising human error.

Similarly, in clinical examination, AI tools analysing comprehensive patient data can flag findings that may otherwise be overlooked, thereby strengthening diagnostic accuracy and improving treatment.

These are two examples cited by India's Science and Technology Minister Jitendra Singh, himself a doctor and a diabetes specialist, at an event in New Delhi on 21 February, highlighting the need for AI in the healthcare sector.

He said AI-developed tools are set to eliminate subjectivity in medical diagnosis, ensuring more precise and specific treatment for patients. His remarks came soon after the India AI Summit in New Delhi. The use of AI in health care was one of the most hotly debated subjects during the Summit.  

The world has seen the shift from the time dominated by traditional clinical learning to one driven by imaging and molecular tools, ultrasound, CT scan, MRI and now genomics.

"The dictum of the day has shifted: where earlier medical mastery depended solely on extensive reading, today to tech-driven pathology," he said.

At a time when medicine is becoming deeply interdisciplinary, integrating MedTech, engineering and advanced data sciences, Singh said that with increasing super-specialisation, opportunities for cross-disciplinary deliberation often get limited.

Earlier, medical mastery depended solely on extensive reading; today, AI-enabled systems complement and enhance clinical judgement.

Jitendra Singh pointed to a model under which AI-assisted telemedicine services operate alongside physical doctors in rural India. While AI enhances efficiency and reach, the presence of a human doctor reassures patients and builds trust.

Such models, he said, are particularly suited to India's diverse social and linguistic landscape, where technology must adapt to local realities.

Singh emphasised that India is entering a new era of genomics and gene therapy. Under the Department of Biotechnology, large-scale genome sequencing initiatives are underway, with an initial target of sequencing one million individuals.

He referred to successful clinical research in gene therapy for haemophilia conducted in collaboration with premier medical institutions.

According to him, personalised prescriptions based on genetic profiling, environmental factors and lifestyle determinants are expected to become the norm in the time to come. AI-driven diagnostic analytics, combined with genomic insights, will allow physicians to tailor treatments to individual patients rather than adopting a one-size-fits-all approach.

A robust diagnostics ecosystem, supported by AI, genomics and credible quality standards, will play a decisive role in ensuring that preventive and precision medicine become accessible to all.

AI has already announced its presence in several health applications, including reading radiological images, predicting tuberculosis through cough sounds or disease mapping and diagnosing conditions like cancers and silent heart attacks.

At the same time, as Jitendra Singh himself acknowledged at another event on 20 February, artificial Intelligence can substitute everything, but it cannot substitute human integrity. Here lies the dilemma and the debate over how much AI can and should be allowed to influence healthcare and how, at the end of the day, human oversight remains so important.

One of the key challenges is to conduct trials and regulatory oversight of AI products, particularly how an AI arrives at a diagnosis or treatment recommendation. Another question to be answered is how reliable the data fed into AI is, which reaches a conclusion? Will pathologists trust the AI tool in the absence of a satisfactory explanation for AI-driven decisions, risking patients' safety?   

According to a report in The Indian Express, a health start-up clinic on Cloud focuses has developed a network of services, combining physical kiosks with AI-enabled digital infrastructure. These provide instant health screenings for over 60 conditions, along with real-time tele-doctor consultations and cloud-based medical records.

Abhay Agarwal, CEO of the start-up, claimed accuracy comparable to laboratory results (90–95% accuracy).

"Over 40 health parameters can be measured in about 10 minutes, with only five blood tests required, and results are instantly delivered," he said. The question is: how does one verify such claims about accuracy?

During the India AI Summit in Delhi last week, India's Health Minister Jagat Prakash Nadda launched two digital health initiatives, one of which is called the Benchmarking Open Data Platform for Health AI, which is expected to provide a structured mechanism for testing and validating AI solutions before deployment.

Nadda reiterated that AI solutions must be rigorously evaluated for performance, reliability and real-world readiness. It is clear that trust, safety and accountability must remain central to India's health AI journey.

Benchmarking Open Data Platform for Health AI, developed by the Indian Institute of Technology in Kanpur in collaboration with the National Health Authority, is expected to help systematic evaluation of AI models using diverse, anonymised real-world health datasets.

The platform is designed to assess the performance, bias and generalizability of AI systems before pressing them into service.

If AI is becoming an increasingly inescapable reality in day-to-day human life, so is, and perhaps more, the need for human control, especially in the healthcare sector, because it is a question of life and death.