ChatGPT Outperforms Human Doctors At Accurately Diagnosing Patients

Hello!

Could AI Chatbots One Day Replace Your Doctor?

It’s looking more likely than you might think. A recently published study has sparked fresh debate about the role of artificial intelligence in clinical decision-making.

Study Design and Methodology

In the experiment, 50 doctors were asked to diagnose medical conditions based on case reports. Some participants were randomly assigned to use ChatGPT to assist with their reasoning, while others worked independently. Performance was evaluated not only on the accuracy of the final diagnosis but also on the clarity and quality of the doctors’ explanations.

Key Findings

On average, doctors working alone scored 74 percent. Those who collaborated with the AI chatbot reached 76 percent. However, ChatGPT operating on its own achieved a striking 90 percent—significantly outperforming both groups of human physicians.

The research, published in a peer-reviewed journal, was limited in scope: the 50 doctors reviewed just six case studies. Despite its modest size, the study carries important implications for how AI might be integrated into healthcare and how human biases can influence diagnostic decisions.

Expert Reaction

“I was shocked at the results,” said study co-author Adam Rodman, an internal medicine specialist at Beth Israel Deaconess Medical Center in Boston, during an interview on the New York Times podcast Hard Fork. “My hypothesis going in was that people using ChatGPT would be the best. So I am surprised by this.”

The cases, drawn from real patients, were deliberately challenging. Rodman suggested that the results may reflect not only the AI’s analytical strengths but also a degree of resistance among clinicians to accepting AI-generated suggestions.

Why Doctors Lagged Behind

Physicians using ChatGPT sometimes dismissed the chatbot’s input and doubled down on their initial assessments. Another contributing factor may have been limited familiarity with the tool at the time.

Rodman cautioned against concluding that ChatGPT is inherently more competent than human doctors. “The difference is that the people who put the case together—the information, if you want to think about the prompts—were expert clinicians,” he explained. “We organize it in such a way.”

In other words, human experts performed the critical work of gathering and structuring accurate medical information—something current AI systems cannot yet replicate independently.

Study Goals and Limitations

The primary aim was to assess how effectively a chatbot could support doctors, which proved modest. The study was not designed to prove AI superiority, and earlier research has shown ChatGPT struggling with diagnostic tasks.

Notably, the experiment was conducted in 2025 using an older version of ChatGPT. The results could be even more impressive in 2026 with newer models.

“Maybe AI models are better at making diagnoses than human doctors. But I don’t think that’s the case with GPT-4 Turbo, which was the model that was used here,” Rodman said. “But it’s going to be true at some point, and we’re quickly approaching that.”