Hospitals Are Deploying AI Like Crazy. We Still Don’t Know If It Helps Patients

I don’t need to tell you that AI is everywhere. Or that it’s been creeping into hospitals for years now.

Doctors use AI scribes to take notes during appointments. Algorithms trawl through patient records to flag people who might need extra support. AI interprets chest x-rays and lab results. A growing pile of studies says these tools are accurate. But accuracy isn’t the same as usefulness.

The real question — the one we keep skipping — is whether using AI actually makes patients healthier. And right now, we don’t have a good answer.

That’s the argument Jenna Wiens from the University of Michigan and Anna Goldenberg from the University of Toronto make in a new paper in Nature Medicine. Wiens has spent over a decade trying to get clinicians interested in AI. For years she was pitching to a mostly indifferent audience. Then something shifted. “A switch flipped,” she told me. Suddenly providers are eager — not just interested, but actively deploying these tools.

The problem? Most aren’t rigorously checking whether they work.

Take ambient AI scribes. These tools listen to doctor-patient conversations, transcribe them, and generate summaries. Multiple products are already out there, and adoption has been fast. A few months ago, a staffer at a major New York medical center told me doctors are “overjoyed.” The tech lets them focus on the patient instead of typing notes. Early studies back this up — they show reduced burnout and higher satisfaction.

Great. But what about patient outcomes? “Researchers have evaluated provider and clinician satisfaction, but not really how these tools are affecting clinical decision-making,” Wiens says. “We just don’t know.”

The same applies to predictive tools that forecast health trajectories or recommend treatments. Even an accurate AI might not improve outcomes. Say an AI speeds up chest x-ray interpretation. How much will a doctor actually rely on its analysis? How does it change the way they interact with the patient or choose treatment? And what does that mean for the patient in the end?

Those answers probably vary by hospital, department, and even individual clinician experience. Wiens points out that research on AI in education suggests these tools can change how people process information. Could AI scribes alter how a doctor thinks about a patient’s case? Could they affect how medical students learn to reason through clinical data? “We like things that save us time, but we have to think about the unintended consequences,” she says.

A study published in January 2025 by Paige Nong at the University of Minnesota found that about 65% of US hospitals use AI-assisted predictive tools. Of those, only two-thirds evaluated accuracy. Even fewer checked for bias. That number has probably climbed since then, and Wiens thinks the gap between deployment and evaluation has only widened.

She’s not anti-AI. “I do believe in the potential of AI to really improve clinical care,” she says. She just wants more data. “I have to believe that in the future it’s not all AI or no AI. It’s somewhere in between.”

The worst-case scenario isn’t necessarily that AI harms patients — though that’s possible. More likely, Wiens argues, these tools just aren’t delivering the benefits hospitals assume they are. And without proper evaluation, we’ll never know the difference.