Artificial general intelligence (AGI) refers to AI systems that can match or exceed human cognitive abilities across a wide range of tasks, including complex medical decision-making.
With tech leaders predicting AGI-level capabilities within just a few years, clinicians and patients alike may soon face a historic inflection point: How should these tools be used in healthcare, and what benefits or risks might they bring? Last month’s survey asked your thoughts on these pressing questions. Here are the results:
My thoughts:
I continue to be impressed by the expertise of readers. Your views on artificial general intelligence (AGI) closely align with those of leading technology experts. A clear majority believes that AGI will reach clinical parity within five years. A sizable minority expect it will take longer, and only a small number doubt it will ever happen.
Your answers also highlight where GenAI could have the greatest impact. Most respondents pointed to diagnosis (helping clinicians solve complex or uncertain medical problems) as the No. 1 opportunity. But many also recognized the potential to empower patients: from improving chronic disease management to personalizing care. And unlike the electronic health record, which adds to clinicians’ workloads (and contributes to burnout), GenAI is widely seen by readers as a tool that could relieve some of that burden.
Ultimately, the biggest concern may lie not with the technology, itself, but in who controls it. Like many of you, I worry that if clinicians don’t lead the way, private equity and for-profit companies will. And if they do, they will put revenue above the interests of patients and providers.
Thanks to those who voted. To participate in future surveys, and for access to timely news and opinion on American healthcare, sign up for my free (and ad-free) newsletter Monthly Musings on American Healthcare.
* * *
Dr. Robert Pearl is the former CEO of The Permanente Medical Group, the nation’s largest physician group. He’s a Forbes contributor, bestselling author, Stanford University professor, and host of two healthcare podcasts. Check out Pearl’s newest book, ChatGPT, MD: How AI-Empowered Patients & Doctors Can Take Back Control of American Medicine with all profits going to Doctors Without Borders.
More than two-thirds of U.S. physicians have changed their mind about generative AI and now view it as beneficial to healthcare. But as AI grows more powerful and prevalent in medicine, apprehensions remain high among medical professionals.
For the last 18 months, I’ve examined the potential uses and misuses of generative AI in medicine; research that culminated in the new book ChatGPT, MD: How AI-Empowered Patients & Doctors Can Take Back Control of American Medicine. Over that time, I’ve seen the concerns of clinicians evolve—from worries about AI’s reliability and, consequently, patient safety to a new set of fears: Who will be held liable when something goes wrong?
From safety to suits: A new AI fear emerges
Technology experts have grown increasingly certain that next-gen AI technologies will prove vastly safer and more reliable for patients, especially under expert human oversight. As evidence, recall that Google’s first medical AI model, Med-PaLM, achieved a mere “passing score” (>60%) on the U.S. medical licensing exam in late 2022. Five months later, its successor, Med-PaLM 2, scored at an “expert” doctor level (85%).
Since then, numerous studies have shown that generative AI increasingly outperforms medical professionals in various tasks. These include diagnosis, treatment decisions, data analysis and even expressing empathy.
Despite these technological advancements, errors in medicine can and will occur, regardless of whether the expertise comes from human clinicians or advanced AI.
Fault lines: Navigating AI’s legal terrain
Legal experts anticipate that as AI tools become more integrated into healthcare, determining liability will come down to whether errors result from AI decisions, human oversight or a combination of both.
For instance, if doctors use a generative AI tool in their offices for diagnosing or treating a patient and something goes wrong, the physician would likely be held liable, especially if it’s deemed that clinical judgement should have overridden the AI’s recommendations.
But the scenarios get more complex when generative AI is used without direct physician oversight. As an example, who is liable when patients rely on generative AI’s medical advice without ever consulting a doctor? Or what if a clinician encourages a patient to use an at-home AI tool for help interpreting wearable device data, and the AI’s advice leads to a serious health issue?
In a working paper, legal scholars from the universities of Michigan, Penn State and Harvard explored these challenges, noting: “Demonstrating the cause of an injury is already often hard in the medical context, where outcomes are frequently probabilistic rather than deterministic. Adding in AI models that are often nonintuitive and sometimes inscrutable will likely make causation even more challenging to demonstrate.”
That paper, published earlier this year in the New England Journal of Medicine, is based on hundreds of software-related tort cases and offers insights into the murky waters of AI liability, including how the courts might handle AI-related malpractice cases.
However, Mello pointed out that direct case law on any type of AI model remains “very sparse.” And when it comes to liability implications of using generative AI, specifically, there’s no public record of such cases being litigated.
“At the end of the day, it has almost always been the case that the physician is on the hook when things go wrong in patient care,” she noted but also added, “As long as physicians are using this to inform a decision with other information and not acting like a robot, deciding purely based on the output, I suspect they’ll have a fairly strong defense against most of the claims that might relate to their use of GPTs.”
She emphasized that while AI tools can improve patient care by enhancing diagnostics and treatment options, providers must be vigilant about the liability these tools could introduce. To minimize risk, she recommends four steps.
Understand the limits of AI tools: AI should not be seen as a replacement for human judgment. Instead, it should be used as a supportive tool to enhance clinical decisions.
Negotiate terms of use: Mello urges healthcare professionals to negotiate terms of service with AI developers like Nvidia, OpenAI, Google and others. This includes pushing back on today’s “incredibly broad” and “irresponsible” disclaimers that deny any liability for medical harm.
Apply risk assessment tools: Mello’s team developed a framework that helps providers assess the liability risks associated with AI. It considers factors like the likelihood of errors, the potential severity of harm caused and whether human oversight can effectively mitigate these risks.
Stay informed and prepared: “Over time, as AI use penetrates more deeply into clinical practice, customs will start to change,” Mello noted. Clinicians need to stay informed as the legal landscape shifts.
The high cost of hesitation: AI and patient safety
While concerns about the use of generative AI in healthcare are understandable, it’s critical to weigh these fears against the existing flaws in medical practice.
Each year, misdiagnoses lead to 371,000 American deaths while another 424,000 patients suffer permanent disabilities. Meanwhile, more than 250,000 deaths occur due to avoidable medical errors in the United States. Half a million people die annually from poorly managed chronic diseases, leading to preventable heart attacks, strokes, cancers, kidney failures and amputations.
Our nation’s healthcare professionals don’t have the time in their daily practice to address the totality of patient needs. That’s because the demand for medical services is higher than ever at a time when health insurers—with their restrictive policies and bureaucratic requirements—make it harder than ever to provide excellent care. Generative AI can help.
But it is imperative for policymakers, legal experts and healthcare professionals to collaborate on a framework that promotes the safe and effective use of this technology. As part of their work, they’ll need to address concerns over liability. Ultimately, they must recognize that the risks of not using generative AI to improve care will far outweigh the dangers posed by the technology itself. Only then can our nation reduce the enormous human toll resulting from our current medical failures.
Day one of the healthcare strategy course I teach in the Stanford Graduate School of Business begins with this question: “Who here receives excellent medical care?”
Most of the students raise their hands confidently. I look around the room at some of the most brilliant young minds in business, finance and investing—all of them accustomed to making quick yet informed decisions. They can calculate billion-dollar deals to the second decimal point in their heads. They pride themselves on being data driven and discerning.
Then I ask, “How do you know you receive excellent care?”
The hands slowly come down and room falls silent. In that moment, it’s clear these future business leaders have reached a conclusion without a shred of reliable data or evidence.
Not one of them knows how often their doctors make diagnostic or technical errors. They can’t say whether their health system’s rate of infection or medical error is high, average or low.
What’s happening is that they’re conflating service with clinical quality. They assume a doctor’s bedside manner correlates with excellent outcomes.
These often false assumptions are part of a multi-millennia-long relationship wherein patients are reluctant to ask doctors uncomfortable but important questions: “How many times have you performed this procedure over the past year and how many patients experienced complications?” “What’s the worst outcome a patient of yours had during and after surgery?”
The answers are objective predictors of clinical excellence. Without them, patients are likely to become a victim of the halo effect—a cognitive bias where positive traits in one area (like friendliness) are assumed to carry over to another (medical expertise).
This is just one example of the many subconscious biases that distort our perceptions and decision-making.
From the waiting room to the operating table, these biases impact both patients and healthcare professionals with negative consequences. Acknowledging these biases isn’t just an academic exercise. It’s a crucial step toward improving healthcare outcomes.
Here are four more cognitive errors that cause harm in healthcare today, along with my thoughts on what can be done to mitigate their effects:
Availability bias
You’ve probably heard of the “hot hand” in Vegas—a lucky streak at the craps table that draws big cheers from onlookers. But luck is an illusion, a product of our natural tendency to see patterns where none exist. Nothing about the dice changes based on the last throw or the individual shaking them.
This mental error, first described as “availability bias” by psychologists Amos Tversky and Daniel Kahneman, was part of groundbreaking research in the 1970s and ‘80s in the field of behavioral economics and cognitive psychology. The duo challenged the prevailing assumption that humans make rational choices.
Availability bias, despite being identified nearly 50 years ago, still plagues human decision making today, even in what should be the most scientific of places: the doctor’s office.
Physicians frequently recommend a treatment plan based on the last patient they saw, rather than considering the overall probability that it will work. If a medication has a 10% complication rate, it means that 1 in 10 people will experience an adverse event. Yet, if a doctor’s most recent patient had a negative reaction, the physician is less likely to prescribe that medication to the next patient, even when it is the best option, statistically.
Confirmation bias
Have you ever had a “gut feeling” and stuck with it, even when confronted with evidence it was wrong? That’s confirmation bias. It skews our perceptions and interpretations, leading us to embrace information that aligns with our initial beliefs—and causing us to discount all indications to the contrary.
This tendency is heightened in a medical system where physicians face intense time pressures. Studies indicate that doctors, on average, interrupt patients within the first 11 seconds of being asked “What brings you here today?” With scant information to go on, doctors quickly form a hypothesis, using additional questions, diagnostic testing and medical-record information to support their first impression.
Doctors are well trained, and their assumptions prove more accurate than incorrect overall. Nevertheless, hasty decisions can be dangerous. Each year in the United States, an estimated 371,000 patients die from misdiagnoses.
Patients aren’t immune to confirmation bias, either. People with a serious medical problem commonly seek a benign explanation and find evidence to justify it. When this happens, heart attacks are dismissed as indigestion, leading to delays in diagnosis and treatment.
Framing effect
In 1981, Tversky and Kahneman asked subjects to help the nation prepare for a hypothetical viral outbreak. They explained that if the disease was left untreated, it would kill 600 people. Participants in one group were told that an available treatment, although risky, would save 200 lives. The other group was told that, despite the treatment, 400 people would die. Although both descriptions lead to the same outcome—200 people surviving and 400 dying—the first group favored the treatment, whereas the second group largely opposed it.
The study illustrates how differently people can react to identical scenarios based on how the information is framed. Researchers have discovered that the human mind magnifies and experiences loss far more powerfully than positive gains. So, patients will consent to a chemotherapy regiment that has a 20% chance of cure but decline the same treatment when told it has 80% likelihood of failure.
Self-serving bias
The best parts about being a doctor are saving and improving lives. But there are other perks, as well.
Pharmaceutical and medical-device companies aggressively reward physicians who prescribe and recommend their products. Whether it’s a sponsored dinner at a Michelin restaurant or even a pizza delivered to the office staff, the intention of the reward is always the same: to sway the decisions of doctors.
And yet, physicians swear that no meal or gift will influence their prescribing habits. And they believe it because of “self-serving bias.”
In the end, it’s patients who pay the price. Rather than receiving a generic prescription for a fraction of the cost, patients end up paying more for a brand-name drug because their doctor—at a subconscious level—doesn’t want to lose out on the perks.
Thanks to the “Sunshine Act,” patients can check sites like ProPublica’s Dollars for Docs to find out whether their healthcare professional is receiving drug- or device-company money (and how much).
Reducing subconscious bias
These cognitive biases may not be the reason U.S. life expectancy has stagnated for the past 20 years, but they stand in the way of positive change. And they contribute to the medical errors that harm patients.
A study published this month in JAMA Internal Medicine found that 1 in 4 hospital patients who either died or were transferred to the ICU had been affected by a diagnostic mistake. Knowing this, you might think cognitive biases would be a leading subject at annual medical conferences and a topic of grave concern among healthcare professionals. You’d be wrong. Inside the culture of medicine, these failures are commonly ignored.
The recent story of an economics professor offers one possible solution. Upon experiencing abdominal pain, he went to a highly respected university hospital. After laboratory testing and observation, his attending doctor concluded the problem wasn’t serious—a gallstone at worst. He told the patient to go home and return for outpatient workup.
The professor wasn’t convinced. Fearing that the medical problem was severe, the professor logged onto ChatGPT (a generative AI technology) and entered his symptoms. The application concluded that there was a 40% chance of a ruptured appendix. The doctor reluctantly ordered an MRI, which confirmed ChatGPT’s diagnosis.
Future generations of generative AI, pretrained with data from people’s electronic health records and fed with information about cognitive biases, will be able to spot these types of errors when they occur.
Deviation from standard practice will result in alerts, bringing cognitive errors to consciousness, thus reducing the likelihood of misdiagnosis and medical error. Rather than resisting this kind of objective second opinion, I hope clinicians will embrace it. The opportunity to prevent harm would constitute a major advance in medical care.
HENRY KOTULA 