As lawmakers develop a national artificial intelligence framework, health insurers are already using automated decision-making tools to influence Americans’ access to care.
Barack Obama spent an hour last week telling House Minority Leader Hakeem Jeffries and a room of Democratic donors that artificial intelligence is “moving very fast in private hands” and that the party needs a real plan before it gets away from them entirely. He said a public framework on AI safety was urgently needed and that Democrats should make AI safety a “central agenda” in January when the next Congress is sworn in and when attention begins to shift to the 2028 elections.
Obama’s remarks were reported this past weekend alongside similar warnings and calls from AI industry leaders.
House Democrats apparently are working on the kind of “framework” Obama called for. Jeffries launched the House Democratic Commission on AI and the Innovation Economy last December, and the commission reportedly has a policy framework due this fall. That’s good news – and a good start. Rules governing how AI is built, tested and deployed will inevitably shape how health insurers can use it. But that framework also has to account for what happens when AI moves into high-stakes, industry-specific decisions. But if it doesn’t mention health insurance by name, it will have missed the industry that is already furthest along in using AI to make decisions about who lives, who dies, and who pays — with almost no oversight at all.
THE BOTTOM LINE: Health insurers are already using algorithms to influence coverage decisions. Any federal AI framework should require meaningful human review, clinician override authority, transparency and patient appeal rights.
This isn’t a hypothetical harm sitting somewhere out on the horizon, the way a lot of the AI safety conversation still is. It’s happening in claims systems right now. And the company doing it loudest and proudest is none other than the biggest and most profitable health insurance conglomerate, UnitedHealth.
UnitedHealth told shareholders and Wall Street financial analysts when it announced first quarter profits in April that it’s spending $1.5 billion on AI in 2026 alone, with executives promising a “conservative” 2-to-1 return within 12 to 18 months. A third of that money is going into new AI software products the company hopes to sell to other health systems. The other two-thirds is going into what Optum Insight’s CEO called “signature end-to-end processes” — the internal machinery of how the company handles care.
Some of that machinery is aimed at speeding up prior authorization decisions, and the company points to real numbers: turnaround times cut, call volumes down, a pharmacy tool that reportedly shrank prescription approval from eight hours to under 30 seconds. That’s good if true and the whole story. Nobody is nostalgic for eight-hour prior auth waits.
But the same earnings calls that tout faster prior auth decisions (denials as well as approvals) are the ones that tout a lower medical loss ratio — the industry’s term for the share of premium dollars that actually goes to patient care. UnitedHealth’s second-quarter medical care ratio fell 270 basis points this year, and executives credited the AI investment directly for that unexpectedly big decline in medical spending. When a company brags to Wall Street that artificial intelligence helped it spend less on medical care, that is not a customer-service story. It’s a story about how the company is boosting profit margins, and patients are the input being optimized.
The algorithms already have a body count
UnitedHealth and Humana are both being sued right now over their use of an algorithm called nH Predict, which families allege was used to cut off coverage for elderly and disabled patients in extended care — sometimes overriding the judgment of the company’s own medical staff — based on a tool plaintiffs say gets it wrong up to 90% of the time on appeal. Cigna is fighting a parallel case over its PxDx system, which ProPublica found had denied more than 300,000 payment requests in a two-month span, with a company doctor spending an average of 1.2 seconds per claim.
Both cases, which are still working their way through the courts, rest on the same basic allegation: that a health plan let software stand in for the individualized medical review its own policies promised. A federal judge has already allowed the Cigna case to move forward on exactly that theory.
Elizabeth Nicholas, writing in Vanity Fair last week about her own fight with her insurer during breast cancer treatment, put the trajectory more starkly than I have. Her argument is that the industry has already trained the humans who run it to set their humanity aside and act like machines — which is exactly what will make humans so easy to replace. (Several big insurers, including Cigna where I used to work, have said they are laying off thousands of workers this year.) Soon, she writes, “there won’t even be executives left to email; only code,” carrying out profit directives with no capacity for hesitation or mercy. Her essay’s whole premise was that she still had a CEO’s name to put in an email she sent begging the insurer to reverse a denial of a life-saving treatment. Take the human off both ends of that exchange and there’s no one left to shame, sue, or vote out.
A handful of states aren’t waiting for Congress. Colorado now requires bias audits and guaranteed appeal rights for AI-driven coverage decisions. But even bias audits raise a bigger question of what exactly counts as bias when an insurer builds these tools? An algorithm can pass checks for discrimination and still be designed to advance the insurer’s financial interests, including reducing spending on medical care. And as insurers increasingly build their tools on general-purpose AI models, biases embedded in those underlying systems can carry into whatever gets built on top of them.
California and Texas have both moved to require that a licensed physician, not an algorithm alone, sign off on any denial based on medical necessity. That’s real progress — and it’s also proof of how far behind federal policy is. Coverage decisions shouldn’t depend on which state you happen to live in.
Jeffries has already put two members of his caucus in charge of a group that presumably will come up with recommendations on health care reform priorities. Alexandria Ocasio-Cortez of New York and Terri Sewell of Alabama are co-conveners of House Democrats’ Cost of Living Healthcare Working Group, tasked with building out the party’s affordability agenda on health care. So far, the public framing of that group has been about premiums, Medicaid cuts, and ACA tax credits. None of that is wrong. But if AI’s growing role in coverage denials isn’t part of what Ocasio-Cortez and Sewell put forward, the working group will have missed the fastest-moving cost driver in the field they were assigned to cover. They’re the two members with the standing and the mandate to put specific, concrete AI proposals on the table — not vague concern, but actual legislative language on human review requirements, transparency, and audit rights. That’s the natural home for this work, and it shouldn’t wait for the broader framework Jeffries is still finishing.
Every argument Obama made to Jeffries about why Democrats need an AI framework applies with more force, not less, to health insurance. He talked about job displacement from AI — insurers are already using it to displace human medical judgment. He talked about AI moving fast in private hands — seven for-profit companies control most of the American health insurance market, and they answer to shareholders, not patients. He said he didn’t want to be a “doomer,” and neither am I. AI genuinely could help identify fraud, speed up legitimate approvals, and cut the paperwork that eats a doctor’s week. Nobody serious is arguing it should be banned from health care.
The argument is narrower than that: When an algorithm is making or heavily influencing a decision about whether a person gets the care their doctor ordered, someone accountable has to be able to explain why, a licensed clinician has to be able to overrule it, and the patient has to have a real path to appeal. Right now, in most of the country, none of that is guaranteed.
Any Democratic AI framework that talks about jobs, misinformation, and existential safety risk while staying silent on the algorithms already deciding who gets a breast cancer treatment, a hip replacement or a nursing home stay isn’t a serious framework. It’s an incomplete one. Jeffries has the chance to make sure it isn’t — and given that he reportedly brought up the resignation letter of an Anthropic employee warning about the dangers of AI development in his conversation with Obama, he’s clearly already thinking about where AI could do real damage. Health insurance should be at the top of that list, not an afterthought to it.
