What AI can’t do: The case for trust as a clinical strategy

Originally published on LinkedIn. Follow Ameya Kulkarni, MD, on LinkedIn here.

Ameya Kulkarni, MD

A few years ago, a patient (let’s call him “John”) came to me with stable angina. John’s symptoms troubled him enough for us to consider PCI (percutaneous coronary intervention). But every time we got close to the procedure, he’d postpone. I’d ask him why, and he’d say he’s just not ready. I felt like I wasn’t getting the whole story. But I also realized he wasn’t ready to tell it.

Then, one day he told me. His daughter was getting married, and he was worried that this procedure would prevent him from going. Once the wedding was over, he really wanted to do the procedure. Until then, he wasn’t ready. So, we made a plan – the right meds to get him through the wedding, then the PCI to take care of his symptoms.

I have been thinking about John a lot recently, especially as we start to think about how AI fits into clinical care. Would “Dr. AI” feel the weight of words unsaid? Would it know when to ask, and when not to? Would it invest in earning John’s trust so that it could understand the very human reasons why John chose to wait?

The question nobody is asking loudly enough

When we talk about AI in healthcare, we spend enormous energy on what is gained: efficiency, pattern recognition, reduced administrative burden. We also talk about what might be lost: jobs, human connection, the feel of a conversation that meanders between the operationally relevant and the emotionally fulfilling.

But I think there’s a third question that matters just as much: what’s left?

What is left for physicians when AI handles the tasks it’s genuinely good at? What remains irreducibly human in the work of medicine?

I see three truly human tasks in a world where AI powered medicine is the norm:

The first is navigating when no ground truth exists. AI is extraordinarily powerful at reasoning when there is a right answer to be found. But what happens when the ‘right answer’ doesn’t exist? Medicine is as much navigating complex preferences, incomplete information, and competing values as it is finding the ‘right’ diagnosis and treatment. No parametric model can capture the richness of biologic illness contextualized in the beautiful irrationality of the human condition.

The second having the right conversations when patients are uncertain about the ground truth. Take vaccine hesitancy as an example: the science on the MMR vaccine is unambiguous – it saves lives and reduces suffering. And yet some patients hesitate, for reasons that aren’t always easy to articulate. In these cases, our job as physicians isn’t to try to combat that hesitancy with data. It is to understand their anxiety, sit with it, and help them to contextualize our knowledge so they can make a decision that best reflects their values.

The third is the willingness to accept human arbitrariness. Humans are arbitrary by nature. It is a part of the beautiful, frustrating, complexity of what it means to be a person. We make seemingly different decisions on different days for reasons that we don’t always want to share. John had his reasons for choosing this therapeutic path. He didn’t need someone to convince him otherwise or to have a rational conversation. He needed me to feel the words unsaid, to give him the space to say them, and to react in a way that optimized for him, not just the disease he had.

When there is no ground truth, rejected ground truth, or simply the capricious human spirit, what is required is uniquely human: trust. And the more that technology makes the rational part of delivering care easier, the more important it becomes that patients trust their doctors to navigate their very human, and sometimes seemingly irrational, preferences and values.

Making trust structural, not aspirational

We often talk about trust in health care like a personality trait – an ideal condition all clinicians aspire to have. But it’s not enough to aspire to build trust in a medical system. We need to create the structures and systems that make trust the defining feature of the care we deliver.

That’s why, when building our executive leadership team at the Mid-Atlantic Permanente Medical Group, I created a new role: a Chief Trust Officer. This is someone whose explicit mandate is to ensure that trust is treated as a strategic priority across everything we do: how we deploy technology, how we communicate with our teams, how we engage in our communities, how we show up for patients, and how we hold ourselves accountable.

Trust isn’t just a value statement for us. It’s an operational commitment.

As we build our operational plans and evaluate new technologies and workflows, our Chief Trust Officer will ask the hard questions: Does this build or break trust between physicians and patients? If it creates distance from our patients, what can we do to mitigate that gap? And what are the strategies we must consider in everyday work to make trust easier to build and to keep?

We are at the dawn of a revolution in health care. But as AI changes almost everything about how medical care is delivered, what cannot change is our commitment to the most sacred parts of practicing medicine. It is our job – as physicians, as leaders, as institutions – to fiercely protect the most human part of medicine: trust.

Our patients, and our humanity, are depending on it.

Ameya Kulkarni, MD, is the executive medical director of the Mid-Atlantic Permanente Medical Group, one of the largest multispecialty medical groups on the East Coast, serving more than 750,000 Kaiser Permanente members across Maryland, Virginia, and Washington, D.C. He is a board-certified interventional cardiologist with a longstanding focus on innovation, physician wellness, and building healthcare systems that work for everyone.

The post What AI can’t do: The case for trust as a clinical strategy appeared first on Permanente Medicine.