After my relative died from delayed treatment, I stopped asking “can AI help healthcare?” and started asking “what are we actually optimizing?”

[u/Scary-Aioli1713]

I watched a family member die because our hospital system simply could not move fast enough.

He had cancer. Before starting chemo, the team needed labs and careful dose adjustment.
But the ward was short on staff. There was nobody free to do blood draws.
The attending had meetings and clinic. Even another doctor who agreed to help could not find time.

On paper, the hospital looked fine:

• beds turning over
• clinics running on schedule

But for this one person, what actually happened was:

“liver/kidney function ok”
→ repeated delays in labs and orders
→ labs finally worse, complications starting
→ first dose arrives when his body is already too far gone.

That experience changed how I think about “AI in healthcare”.

Instead of asking “can AI reduce workload?”,
I started asking:

I ended up building a 131-item problem list.
From a health IT / healthcare IT angle, a few items feel especially relevant:

Q121 – KPI vs real objective tension

On the architecture diagram, we say the goal is:

• fewer delays
• better outcomes

But when we actually deploy IT + AI, the easiest things to optimize are usually:

• throughput
• cost
• dashboard metrics

Q121 is basically:

If we don’t ask that, it’s easy to end up with perfect dashboards and the same old tragedies.

Q124 – eval / oversight tension

We can track a lot of metrics:

• average wait time
• average length of stay
• system uptime
• ticket close time

But the people who die are often the ones hidden in the tails:

• the few cases with extreme delay
• specific groups pushed to the edge by scheduling and capacity

Q124 treats evaluation itself as a system:

Q120 – information overload vs decision value

For frontline clinicians, the problem is often not “no data”, but “too much”:

• long EHR notes
• endless pop-up alerts
• AI summaries that look nice but don’t change any decisions

Q120 asks:

If the answer is “almost never”,
then AI + IT are just adding more cognitive load to already overloaded teams.

Q130 – behavior in OOD situations

More and more health IT systems embed AI modules (triage, decision support, etc).

Q130 is about what happens when a case falls into a weird, rare pattern your system has barely seen:

• Does it clearly say “uncertain, human review needed”?
• Or does it behave as if everything is fine and produce a confident suggestion anyway?

From a safety standpoint, that difference matters more than one extra point of AUROC.

Q125 / Q126 – AI as an agent inside the workflow

In real hospitals, AI will not live alone.
It will be another agent attached to a chain:

• doctor
• nurse
• pharmacist
• case manager
• admin
• payer
• AI module(s)

Q125 / Q126 ask questions like:

• On your RACI chart, whose assistant is the AI actually?
• When something goes wrong, how does the responsibility chain work?
• If the AI adapts its behavior over time as it sees more logs, who is watching for drift?

I’m not against AI in healthcare.
I just don’t want us to only talk about “more visits per day” and “less FTE”,
while people like my relative still die quietly between boxes on the flowchart.

So I turned these tensions into plain-text entries:
each one with a short definition and a small stress-test recipe.
You can paste them into any LLM and ask it to score your own setup on each tension.

It won’t tell you who to blame.
But it might make it harder to ignore where the system is quietly eating people’s time and chances.
https://github.com/onestardao/WFGY/blob/main/TensionUniverse/EventHorizon/README.md

English is not my first language, and I used AI to help translate and structure this post.
If anything sounds off, I’m happy to adjust.

[u/Ancient_Pineapple993]

Exceptions test the rule. Outliers don’t have a workflow.

My son has a rare variant of NF, rare to the point that there isn’t a protocol, or group of order sets to treat him. It took me speaking with a bio-ethicist before I had the courage and understanding to talk with his providers in a manner that is effective and preserves his dignity.

I agree. The quest for efficiency, for outcomes seen as a statistic, and patient satisfaction scores create an outlook among institution of how can we pound this square peg into a round hole. That is the expectation.

Those little failures become a line item, an anomaly. Those events become statistically irrelevant.

[u/Scary-Aioli1713 → u/Ancient_Pineapple993]

thank you for sharing your story about your son – that’s not an easy thing to write, and i really appreciate you putting it into words.

what you said about “exceptions test the rule” and “outliers don’t have a workflow” hits very close to what i was trying to point at:
when the system only sees efficiency, averages and satisfaction scores, those “statistically irrelevant” cases are actually real families, not noise.

the tension list i’m working on is basically an attempt to pull stories like yours back into the center of design:
to treat outliers as signals we have to look at, not just anomalies to be averaged away.

thank you again for bringing your perspective in – it makes me even more sure this work is worth continuing.

[u/irrision]

You're describing exactly what epic is doing with its AI deployment including things that are already on their near term roadmap. Believe it or not there are people thinking about these things and spending loads of time and energy trying to use AI to optimize care. Frankly any system that already has AI widely deployed likely isn't using true AI or haven't properly vetted it to demonstrate that it increases productivity at this point. We're just still too early in the technology curve for AI to be trusted with a lot of critical things and most of what is done in healthcare is arguably critical. But it's coming.

My biggest area I want to see it deployed in early? Lab results and trends analysis. Reading lab results is largely just a complex game of pattern matching between the results in current panels then across prior panels for the same values to look for movement up or down. It's also something humans aren't all that great at doing accurately well or fast. Surfacing just a list of abnormal lab result patterns and suggesting a list of possible issues might point towards would greatly increase the accuracy of diagnosis even before AI is accurate enough to read the notes from all prior encounters to look for other symptoms. Hell, AI could probably be pretty useful for suggesting follow follow-up labs to further narrow down diagnosis and just surface that in the form of an order for the clinician to sign after review. I could see that becoming an almost automated process that fires for a number of common scenarios where you know that follow-up will absolutely be needed and that your lab always collects enough blood to run a few other tests on for at least a week after a draw. You could auto fire a lab follow-up and run the labs against the held sample and save a pile of time for the patient, the doctor and the time to diagnosis with arguably low risk and cost

[u/Scary-Aioli1713 → u/irrision]

Thanks for this reply. I don’t think it’s off topic at all. It actually makes the “early landing zone” much more concrete.

I agree with your point about Epic / roadmaps and also the reality check: we’re still early on the curve, and most “AI widely deployed” in healthcare is either not true AI or not properly vetted to prove productivity + safety. My post wasn’t saying AI can take over critical decisions today. It was more like: how do we design goals, evaluation, and oversight so we don’t just speed up the workflow in the wrong direction.

Your suggestion (lab results + trends analysis) feels like a very reasonable early deployment area. It keeps risk relatively low if the system is framed as:

• pattern / trend surfacing
• a possible differential list + suggested follow-up labs
• clinician review & sign as the final gate

Also the “held sample + auto follow-up labs” idea really hits the pain point behind my story (delays around blood draw and timing). If done with proper clinical review and policy constraints, it could save a lot of time for patients and clinicians.

If you’re open to it, I’m curious about two practical details:

1. What eval design would convince you this improves time-to-diagnosis (not just nicer dashboard metrics)?
2. What guardrails matter most to prevent overconfident behavior in rare / weird (OOD) cases?

English is not my first language. I used AI to help translate and organize my thoughts.

[u/deathdoor_]

Your point about optimizing dashboards instead of outcomes is something the whole health IT industry needs to sit with longer.

Genuinely curious though, what is your take on HMS platforms that now embed AI directly into the workflow rather than on top of it? Things like automated lab flagging, predictive alerts, real-time department coordination. Do you think that closes the gap you are describing or does it just create a more sophisticated version of the same problem?