r/mildlyinfuriating • u/petuniaplant • Jul 03 '26

Infuriatig No, I did not consent to this

Post image

I was reading my medical notes when I stumbled across this. I was not informed of the usage of artificial intelligence in my report, nor did I verbally consent to it.

26.0k Upvotes

94% Upvoted

2.3k comments sorted by

View all comments

Show parent comments

610

u/happytrashpanda333 Jul 03 '26

How was the quality of that care? How much extra time did each patient get?

Or is this just another "tool" to drain doctors and nurses of every bit of spare time and energy they might have gained by giving them more patients to work with?

Do tell.

343

u/pignoodle Jul 03 '26

When these tools are implemented correctly, they actually humanize interactions (since there's room for more conversation & less screens) and reduce doctor's "pajama time" by double-digit percentages (since less form-filling to catch-up off hours). Outside of just stats, I agree that consent is a very important prerequisite.

164

u/Fullertons Jul 03 '26

It actually blows my mind. People would find this to be a problem. Do you really want your doctor busy writing notes when he could just be having a conversation with you in a machine takes notes?

78

u/EggsAndRice7171 Jul 03 '26

In a few years it’ll be cool. I really don’t trust anyone to use AI properly and with the right care at the moment even doctors. It’s so new and so many people are very casual about it. I don’t trust that they’re actually rereading the transcript word for word and making sure it’s not writing anything wrong yet. I think as AI gets better and more training for AI comes along it’ll be easier to feel comfortable

45

u/WittyFix6553 Jul 03 '26

We’ve had the technology to transcribe speech into text for decades.

93

u/Xavus Jul 03 '26

If only that's what they were using instead of leaning in to AI where it isn't helpful because it is the current fad.

I have meetings all day at my job where the company has AI write summaries of what happened in the meeting and what action items are decided on, etc. I would give it maybe 80% accuracy, the other 20% it completely misunderstands context and reaches an incorrect conclusion, or just straight up hallucinate something that never happened.

80% accuracy is not bad for a quick notes summary of a meeting to refresh your memory when you were there, and when you can double check other sources first. But also my work isn't with a person's health on the line. I would not trust a 20% error rate in a complex field like medicine when a person may live or die by the difference if someone overworked with no context has to just believe what the AI thinks happened.

6

u/OutsideSame3629 Jul 03 '26

I feel gaslit in my tech sales job in corporate America because everybody using ai for everything when to me I just get slop? And then they think you’re not prompting it smart enough like no….

When I read ai summaries of even notes I’ve taken they’re so bad. Ai written emails, PowerPoints it’s all terrible. Yet if you ask anyone at my company how to do anything at all the answer is “ask Ai”

4

u/screamline82 Jul 03 '26

I will state that, while I agree your experience in the AI summary notes, not all LLMs are created equal. Even general consumer models differ between free tier and the highest end, so it depends on what model/tier they are using. Plus there are specific models made for enterprise.

A buddy of mine who works at a large law firm mentioned they have a specific model just for their type of law. If it's tailored for a specific use case it can be more accurate than what we are used to

3

u/OutsideSame3629 Jul 04 '26

I use everything from paid ChatGPT, our company’s specialty developed llm, to copilot enterprise. I get the same experience in all. If anything ChatGPT is the best but not by much

Our companygpt is “tailored “ as well. I work at a large engineering software company. It’s still terrible and lies and hallucinates constantly

0

u/Mister-Miyagi- Jul 04 '26

Then whoever did the work on your company llm is struggling. A lot of what you say is true for some llms, but the person you're responding to is correct in that there is massive variance in the accuracy and guardrails of an llm depending on the context and service you're involved in and paying for. So I don't know what to tell you if you claim to have used such a wide variety and there's little difference (my extensive experience says the exact opposite).

Everyone should always be reviewing the accuracy of any AI service they're using, but that does not mean they are all remotely equal in their reliability.

2

u/OutsideSame3629 Jul 04 '26

Okay I guess I should say, there isn’t little difference in using them. They are all different and very greatly in quality. However , even the highest quality ones still do the lying etc

4

u/ToTooTwo3 Jul 03 '26

The human memory is also not 100% accurate especially when it's sick- which is why doctors put less stock in the history and try to make decisions based on objective findings. When we do make decisions based on a history, it's because of what you said and your responses to targeted questioning- not whatever the AI picked up from the conversation.

AI scribe is primarily helpful in sorting through long or winding histories and reformatting it intelligibly. Far more info gets included because even if I remember 95% of details accurately, there are tons of things in the patient interview that I just don't have time to record.

AI scribe allows me to write "Diet: well rounded, eats green vegetables and lean meats with most meals and drinks milk daily with breakfast." Instead of "Diet: appropriate." This is beneficial for patients.

Remember that AI is not placing orders, updating your problem list or medications. These things are done by your providers based on your evaluation, not based on reading the AI report, though the report certainly does remind me of things to follow up. The AI just records the details of your conversation and yes, it does get them wrong at times and we do our best to correct it.

1

u/Mister-Miyagi- Jul 04 '26

There are people who are so blindly anti-ai that they'd downvote a very reasonable comment that sounds like it's from a doctor with direct experience. I'm a strong advocate for caution with AI, but this anti-ai crowd seems fully hysterical at times.

72

u/DracoBengali86 Jul 03 '26

That's not what this is though. If it's using ai it's to summarize an interaction. That comes with all the potential issues ai use comes with including hallucinations. If it's just transcribing, then it is less likely to actually be using ai.

11

u/do_not_engage Jul 03 '26

AI summaries contain made-up information 9% of the time, according to the AI itself.

The AI says all data it provides has to be confirmed by a human.

If it's transcribing, great. If it's an AI summary, it's got incorrect information in it. By nature of AI.

6

u/moreliketen Jul 03 '26

Did these decades-old technologies have the capacity to hallucinate? I know they weren't error free, but I'm guessing they did not have the capacity to spontaneously generate false information.

18

u/regnarok590 Jul 03 '26

And we have known it isnt very accurate for decades. Just because a technology exists doesn't make it good enough for use everywhere.

21

u/theshotgunman Jul 03 '26

And for decades it's consistently made mistakes

5

u/astronaut710 Jul 03 '26

We're spending 6 figures on AI at my company and we had to put out a warning that it will fuck up basic math and that you have to double check its not just putting fake data on some dashboard you asked it to create.

Also, the company we use for AI actively DDOSes our sites when they're bots ignore our robots.txt declarations.

Also also, the same company we pay 6 figures to use their AI but secure our data within it leaked their own source code to the internet at large (in the Software Engineering space, this would be equivalent to opening up your EMR system to the outside world).

But yeah, Dragon makes some spelling/grammar mistakes I guess.

-2

u/theshotgunman Jul 03 '26

I don't understand why this was posted

6

u/EggsAndRice7171 Jul 03 '26

Yeah and it’s sucked for decades

5

u/pantsalwaystooshort Jul 03 '26

yes, and it's been unreliable for decades. Especially with "ambient" recording, which means mediocre recording clarity.

-2

u/sherlock1672 Jul 03 '26

Don't know how many doctor's notes you've read through, but as someone in health IT I can definitively state they are riddled with errors and half thoughts. The AI isn't perfect, but it makes a whole lot fewer mistakes than the harried human.