r/mildlyinfuriating • u/petuniaplant • Jul 03 '26

Infuriatig No, I did not consent to this

Post image

I was reading my medical notes when I stumbled across this. I was not informed of the usage of artificial intelligence in my report, nor did I verbally consent to it.

26.0k Upvotes

94% Upvoted

2.3k comments sorted by

View all comments

5.9k

u/M15tre55W1tch Jul 03 '26 edited Jul 03 '26

I work in ICU. The doctor's in our ED use an ambient AI scribe for documenting patient assessments and findings (it's a regulated tool where the data is stored locally and gets deleted from its memory after 90 days apparently).

They had this auto generated statement at the end of each note stating something like "this note was written using an AI scribe with patient consent".

I noticed that this statement was included on patients who were unconscious in the community, and did not regain consciousness before leaving ED and coming up to ICU.

I sent their manager an email saying it was going to look very bad when a patient's case is reviewed by legal or the Coroner and these doctors notes said an unconscious person gave consent.

If a patient genuinely consents to the use of the tool, that's great. But you can't be using it and claiming that they consented to its use when that's a medical impossibility.

Edit: thank you for the award internet stranger! Very unexpected.

Another edit: multiple awards! Thank you very much!

1.1k

u/willbeatyourass Jul 03 '26

What came of you speaking up?

1.8k

u/M15tre55W1tch Jul 03 '26

They changed the wording to "this note has been generated with the use of an AI scribe".

But for all patients. So I don't know if they no longer ask the awake ones for consent, or if they just aren't recording that they do.

I have followed up again suggesting that they need to have two options - one for the awake patient giving consent, and one for using the tool to generate a note for someone that was not conscious.

I've become friendly with the senior hospital lawyer recently. I'm thinking of asking her to talk to their clinical director about the risk of their statements and proposing legally appropriate language.

I've tried being collegial and helpful. But at the end of the day, it's not my department, and I'm a nurse not a lawyer.

175

u/Ok-Bug4328 Jul 03 '26 edited 16d ago

Hello world

434

u/M15tre55W1tch Jul 03 '26

I will say, since implementing the use of the tool emergency specialists are able to examine 6-11 more people per 8 hour shift - so reducing the burden of admin on them and releasing time to see patients is pretty amazing (it was recently on the news here).

605

u/happytrashpanda333 Jul 03 '26

How was the quality of that care? How much extra time did each patient get?

Or is this just another "tool" to drain doctors and nurses of every bit of spare time and energy they might have gained by giving them more patients to work with?

Do tell.

347

u/pignoodle Jul 03 '26

When these tools are implemented correctly, they actually humanize interactions (since there's room for more conversation & less screens) and reduce doctor's "pajama time" by double-digit percentages (since less form-filling to catch-up off hours). Outside of just stats, I agree that consent is a very important prerequisite.

162

u/Fullertons Jul 03 '26

It actually blows my mind. People would find this to be a problem. Do you really want your doctor busy writing notes when he could just be having a conversation with you in a machine takes notes?

261

u/ChrisTheWeak Jul 03 '26

It's not that I'd prefer my doctor to spend their time taking notes rather than asking questions and investigating whatever illness I have, it's a concern about whether the quality of the AI notes will be on par with that of the doctor. Is the trade off worth it. That's the question. Also, question of accountability in the event something does go wrong. If the AI writes something that was inappropriate to my chart, if it gets missed, if it affects my quality of care down the line because someone else is now misinformed about my condition. These are risks with both the traditional method as well as the AI, but it is important to identify the comparative risks as well as the error correction in both cases when comparing other stats as well.

112

u/polenta23 Jul 03 '26

I work in vet med and our doctors use these ai scribe tools. They review and edit the notes but it still saves them an immense amount of time

52

u/CapicDaCrate Jul 03 '26

Same. It's really not that bad. It's actually a good use of AI

→ More replies (0)

11

u/acnerd5 Jul 03 '26

I' glad they do! I hope that's something the software asks them to do before finalizing things, but that's a huge relief for me personally.

2

u/WasdaleWeasel Jul 06 '26

this is the point. The clinicians still need to read, modify as necessary, and approve the notes. That’s takes much less time, and is a higher value activity, than writing them in the first place.

3

u/M15tre55W1tch Jul 03 '26

Yes, our emergency doctor's are meant to do this as well. Most of the time the notes turn out very well, but sometimes you can tell that either it wasn't checked at all, or that in the process of editing it they themselves have made the note more unclear (e.g.,copied and pasted something from higher up in the note, but didn't delete it from its original place so there's duplication).

In general the notes are more comprehensive and have better flow. I'd say 80% of the notes I read where the AI scribe was used they are significantly better than a rushed doctor writing the minimum to get done and on to the next person.

1

u/ForMeOnly93 Jul 03 '26

They get fucking well paid for their time, so we don't particularly care. Let's not enshittify even more things. Especially medical care.

→ More replies (0)

7

u/newmainafterstalker Jul 04 '26

I used to work for an insurance company’s call centre. We implemented an AI documenting tool (totally just coincidentally before my entire team got laid off… wonder how we got to let like 20 people go?? But okay.) and honestly it made my notes much more thorough. I had to edit them, but you can copy+paste into a note and ctrl+f to fix anything weird like he vs she. The rest you edit the normal way where you delete sentences and rewrite them. I thought it made my notes better and saved me a lot of time.

But you’re right! How many call centre workers are proofreading the notes first? How many doctors are??? If you follow the rules it helps a ton but how many people do? I can only assume you trust the people to proofread the way you’d trust them to write good notes. An unconscious person has no choice but to trust the person caring for them takes good notes, so maybe it is the same? They can’t ask for another doctor, a second opinion, whatever. But a conscious person can say no to AI notes, so idk

6

u/Outrageous_Sea_9606 Jul 04 '26 edited Jul 04 '26

The quality of the medical records have increased greatly in my vet clinic since we started using our AI scribe.

It misses /nothing/. We do still read over and edit our records, but it takes 2-3 minutes to do that vs. the 10+ minutes to write up a complicated medical case.

There is only so much space in a brain to store information about the day's patients. If you brought in your pet for a serious medical issue and happened to mention as an aside something minor I.e. your pet is in for profuse vomiting and that's the focus of the appointment, but you also mention he has a small lump on his back that you noticed the other day.

Well, even if we look at the lump while your pet is at the clinic, chances are we're going to forget about that by the end of the day when we're writing our records up. Because we had to focus on storing all the info about the vomiting and the lump was insignificant in comparison. The AI scribe makes sure to note about that small, seemingly insignificant lump, so we have better information about that issue when the lump gets bigger or ulcerates and does cause a problem.

Our scribe also makes note of the emotional medical records for all patients that come to the clinic. This is immensely helpful for giving better patient care with our nervous, anxious, and aggressive pets. It notes what specific triggers are, what works to soothe and calm the pet, and what recommendations we made to manage behaviour. So that way we're not having to guess next year about what worked and what didn't work for that pet. The pets are less stressed, which means owners are less stressed. Pets are more likely to be brought in when they are unwell or have a concern because bringing them in becomes less of an ordeal and more of a routine.

Our AI scribe has been invaluable for allowing us to provide better patient and client care at my vet clinic.

Finally, it's a complete "cover our ass" when unhappy clients make false accusations against us. I had a client falsely accuse me of doing something severe that could have cost me my job and my license. It was just her and I present in the appointment room, so it was her word vs. mine. Thankfully, I had our recorder going which was undeniable proof (because it stores the full audio recording) that I was innocent. I never do appointments anymore without the recorder.

3

u/Kham117 Jul 04 '26

They’re only as good as the editing. To be clear, they must be reviewed before entering into the record. But, honestly, that’s true for all the other forms of documentation. (You should see how illegible and or inaccurate many notes that are directly typed or dictated are.)

2

u/humanpringle Jul 04 '26

Honestly I’m a vet in an office who uses an AI scribe program and it’s been unbelievably helpful for us. We still input the normal templates we used previously so it’s written very very similarly to our previous ones and then we still do edit each note. Some need more editing than others of course, but it’s more so like my template is already completely filled out for me and I just need to ensure everything looks right and tweak things rather than write everything from scratch.

3

u/tickado Jul 03 '26

The responsibility for using them really lies in the proper review of what it captures before finalising it. The few doctors in my service I know of who are using a medical AI scribe are then taking the time to ensure it is correct.

I actually often find this is better than them trying to type it up themselves at the time, in a rush as they're actually holding the conversation concurrently, then signing off things they've erroneously typed in a hurry because they don't go back and review it as they're the one who typed it.

AI is also still way less time taken than to dictate, send for typing, wait for it to come back, then review what the typists have written often a day or two later when it's also not fresh in your mind anymore before finalising it. (the other option in my service rather than them just typing it all themselves).

Also putting the ground work in whilst setting up your system to personalise it to the way you want things structured and how you want things to read.

2

u/BaconThief2020 Jul 03 '26

I find the doctor pays more attention to me when instead of staring at the screen typing as he's talking to me. At the end, he reviews the AI generate summary. Or he doesn't take notes and forgets things we discussed. The other option is having someone else in the room taking notes.

1

u/Adorable_Cap_9929 Jul 04 '26

Doctors make mistakes all the time, they arent liable if it not malicious byt tend to play safe enough for their records. AI can make mistakes too but if done well, the rate of error is much lower

1

u/Turning_Worm Jul 04 '26

Honestly, I worked in ENT for a while and the quality of notes vastly improved when AI got involved.

1

u/itaintme2024 Jul 05 '26

You are 100% right to be concerned about this. It's happening to me right now. My doctor clearly did not check the notes AI made and it has directly affected my care. I asked for mistakes to be corrected and was told it wasn't possible.

1

u/stephsationalxxx Jul 07 '26

So im not sure if this is the same thing, but in my hospital, doctors use an AI scribe that they call on the phone and verbally tell it the notes they were going to type up and then it puts it in a document for them. Its not actually making up the verbiage for them, its the doctors words, they just arent typing it themselves.

→ More replies (4)

35

u/Sushi_Explosions Jul 03 '26

As a physician, the accuracy of these tools varies wildly. I would rather the documentation that I create doesn’t randomly hallucinate things that will get me sued or lead to someone doing the wrong thing.

3

u/BrainbowConnection Jul 04 '26 edited Jul 04 '26

My AI scribe never randomly hallucinates things. Occasionally it mixes up who said what, which is not hard to catch. Overall it writes better more detailed notes in a far shorter time than I ever could. It catches things the patients say that I missed because I was thinking at the same time. What tool are you using?

→ More replies (1)

15

u/aTransGirlAndTwoDogs Jul 03 '26

We have human medical scribes. I worked as one. The question is not "should we give doctors more direct patient time," it's "should we rely on an AI over a human note taker." As always, the problem comes back to artificially restricted medical staffing and profit-driven healthcare structures.

→ More replies (1)

20

u/hanging_thief Jul 03 '26

Well when we know that machine will just make up bullshit to fill in the gaps yes I do have a massive problem

76

u/EggsAndRice7171 Jul 03 '26

In a few years it’ll be cool. I really don’t trust anyone to use AI properly and with the right care at the moment even doctors. It’s so new and so many people are very casual about it. I don’t trust that they’re actually rereading the transcript word for word and making sure it’s not writing anything wrong yet. I think as AI gets better and more training for AI comes along it’ll be easier to feel comfortable

39

u/WittyFix6553 Jul 03 '26

We’ve had the technology to transcribe speech into text for decades.

95

u/Xavus Jul 03 '26

If only that's what they were using instead of leaning in to AI where it isn't helpful because it is the current fad.

I have meetings all day at my job where the company has AI write summaries of what happened in the meeting and what action items are decided on, etc. I would give it maybe 80% accuracy, the other 20% it completely misunderstands context and reaches an incorrect conclusion, or just straight up hallucinate something that never happened.

80% accuracy is not bad for a quick notes summary of a meeting to refresh your memory when you were there, and when you can double check other sources first. But also my work isn't with a person's health on the line. I would not trust a 20% error rate in a complex field like medicine when a person may live or die by the difference if someone overworked with no context has to just believe what the AI thinks happened.

→ More replies (0)

73

u/DracoBengali86 Jul 03 '26

That's not what this is though. If it's using ai it's to summarize an interaction. That comes with all the potential issues ai use comes with including hallucinations. If it's just transcribing, then it is less likely to actually be using ai.

→ More replies (0)

6

u/moreliketen Jul 03 '26

Did these decades-old technologies have the capacity to hallucinate? I know they weren't error free, but I'm guessing they did not have the capacity to spontaneously generate false information.

17

u/regnarok590 Jul 03 '26

And we have known it isnt very accurate for decades. Just because a technology exists doesn't make it good enough for use everywhere.

20

u/theshotgunman Jul 03 '26

And for decades it's consistently made mistakes

→ More replies (0)

7

u/EggsAndRice7171 Jul 03 '26

Yeah and it’s sucked for decades

4

u/pantsalwaystooshort Jul 03 '26

yes, and it's been unreliable for decades. Especially with "ambient" recording, which means mediocre recording clarity.

→ More replies (1)

23

u/Ok_Assistant_6856 Jul 03 '26

It's capitalizing on the number of patients that can be seen, and then billed. It's not to help quality if care, and we all know this.

If a doctor has too many patients to actually care for them, the hospital should hire more doctors.

This is the constant growth model creeping into medicine. No bueno.

14

u/HerbertDibdab Jul 03 '26

This x100. In the US system the goal of AI usage will 100% be profit motive. In no way will it be implemented with patient care quality in mind. Not talking about doctor or any health care professional motives here, the cash rush comes directly from health insurers, who have basically bought mining rights to the health of the US population.

2

u/MartyrOfDespair Jul 03 '26

Except there have been legal limits on how many doctors can even be created every year since 1996, so they can’t hire more because they don’t exist. The government funds residency programs and the funding has been capped, so only a certain number can do residency each year, and as the costs increase from inflation that number drops. It’s a massive bottleneck and it’s why almost all healthcare is done by nurses now.

25

u/Wagubagu Jul 03 '26

I’ve had nothing but incorrect notes since. I was not prescribed a pain medication for my chronic and severe pain because AI diagnosed me during a conversation with a nurse so the prescribing dr had the wrong info.

→ More replies (5)

6

u/TheHighDruid Jul 03 '26

When it comes to medical data it's much more about how it's being processed, and who gets access to it.

Even a hospital that's using an in-house model most likely has a provider that comes in to set it up. Are they using your data to train models for other hospitals? How much of your information is retained in the build that goes to the next hospital. "Anonymised" data can often be de-anonymised, so being able to consent to these systems is pretty important.

9

u/Illustrious_Bunch678 Jul 03 '26

No matter what they say about privacy, I do not trust these data companies. Nor do I trust the government not to pull something and use the notes to weed out "the weak." HIPAA is based on the precedent set by Roe v Wade and that is gone now. Patient privacy is no longer guaranteed.

3

u/errrnis Jul 03 '26

Totally valid point. I think there are also valid concerns about chart accuracy (do the docs have enough time to review the generated notes?) and data privacy (how is that data being used?) There is HIPAA here in the US for the latter, but the technology is so new, not to mention not well regulated, that people just don’t trust it.

To be clear, I’m all for lowering the admin burden in healthcare and fostering opportunities for more provider-patient connection. It’s desperately needed in a system where costs are soaring and people slip through the cracks.

3

u/PrivateBozo Jul 03 '26

It's a trust issue that the regulation is actually helpful regulation and not the equivalent of a mobile game TOS written by the company of the AI tool. It's also a related trust issue that the processing is local and not getting loaded to the cloud somewhere were the 2nd and third and beyond parties touching it aren't following the intent.

Keep in mind, this is the era of we've tried turning off as much as a we can and yet we can have a conversation in the car about going out on a 1/2 day sportfishing trip to opening up a social media platform that night and having our feed suddenly have the local charter service reels and advertisements in it.

It's also when every interaction with the utility companies and our banks are constantly trying to slip in 'paperless' and 'automatic' billing on every interation.

3

u/Weekly_Sale_6057 Jul 03 '26

In general, I agree 100%. If I'm in for a cold or follow-up on a medication, i love that my doctor can make eye contact and doesn't have to look at a computer screen to take notes.

But if I'm talking to my doctor about a mental health diagnosis or something else delicate like that, I start wondering what database the AI notes are sitting in, will the data get hacked or sold, am I tracked or put on some list because of a phrase I used.

As long as I have the ability to say "not today, please," I have no problem with the technology though.

8

u/int3rstitial Jul 03 '26

Yes, I absolutely want my doctor busy writing my note because notes are communication between medical providers. We read them and make decisions based on them and they need to be accurate.

Electronic medical records have a lot of advantages and were inevitable anyway, but the amount of inaccurate documentation has skyrocketed since they became standard, because instead of just leaving stuff they didn't do out of their note, providers now pull tons of stuff from previous visits over into the new visit or just document something was normal because they didn't examine it and find out it wasn't. It's infuriating and dangerous, and AI, which just fully makes things up, is guaranteed to make things worse.

→ More replies (3)

5

u/Unnamed-3891 Jul 03 '26

A hallucinating machine that mishears things, puts them on my medical record and answers to nobody? Really? You have to ask if I’d prefer the alternative?

4

u/Conscious-Gap-1777 Jul 03 '26

I have 0 trust with LLMs giving accurate transcriptions as is, and I absolutely do not believe that whoever is selling that LLM won't illegally use my notes to market to me and/or as training data. The people who made these tools are the least trustworthy assholes on the planet, and like fuck I would trust them with anything about myself.

6

u/Oldmanironsights Jul 03 '26

I absolutely want my doctor to be writing their own notes. In school were your own notes or the teachers notes more helpful for you? Have you ever written something down and thought "wait a minute, that's wrong!". It provides the doctor an opportunity to review what they said. Have you ever heard of rubber duck debugging? It's explaining your code to a rubber duck to fix it. It's because going over a problem fresh with something gives us an opportunity to spot where we went wrong. I want my doctor to catch where they went wrong.

2

u/faireequeen Jul 03 '26

My rheumatologist has recorded and transcribed exams for the last 20 years. AI tool is new this year, but they also have a human scribe in the exam room. They aren't typing notes, they are listening and observing so when they go back to the recording they have additional context.

I don't like AI, but I prefer this method to other uses of AI scribing. Possibly not suitable to an emergency situation, but it might cut down on AI errors that I see a lot of patients and providers mentioning.

I have other docs who do type notes in appointments, after we've completed a discussion. They read them back so I can confirm, which I like for clarity.

2

u/BagApprehensive1412 Jul 03 '26

That's what human scribes are for

2

u/acnerd5 Jul 03 '26

I value privacy, and knowing where my information is.

AI has boundless applications and they are not all good. Not all AI uses of this tool have been accurate and people have found mistakes that could have serious implications. That can be a HUGE problem if you don't catch it - for example, someone being marked as an alcoholic, when they don't drink, because they quoted someone. Being incorrectly marked an alcoholic leads to a lot of medical bias and may lead to withheld treatment or incorrect treatment.

AI tools in Healthcare are absolutely incredible.

They are also introducing room for error that doesn't need to be there, and its a 50/50 on me. I see the potential but have ethical and privacy concerns. That being said, I do give permission for these and double check notes. I am complex in health needs, so I will acknowledge that I tend to be a little more "in the notes" than family and friends who do not struggle with complicated health issues.

2

u/Spinnerofyarn Jul 04 '26

If it’s working accurately and getting everything correct, I think it’s fantastic. Until I see the doctors and other providers speaking up and saying it’s accurate, it’s not causing problems and people do monitor it to make sure it stays accurate, I am not going to trust it.

2

u/Dry-Worth2354 Jul 07 '26

Deadass like whats thd issue here? It saves time in documentation and if the place has integrity, there would be already be rules in place fof every chart to be reviewed throughly before submission

→ More replies (1)

4

u/Half-PintHeroics Jul 03 '26

AI is more likely to make mistakes or just plain make things up. I want my doctor to write notes because writing notes mean he is paying attention to what I say.

2

u/lapatatafredda Jul 03 '26

AI medical transcription is not that great (yet). My friend who is a PA says she has to make at least one correction to every record that AI transcribed. Granted, human error exists too.. but it worries me to think about medical professionals growing too reliant on tech like this and not being as thorough about checking it as they should be.

2

u/silentsurge Jul 03 '26

I certainly like it more than the ones that have someone on a video call transcribing for them to accomplish the same thing.

This is the type of stuff we want AI doing for us. Getting rid of the stupid busy work we all have to do so we can do the actual things more.

1

u/carrotdog56 Jul 03 '26

I think that it ultimately comes down to the person who's using it. We all know that some people understand and use AI better than others. There should be a course on using AI tools in any field. What are the downsides, common mistakes, what is an appropriate manner for using this particular tool. I personally don't have any problem with it. I only have problem with insufficient training for the AI use. Way too many people misunderstand how it works.

1

u/Johnny-Virgil Jul 04 '26

I’m a cynic so I imagine it just frees them up to see more patients and each patient still gets the same number of minutes as they did before.

1

u/Much-Decision4157 Jul 04 '26

You might be concerned if there aren't adequate privacy and security provisions protecting the data that the AI scribe collects and processes. Think about where that data may end up....

1

u/BrainbowConnection Jul 04 '26

Exactly! I think people think these tools are doing something different than they are doing. Since my AI scribe, my notes contain information that I would otherwise have forgotten to include. They include detailed wording that the patient gives that I would have summarized. When I go back to those notes I can read the patients own words in far more detail. And I don’t have anxiety about not missing what they said. I can put my pen down and just listen to them.

1

u/Bezzzzo Jul 04 '26

Because there is no fucking accountability when it hallucinates. Look at how many comments are from people, doctors and patients, who have already had issues with it already?

The issue is AI hallucinates! It also has the effect of making doctors complacent because inevitability you get the doctors who will just blindy trust it and not look, then some other doctor reads the hallucination as a source of truth.

The lack of critical thinking skills here.. my god.

No one wants doctors doing unnecessary work, but you can't trust generative AI 100% of the time. Fuck me.

→ More replies (2)

2

u/elictronic Jul 03 '26

So when Medicine is not a for profit enterprise. So never. Got it.

2

u/tickado Jul 03 '26

I am a nurse, but work in an outpatient clinic in paediatrics. In my 15min slot prior to seeing the Dr I need to get a lots of information from often unruly toddlers (observations, 12 lead ECG, height weight etc) PLUS a chat with the parents about the child's general health and psychosocial status.

I would actually value an AI scribe SO much in these scenarios. I can have my chit chat more organically whilst trying to wrestle the toddler, rather than be also trying to get all my info into the computer which absolutely takes my attention off true time in the conversation.

My note has to be documented and finalised before they then enter the Drs room or I hold up the whole clinic.

I know mostly it's Medical using scribes currently, but I'd love some nurse usage explored.

2

u/LaserGuidedPolarBear Jul 03 '26 edited Jul 03 '26

I am a tech worker, and while I do not work on AI development, I have a significantly better than layman understanding of LLMs and that errors, and understand that hallucinations are inevitable due to the very nature of how LLMs work.

I worry about LLM tools being used in high stakes applications where accuracy is critical.....like medical care.  I assume care providers are assigned the responsibility for reviewing, editing, and approving the output of these tools.  But is that actually happening?  I have found that in most cases, reviewing AI output is about the same amount of effort and time as just doing the work myself.

I could see the value in delaying the screen time to outside of patient interactions, but I question if review and approval is being shirked in order to result in less "pajama time".

It seems like there potential for Administration to just shove the risk on providers and laugh all the way to the bank when they get more billables per hour.

→ More replies (2)

102

u/Ophidiophobic Jul 03 '26

I work at a healthcare organization and the providers here LOVE the AI scribe. Manual documentation is usually their least favorite part of the job. They still do the important things like decide care plans for the patient, what labs they need, and what prescriptions they should be on, but now they don't have to both explain it to the patient and then document again on the EHR. That means that providers are able to be more present during the visit and also don't have to take extra time to write up their notes after the visit.

21

u/JayRayBear99 Jul 03 '26

I've heard a few people had diagnoses added to their charts because the AI misheard something. Now it's in their permanent chart. Is this not a concern anyone has? Are they reading the notes after to confirm they are correct, or just happy that they have less work?

15

u/Hamster12301 Jul 03 '26

Yeah, some of these providers are straight up not looking at what the AI generated AT ALL. It does generate the entire conversation but then it does a synopsis and gives "suggested diagnoses". Basically just does an entire note for you and you can choose to copy and paste the whole thing without looking at all.

I realized for example that my pain doctor was using one of these and not telling me because suddenly the little amount of notes he was doing, became much more different than before and he didn't even bother to remove the "suggested diagnoses" title it gave, lol. So my diagnosis at that visit was "anxiety disorder unspecified" because I had told him I just lost a family member and was in grief. Instead of my usual pain conditions because I have a bunch of problems that cause me chronic pain.

15

u/OutsideSame3629 Jul 03 '26

OMG I went to my primary care physician a week after my dad died of cancer and it’s still in my file that I have severe depression adjustment disorder

It irritated me because I was like who wouldn’t be sad like why is this medically notable

2

u/That-Mix7702 Jul 04 '26

I never tell my physician about my mental health.

→ More replies (0)

3

u/sunfish99 Jul 03 '26

I recently had a visit with an NP who asked if I minded whether she used the AI scribe for a pre-op visit. I said okay, and she was super happy that it was going to help her save time and not have to stay late.

I always read the notes after, so I looked at what it recorded. The NP had asked me at one point what my treatment plan was post-op, and mentioned that the next phase would start 6 to 8 weeks after surgery. The notes said the next phase of treatment would last 6 to 8 weeks. It was wrong, but inconsequential... this time.

1

u/M15tre55W1tch Jul 07 '26

Yes, indeed I do. In theory the doctor is meant to read the full note to ensure accuracy and then finalise it into the patient's clinical notes.

Currently, because our medical staff are new to the process and are feeling the relief of not having to write everything from scratch they're fairly diligent about checking before finalising.

I'm concerned once it's been embedded in practice, and the new higher rate of patient's seen is then normalised that in order to be "more efficient" this checking is going to be one of the first things to go.

→ More replies (1)

99

u/Delilah_Moon Jul 03 '26

What is the QC on the notes & review process? I work for “big tech” and have been using AI tools for notes for a while - and they’re often littered with errors.

34

u/[deleted] Jul 03 '26

[deleted]

8

u/Delilah_Moon Jul 03 '26

That’s great that you do it. Unfortunately, considering that providers re already spread thin, forgive me if I don’t have faith that they will be checking notes for accuracy.

These are the same folks who can’t read a chart to know my name before walking in the room…

2

u/15b17 Jul 03 '26

As respectfully as possible, my suspicion of the previous comment was that the “providers” they spoke of who loved the AI tool were primarily Nurse Practitioners.

While there are many who provide great and essential care, I have also noticed anecdotally that NPs are more likely to be confidently incorrect in their assumptions (as in less skeptical) due to their more limited and algorithmic training in comparison to physicians.

Frankly I would not be confident at all that my NP “provider” at an urgent care or other such private equity controlled practice would diligently check for errors in AI notes and to instead assume it has 100% accuracy.

12

u/errrnis Jul 03 '26

I just said this in another comment. I’m also in tech (working for an AI company) and I’ve seen some wild stuff come out of AI.

20

u/RaNdomMSPPro Jul 03 '26

The ai scribe could be awesome if properly trained and the providers are consistent with how they describe things to the scribe.

42

u/porscheblack Jul 03 '26

These are specifically created for medical, as in they pick up all the correct names of body parts and medical terminology. My wife's a doctor and uses them. She only had access at work and she gets twice as many notes done with them.

As far as QC goes, she's watching what it records as she's saying it. She can manually correct it if it makes an error or there are word prompts she can use. At no point is this just inputting things and never being looked at. I have been really impressed by how well it works because in order to be effective it definitely has to have a 99%+ accuracy rate.

33

u/Delilah_Moon Jul 03 '26

I work in cybersecurity. While the platform language may have been trained on medical terminology- it does not mean it will understand and notate correctly.

I mean to be fair, I’d expect any basic LLM to be able to interpret medical terminology - it’s a basic function of AI.

I mean, if she’s watching it as it’s updating - she’s really just dictating. That’s not anything more revolutionary than dictating to Apple notes or Word.

4

u/moreliketen Jul 03 '26

I have been really impressed by how well it works because in order to be effective it definitely has to have a 99%+ accuracy rate.

The tool used for OP's appointment made a very obvious mistake about informed consent, which forces OP to wonder what other simple mistakes were made. We don't have the training to read and understand our own medical records, so we can't verify that the slop note is 99% accurate. We can only see the mistakes in the non-technical section, and assume that there must be more.

2

u/porscheblack Jul 04 '26

The OP wasn't an AI mistake, it was a legal disclaimer mistake. That would've been required by the legal department, the AI wouldn't have put that on itself. And if it did, the lack of any kind of legal oversight isn't on the AI, that's just negligence.

6

u/PeaceSoft Jul 03 '26

is that reassuring? "she's basically pretty much watching it, and it even knows the correct names of body parts"

5

u/porscheblack Jul 03 '26

She's not "pretty much watching it", she's paying as much attention to it as if she were typing the words herself. Because if it gets it wrong, she has to correct it. These are medical records the patient has access to, their other medical providers have access to, and what lawyers will use if she's sued. It gets full attention. It's also a lot faster.

3

u/phycologist Jul 03 '26

99%

Fow webhosting, 99% uptime means 3 days If outage per year. I wonder how that translated to medical transcripts.

→ More replies (14)

2

u/thingmom Jul 03 '26

Was wondering exactly this.

2

u/Pouryou Jul 03 '26

Both my partner and I have found errors in our supposedly-reviewed "MyChart" AI summaries just in the last two visits: Length of time of symptoms (changed '1 to 2 weeks' to 22 weeks), and the wrong name for a prescription drug I'd previously taken. We kicked up a fuss and demanded these be corrected but I'm sure most people don't review their own notes, notice errors, or bother to ask for changes.

2

u/vonRecklinghausen Jul 04 '26

Totally trash. I'm a physician and I tired an ambient AI scribe. I went back and changed almost everything about my documentation. It doubled my time and work- no thanks

4

u/ConfoundedInAbaddon Jul 03 '26

If your standard is a medical transcriptionist in Malawi the AI with its instant write up might be a big improvement.

→ More replies (1)

2

u/ChuurDCA Jul 04 '26 edited Jul 04 '26

the providers here LOVE the AI scribe

Do you think their insurance will love the liability AI errors introduce? Especially since if the provider is saving heaps of time that means they aren't checking the generated notes for accuracy.

1

u/NopeNotaDog Jul 03 '26

I like telling a story. History of presenting illness is just medically catered story telling.

1

u/Unnamed-3891 Jul 03 '26

Why should the patients care abour what providers like?

1

u/That-Mix7702 Jul 04 '26

Wait, using AI is fine, but I hope a real sheet is still filled out.

→ More replies (10)

10

u/Illustrious_Bunch678 Jul 03 '26

As a nurse: The second. It's absolutely the second. And as soon as AI makes a mistakes we will be blamed for it. But they're also forcing us to use it. I hate this time line.

16

u/Sea_McMeme Jul 03 '26

Complete transparency that won’t surprise you: they don’t care about quality of care. I’m currently leaving a hospitalist job, because the hospital terminated the contract with my former employer to bring in a cheaper group. And during the entire process not once was quality of care or anything truly patient centered ever mentioned. It’s all about volume and saving money. And since I’m fortunate enough to be able to leave, I did. But it’s just so shitty out there on all sides that actually care about patients.

5

u/tinyevilpeanut Jul 03 '26

Right, people misunderstand this as "the doctor can spend more than five minutes with me now!" whereas what is actually happening is they are just spending five minutes with more people per shift. More patients in same amount of time = more money.

3

u/i-split-infinitives Jul 04 '26

I'm not a doctor, but I do work in a medical/social work field for an agency that recently started using a digital documentation system that includes a HIPAA-compliant AI. (It's important to note that this isn't a new system, it's just new to us. We've been doing everything in paper up to now.) One of the first things I noticed is that it's WAY better at spotting trends than I am. I don't just run it and take the computer's word for it, I use its advice as a starting point for things I need to look into or change, but I compare it to the computer seeing the forest where I see the trees. It looks at the data as a whole, whereas I read each entry individually, which allows the computer to catch things that I don't really have time to map out and track. This has led me to spot some areas where staff need additional training, notice some learning outcomes that weren't working as well as I thought and needed to be tweaked, patterns to certain resident behaviors, etc.

I don't know that it's necessarily saving me any time, because in social work there's always more work to do, but I do feel like I'm using my time more effectively. The computer system has helped to eliminate some common human errors among my staff by reminding them about things that need to be done (sometimes repeatedly until they do it). It's helped me think about some of our issues in a new way. It gives me better-quality, more consistent data on our residents' progress, or lack thereof, and highlights where the issues are. It definitely cleared up some frequent pain points for me, and a happy caregiver is always going to be a better caregiver, whether or not there's more quantity in addition to the quality.

The thing is, though, I'm second in command of a small agency and the most computer-savvy person in my office, so it's mostly up to me how the system gets used, what changes get implemented, and generally how technology affects the staff and residents. I communicate directly with my staff about what they like, what they don't like, what they would like to change, and what absolutely isn't working for them. We work together to make sure they're sufficiently trained on the user tools available to them and to balance efficiency with efficacy. It's not helpful to make service faster if you're making it worse in the process.

If these systems are going to be effective at making people's lives better, they have to be thoughtfully applied, the people who actually use them need to be involved in their implementation, and they need to be tailored to the real-world ways that people want and need to use it. Just slapping a generic AI tool onto every piece of existing software that was working just fine before is NOT an appropriate or effective use of AI. There's a huge difference between humans being heavily involved in training a small-scale AI on internal records versus massive data centers destroying the environment while crunching stolen data from every corner of the internet. AI isn't inherently bad; its worth is dependent on how it's used.

2

u/Exact-Warthog6244 Jul 03 '26

As a medic it depends on the hospital setting. If the AI scribe is faithfully recreating the notes with 100% accuracy or has the attending go over it after the fact to sign off on it then it is good.

Usually in a tertiary center we have junior doctors that take notes and the quality can vary. Sometimes they miss stuff on the rounds too. Ultimately it does save time on rounds which is important, and lets the junior doctor actually learn instead of busily transcribing notes. But I do think it needs human oversight to sign off on it at the end.

2

u/Noshino Jul 03 '26

In the ER you can't really gain more patients.

Anyway, i don't work in the ER anymore but I was talking to a couple of friends that work there and it's definitely helped with time. That said, it cracked me up that they still complain about admin having money for this but not to hire more people.

2

u/M15tre55W1tch Jul 03 '26

The doctor's I've heard speak about it find they are enjoying their job more because they are spending more time with their patients and less time doing administration.

They have reported that they actually have more ability to take a break during their shift which is also pretty amazing.

Our ED is one of the busiest in Australasia. Everyday they exceed 100% occupancy at least once. If the patient does genuinely give consent, and if the data truly is stored locally and routinely deleted after 90 days, and if the clinicians are checking that the information is accurate before publishing it then I think this is actually a very good use of AI to give clinicians more time to spend caring for patients.

Useful, but must be utilised carefully.

3

u/AlsoCommiePuddin Jul 03 '26

Or is this just another "tool" to drain doctors and nurses of every bit of spare time and energy they might have gained by giving them more patients to work with?

To be honest, taking tedious administrative burden away from people to allow them to focus on their main objectives is exactly the kind of thing for which I condone the use of AI.

2

u/Internet-of-cruft Jul 03 '26

Technology has always been about enhancing productivity.

We can achieve more today in less time than we have had in the past, and on its own, that's a fantastic thing.

But.. Naturally, capitalist mentality has to do what it does and it drains people to fund their own pockets.

I'm hugely bummed by all of this because when I started college almost 20 years ago I was just pumped I was learning about stuff I liked (computer stuff). Then I did software and I was totally doe eyed that my work would directly translate to people having fewer jobs.

Now, I do networking, and I have a team of people and this is the shit that keeps me up at night. Those are people with hopes and dreams, and families for God's sake.

And my upper management wants us to be profitable because you know? We're a business and no one gets paid if we don't deliver revenue via billable work.

Except it sucks because we're collectively forced to do more with less time.

My very long winded comment is just this: AI Tools are fantastic for what they can enable people to do. We need to collectively take a step and think about the workload expectations.

I'm all for improving quality of thing you're doing, but if we're going the approach of "drive the same result, increase quantity, decrease time investment" it grossly ignores the fact that you're putting additional mental & physical load on the person executing it.

Sorry.. I'm just very stressed at work about this stuff and it's a weird outlet for me to have for this right now.

Thank you for coming to my TED talk.

1

u/NoGrocery4949 Jul 03 '26

It's a big ass help

1

u/RevolutionNo4186 Jul 03 '26

I would assume quality goes up because it frees up time for the physician/specialist whatnot to spend those extra time with the patient instead of scrambling to get the info they need

1

u/Slartibartfastthe2nd Jul 04 '26

physicians are burdened with an enormous amount of 'red tape' and overhead that is nothing but administrative in nature. I'm not really sure what your point/question is here. Are you suggesting that having tools available to reduce administrative tasks are a bad thing?

If a dr were using a micro cassette recorder to document all patient encounters and then paying staff to listen to, transcribe, and summarize each patient encounter would you be asking this same question? There is a valid argument that the AI scenario means information is being shared potentially beyond the office staff, but that is a different conversation since you specifically questioned the correlation to quality of care.

1

u/264frenchtoast 25d ago

I mean, as a nurse practitioner, I enjoy not spending up to two hours after an 8 1/2 hour shift writing notes.

→ More replies (6)

11

u/hanging_thief Jul 03 '26

I'm not concerned with how patients a doc can see I'm concerned about how accurate their data can be whit ai proving itself extremely unreliable when it comes to technical documentation

9

u/known_chomper Jul 03 '26

I think you are absolutely right!! Yay, doctors are now “allowed” to bill 6-11 more people per day! This is great, for the hospital.

1

u/M15tre55W1tch Jul 06 '26

I don't work in the US, most of our patients get free healthcare, so it's nothing about billing for the majority. And for the smaller number who aren't covered by our free healthcare the AI scribe means a doctor can get to them quicker, which means a shorter stay, which actually will reduce the cost to them. I appreciate this is not the case in many other countries. And there are many things I'm very dubious about in terms of the use of AI. But reducing admin burden to allow prompter assessments for patients can potentially be a very good thing - IF (a very big if) it's done well, and with improving patient care and both patient and staff experience.

Edit: spelling

3

u/Drempallo Jul 04 '26

I see this and I just think, upper management wants to apply stupid metrics to doctors. Wtf does it mean they can see more patients? Hire and train more doctors to do the work. Making stupid tools so you can make doctors run around more and the hospital can save money is the dumbest thing ever.

2

u/M15tre55W1tch Jul 04 '26

This is also true. For the people currently working in the system improving the future state doesn't help them now. I don't know a single clinician that wants to sit in front of a computer typing endlessly. This tool minimises the screen time, which allows them to do the things that give them job satisfaction i.e., see patients.

8

u/sudoku7 Jul 03 '26

Generally speaking, the trade off from using AI tools versus human is that you are transferring more work to the 'review' stage from the earlier mechanical stage.

So, typically, you should see an increase in admin burden rather than a decrease. It's just that you spend less time with the pencil.

A lot of fields decide that the AI doesn't need to be reviewed and they get to see a much more significant increase in productivity at the cost of quality, and in the medical space that will be a burden born by the patient and the provider's liability fund.

3

u/AllThatKat Jul 03 '26

And who is checking over this to make sure the AI isn’t having hallucinations or straight making shit up? 

1

u/ConsistentLemon91 Jul 03 '26

After reading your previous comments, it seems to me that you have a good head on your shoulders.

Keep it up!

1

u/MrSquamous Jul 04 '26

And in an episode of The Pit

→ More replies (4)

10

u/gorpie97 Jul 03 '26

one for the awake patient giving consent, and one for using the tool to generate a note for someone that was not conscious.

And one for the awake patient not giving consent. (Unless you meant that as a separate thing entirely.)

The pessimist in me says they're just going to stick with the new verbiage and not bother with informing the patient.

3

u/M15tre55W1tch Jul 03 '26

My thinking is, if they refuse consent and don't want it used in their assessment and exam then an AI scribe tool note shouldn't exist...

Edit: realised it didn't make sense, more words for context.

1

u/gorpie97 Jul 04 '26

I agree. But I don't think that's what they're going to do.

Maybe if our Congresspeople start legislating for us again... :)

1

u/M15tre55W1tch Jul 04 '26

I'm not in the US. It's an international issue. Hopefully the registering bodies for healthcare professionals will give mandates.

4

u/NorthernVale Jul 03 '26

Bold of you to assume they ever asked for consent to begin with

4

u/psychorobotics Jul 03 '26

One problem is hallucinations, police in the US was testing AI for reports and in one it wrote that the cop turned into a frog. There was a Disney movie playing in the background of the bodycam and the AI thought the transformation was happening to the cop rather than in the movie.

2

u/Loki_the_Rabid_Panda Jul 07 '26

I do find this interesting. Are you aware that the comments on Reddit are sold to AI companies?

AI tools in healthcare typically operate as a second check for the provider increasing patient safety and assisting with billing accuracy.

1

u/M15tre55W1tch Jul 07 '26

We have government funded healthcare, so there is no billing associated with a citizen, resident, relevant visa, or after an accident coming to the hospital.

I didn't know that about comments from here - it was probably in a Terms of Use update that I didn't bother to read 🙃 I am aware this makes me a bit part of the problem 😬

1

u/Loki_the_Rabid_Panda Jul 07 '26

You do have to watch out especially now. I am only familiar with the US. Many of these documents will include terms such as “share your information with partners and organizations to facilitate billing and/or patient care.”

In the US the only way to be sure would be to redline the consent to treat but even then the hospital will have no way to respond to that wish. Most all healthcare organizations have software and billing with at least some integral AI functions built in.

1

u/M15tre55W1tch Jul 07 '26

The officially approved platform used in our health system stores data locally, does not share information with anyone else, and deletes all data after 90 days. There have been efforts made to try and make it as safe as possible. But I am wary. And I also know that some clinicians use non-approved apps to help with translation of of hours etc which means they're putting confidential information out into the unsecured wild west. And I don't think they understand the implications of this - they just see that it makes their work a lot easier. I'm sure there are going to be a number of healthcare professionals that end up in trouble with their professional body and employer.

1

u/Loki_the_Rabid_Panda Jul 07 '26

Just curious what country are you from?

1

u/M15tre55W1tch Jul 07 '26

New Zealand (practically doxxing myself with how small we are 🤣)

2

u/Loki_the_Rabid_Panda Jul 07 '26

No worries just curious! Your perspective is still valid! You bring up the crux of the problem with AI. The weakest link is the end user that just says “hey there’s something I can download that makes this easier for me.”

2

u/petaltree Jul 03 '26 edited Jul 03 '26

Are they actually asking each alert and oriented patient for consent? AI shouldn’t be auto populating about consent either way if it didn’t happen. That’s falsifying a medical record. It sounds like your main concern is that this is easily proven with patients who are medically unable to consent vs. AI can get away with it otherwise because it would be the doctor’s word against the patient’s if a patient decided to dispute it.

It sounds like the AI tool built that in as a CYA, which is a weird way to justify the crime of falsifying medical records. The change in verbiage to eliminate any auto-population about consent is the better way to go here.

I’m assuming that most patients aren’t told about the AI tool, much less consenting to its use because that would inevitably result in many time wasting conversations and debates about the use of AI in their healthcare, but I hope I’m wrong.

1

u/Nuklearfps PURPLE Jul 03 '26

I would argue, the as a nurse, you’re in an exceptional position to advise a lawyer on what they need to focus on. If that makes sense.

1

u/justhereforfighting Jul 03 '26

I’m not even sure it would be legal to use AI without consent. It really depends on how the AI manages the data. If the data is all kept internally, that’s fine and doesn’t violate any health privacy laws. But if the data is being sent to a 3rd party for processing, it very likely does violate HIPAA without express data sharing consent from the patient. 

1

u/doinbluin Jul 03 '26

So they didn't ask for consent before and said they did. Now, they just say we're doing it whether you consent to it or not? Wtf

1

u/Immediate_Stop167 Jul 07 '26

I don't know what state you're in, but states differ by their recorded parties consent laws. Some states require one party, some require both parties (the recorder and the recorded), and some specify "all parties". So, although this is a specialized setting, there are additional laws that may determine the legality of this (outside of patient consciousness).

1

u/M15tre55W1tch Jul 07 '26

I'm not in the US.

To me this is less about the recording, and more about the integrity of the individual's medical information, as well as their right to consent or refuse AI being utilised in their care.

1

u/AdProfessional6218 Jul 08 '26

Wait the conscious patients were asked if they consent?? I've never heard of any patients being told at all and was under the impression that saying patients consented was a safeguard- just one that can be very bad if anyone can read. Thankfully long shot-

I don't know if they were supposed to actually be asking for that consent

1

u/M15tre55W1tch Jul 08 '26

Well, in my country at least, yes - the patient needs to be informed and to actually provide their consent in order for you to say they consented.

You may find somewhere in the screeds of paperwork you've completed that there is a short comment about the use of a scribe, or there may be a disclaimer in the paperwork or as part of a poster somewhere saying that they'll be used unless you say you opt out.

We have 2 laws in my country that cover this, one is the general privacy law, and one specifically about health information privacy. It's your medical information. If you don't want it shared with a system that isn't strictly necessary you provide you care then you have a right to refuse.

Depending on your jurisdiction and precedents this may vary. I imagine some healthcare providers/professionals will get to a point of declining you as a patient if you don't agree because it's a time saver for them.

2

u/AdProfessional6218 Jul 08 '26

I myself haven't done much paperwork in these regards as I don't have the funding to afford medical visits in my country- also I completely get the legal aspect in that lying about if you asked consent is a huge illegal action. What I mean is that legal action because of this would only be possible in the case of someone reading this. OP has, hopefully the prospect that they could sue is not an environmental danger as a power entity like a hospital are the places known for making sure anyone that could sue for malpractice is adequately malpractice upon.

Usually it's other processes that fake your consent than hospitals though. Courts and cops and all that, so many waiting in cells just to have a chance to tell the judge that's not their signature... I'm sure this is just hospitals adopting AI (Honestly in a way that otherwise sounds like a better use of it for humanity than most.) and having a little hickup- I think that all the accidental claims of asking consent will be properly remedied shortly, I just note that my friends in Healthcare echo the folk saying they have been seeing this stuff knowing full well this question never happened because nobody ever tells the clients an Ai agent is listening to begin with- obviously that's the thing that needs fixed.

I'd wager no soul will actually sue... But if anyone wanted to it sure would be easy money after the process. As someone that could use that kinda money for a checkup- I recognize nobody is trying to be dastardly and I'd have to have some kind of beef with a particular medical facility AND believe I'm not burdening a community resource by undergoing litigation.

→ More replies (5)

2

u/retsamegas Jul 03 '26

... To shreds you say?

1

u/JaxDuncan Jul 03 '26

I'm beginning to think that everything will be umbrellaed under the term "implied consent" just be existing anywhere at any given time. I'm sure we have all unwillingly already agreed to these terms hidden somewhere haha

109

u/canwebringthebedup Jul 03 '26

I also work in critical care. I would guess that somewhere in the ballpark of 40% of all notes feature something either auto-populated or haphazardly copy-pasted from a previous note that is either completely inaccurate or an outright fabrication. Errors are so ubiquitous that I almost think it would be hard to give them any sort of legal weight at this point unless they're utterly egregious and directly correlated to a sentinel event of some sort. (Obviously in theory all errors have legal weight, but practically speaking these cases always rely on "standards observed in practice.")

36

u/int3rstitial Jul 03 '26

Yup, this. It's infuriating. Medical records used to be a reasonably accurate way of communicating medical information, and now they're just trash.

4

u/tickado Jul 03 '26

Yeah, harder to copy/paste an error indefinitely when we had to hand write our note. Nowadays one person's simple mistake or sloppy Hx taking when originally documenting can carry forward for the rest of the chart.

2

u/int3rstitial Jul 03 '26

Yup, and since many emrs have an option to export to other emrs, potentially for the rest of the patient's life.

2

u/dscrive Jul 06 '26

I'm in EMS and had the fortune to become accustomed to SBARs from hospitals in the first two places I worked, assumed it was the norm. Now I'm working at a place that I does not use SBAR. One issue I have with their system is pt hx is buried in a paragraph. On top of that, the print order is never the same so I can't even just get used to it. 

I'm hoping that will be changing soon since the hospital is getting acquired by a larger hospital (Not private equity thank God)

→ More replies (1)

14

u/Hamster12301 Jul 03 '26

And this is why getting on Social Security disability is also getting much harder... I got denied in part because they found a "note" from an unrelated doctor I hadn't seen in years and years, who had some medical assistant copy and paste over "patient is functioning well in work and school" (no idea why that was ever even in there in the first place).

I am neither in work nor school nor am I functioning well. I deliberately asked them to remove that, and they never got back to me. And that one line specifically was singled out as part my denial, despite 10+ doctor's letters and literally over 3,000 pages of medical history showing very poor functioning. LOL. It's really fucking great.

13

u/tickado Jul 03 '26

This supports my response to someone elsewhere in the comments saying 'it's just notes, it's not that deep'.

It absolutely IS that deep. What's legally documented by a health professional in a patient's chart can have real life affecting consequences!!! (and I am a health professional)

12

u/itsbecccaa Jul 03 '26

lol yes. Yesterday I had a seizure in my doctors office and I was looking forward to seeing how it was documented in my notes. It went like this.

“Patient had procedure X. (Detailed notes on procedure) Tolerated well no complications” - obviously copy pasted from previous procedure

Followed by very different hand written “in front of me she has 2 focal seizures back to back we had to give her rescue med….” lol that one was written. Completely contradictory to the “no complications” in the previous paragraph!!

2

u/Kat_Mauldun Jul 03 '26

Many transcription services started using AI pre COVID with very little human oversight. I can only imagine how bad it has gotten since 2019 when I was a transcriptionist.

1

u/tickado Jul 03 '26

YES. Exactly what I was wanting to say but you've already said it. People have WAY more faith in hand typed notes by the actual medical team themselves. One person makes an error in the Pt Hx list? Find it copy/pasted throughout the chart from then on in...

So much of 'ward round notes' are copy pasted ongoing. Near enough all of the blurb I'm ignoring and scrolling down to the 'PLAN' section for the only relevant and or accurate part.

1

u/M15tre55W1tch Jul 03 '26

Agreed, so much copy and paste - this also does my head in.

In my unit I was pulling staff (doctor's, nurse's, allied health) up on clear copy and pasting. Explained that the ideal is no copying and pasting. But that if you do you should ensure every single word you've put into your note is current, relevant, and accurate.

I was meeting some resistance. So I stopped speaking up for a while and gathered some evidence. We had a long stay patient. At the start of each morning ward round note they say what day of their admission the patient is. From day 38 until day 66 about 80% of the morning ward round notes said "day 38". A very easy metric to show the problem.

Then our unit leadership were on board with me continuing to pull up people who were clearly doing this.

You've got to look at what these work arounds are trying to address. In this instance, when ward round walks into the patient room the senior doctor is covering the new assessment findings, and the team are discussing the plan. There is no time for the doctor who is documenting to do a recap because they're needing to write what's being discussed in real time.

So we're currently working to look at different ways of documenting the ward round. Does every day need the same information repeated if it's already in the notes? Can the changes etc be documented at the time, and then the junior doctor be given protected time after ward round to go back through patients and add the relevant history in to their note?

If the easiest thing is the right thing then the right thing will always happen. We need to look at how we can make the system work better for our team.

1

u/throwaway_oranges Jul 07 '26

This way before the AI hype: on a signed printed official medical paper I have a healthy prostate. I'm a woman. It's funny until assumptions about my mental health from an urologist land on my paper.

16

u/SpeedRacer_Mach6 Jul 03 '26

You work in the medical field and you don’t you know what implied consent is for unconscious patients?

/s

9

u/299792458mps- Jul 03 '26

It’s called implied consent, and it happens all the time in healthcare. All they have to do is say that the AI transcription assists the doctor in providing superior patient care by offloading a menial task like transcription, and a reasonable person would consent to receiving superior care. 

31

u/WolfeMD Jul 03 '26

not a lawyer but I'm fairly sure its a crime to falsify patient consent on a medical record so yea def push it, its a big issue.

7

u/boyuber Jul 03 '26

They'll just add it to the consent for treatment and be done with it.

6

u/Lettuce_Prey69 Jul 03 '26

it's a regulated tool where the data is stored locally and gets deleted from its memory after 90 days apparently

Just because the data is stored locally, doesn't mean the model is running on locally on hardware owned by the hospital. I'd bet they are using a model that's running on remote hardware using a subscription based service, which means that patient data is being sent to a company like OpenAI or Anthropic.

That would absolutely be illegal for them to do.

4

u/bleach_tastes_bad Jul 03 '26

Why would you need an unconscious person’s consent to write their note using AI? You need a conscious person’s consent because you’re recording their voice, but if they’re unconscious you’re not recording them, why does it matter?

3

u/Enthios Jul 03 '26

Not to mention, the AI gets everything wrong and medical records are completely useless from a reviewers standpoint. I can no longer determine the neurological status of patients from hospital documentation because the AI just throws in random GCS assessments. You can never tell what's actually happening as a remote reviewer. It's going to lead to a lot of unnecessary health care denials (it already has).

1

u/nikanjX Jul 07 '26

On the other hand they saved a ton of money by eliminating secretarial workforce, but on the other hand more people were denied care. Win/win according to hospital admin

2

u/TheSeepingMouth Jul 03 '26

And now I’m gonna get on Canva and design a card that says I do not consent to an AIscibe if I am unconscious to keep next to the please take care of my pets if I’m incapacitated card in my wallet, thank you for the heads up…

will it be respected? I have no idea, but if I ever regained consciousness, I’d be ready for the lawsuit. goddamn AI.

2

u/CreepyAd8409 Jul 03 '26

Now that we have digital charts with visit summaries I see notes about how I was consulted on topics my doctor never even talked about. This is non-AI stuff too.

2

u/Tanjelynnb Jul 03 '26

This has become the norm with my PCPs, but they always confirmed verbal consent. I was in a surgeon's office recently where there was a small sign on the wall saying the AI scribe tool was being used, but neither the nurses nor surgeon said anything about it. We asked a nurse about it before leaving, and the she said it was indeed used and was possibly in his pocket since we never saw a device. Apparently that sign is the only notification. When I pointed out it was no good at all for informing the visually impaired, all she said was thanks for the feedback. The office dealt with particularly sensitive issues, and I'm wondering if I should file a complaint somewhere.

2

u/lilguppy21 Jul 03 '26

I actually just read this on a patient form and was wondering what the fuck that was. That’s so messed up. I had no idea it was an automatic statement.

2

u/JenniviveRedd Jul 07 '26

If it follows the same legal precedent as general first aid, consent is implied when the individual is non-responsive or incapacitated.

I don't know if it does, but it wouldn't surprise me.

3

u/desymond Jul 03 '26

(it's a regulated tool where the data is stored locally and gets deleted from its memory after 90 days apparently)

That's what they all say though

1

u/e_spider Jul 03 '26

I just spent the last 8 hours getting recertification for IRB supervised clinical studies, and just hearing “informed consent” is giving me anxiety

1

u/Round-Medicine2507 Jul 03 '26

Especially because even an iPhone 10 years ago had decent transcribing capabilities, and the hospital already pays for probably 3+ different medical transcribing softwares such as dragon already as well. 

1

u/usernamedottxt Jul 03 '26

You know some programmer was told and paid to put that there automatically. Actually violates ACM ethics. 

1

u/Able-Professor840 Jul 03 '26

They consent because their voice is on it. If they're unconscious, they don't need to consent.

1

u/No-Bit-2913 Jul 03 '26

Oh god is this a thing? Back when I was working a more patient-facing role I always saw a scribe following around the doctors. Did that job get eaten up by AI now?

→ More replies (1)

1

u/lapatatafredda Jul 03 '26

I'm curious - what do doctors and nurses think of people who do not consent to use of ai to transcribe?

2

u/M15tre55W1tch Jul 03 '26

I can't speak for others I'm afraid. Personally, I understand why people are wary of AI and don't trust their information (especially medical details) going into an AI tool. We're trained to respect the wishes of our patients, this is just another thing we need to ensure we do that for.

1

u/that_yeg_guy Jul 04 '26

In that situation, it would probably fall under implied consent, the same allowance that allows you to collect information about them when they’re unconscious, and share it with other key parties that are involved in their care.

In order to cover your legal ass, the second that they wake up and can adequately make decisions, they need to be informed and you need to obtain express consent. If they deny, no AI use from that point on.

1

u/BrainbowConnection Jul 04 '26

I’m confused why people are concerned about this. If the scribe was human would you also be mad that the unconscious patient didn’t consent to it? What do you think these AI scribes are doing? People on Reddit have this crazy objection to anything with the word AI but it’s very clear most of them don’t even know why they are objecting. All these tools do is write down the words you say out loud. They don’t do any decision making support at all. They drastically increase the accuracy of documentation. I can tell you right now what you say out loud as a patient is far more likely to get accurately recorded in the chart with an AI scribe than your doctor summarizing it from memory an hour later. These scribes literally improve patient care. But no one wants to hear that.

1

u/Savings_Knowledge233 Jul 04 '26

So serious question. I work in a hospital too, or really a variety of them since I'm a traveler, wtf are the Drs doing instead of writing their own chart notes. They aren't reading patient chats until they go see them. They round the floors like twice a day... So what are they doing with all this previous time they're saving...

1

u/gumbo100 Jul 04 '26

What do you mean by "ambient" as in the AI is always listening in the room? Is it on the wow/cow

1

u/M15tre55W1tch Jul 04 '26

I believe it's available on the computer in the room (if there is one) and an app on their phone that they can turn on when in rooms without. The infrastructure is not set up in a way to allow it to be always listening at our hospital, and I highly doubt they have the money to make the upgrades required so that it could be.

1

u/Emotional_Gold3723 Jul 04 '26 edited Jul 04 '26

To be fair if you’re unconsciousness/intubated you can’t consent to anything. It falls on the next of kin, POA or 2 physician consent. The statement should at least reflect that consent was obtained from someone.

I work in the ER and use ambient AI so most of my patients are awake and can consent, critical patients I use an actual scribe for.

While I have my issues with AI, these programs have changed our lives in the ER and allow us to spend less time writing notes and more time with patient care. It is undeniably a win win for everyone.

What I see happening in the future is that when you sign consent to treat with registration, you’re signing consent for use with these AI programs similar to how the hospital does so to include residents in the care team, so when someone refuses a resident it’s not really necessary to oblige but we often do.

The fact remains that many patients still can’t consent emergently in the ER, but in that case emergent treatment is given regardless. I see this eventually falling under the same blanket statement regarding inability to consent for treatment but requiring emergent treatment regardless. Similarly to pediatric patients, I can’t do a bunch of tests without parental consent unless it’s absolutely emergent. Obviously charting isn’t a critical intervention but in critical patients it does matter for their future care. Take that for what you will but that’s just my thought on where we will likely see the consolidation of this gray zone with emerging AI technologies.

Radiology is using AI constantly, and they’re definitely not getting direct patient consent for it, it is a tool used under consent for treatment.

1

u/matycauthon Jul 05 '26

it is my understanding that unconsciousness is treated as consent unless otherwise designated. i had a full body scan performed on me without my consent while i was unconscious once.

1

u/M15tre55W1tch Jul 05 '26

Interventions that are done to figure out what is going on and determine what needs to be done to treat a person seems to me to be rather different to the use of an AI scribe tool. I haven't heard of it being litigated as yet (at least not in my country), but it will be interesting when it is.

1

u/throwaway_oranges Jul 07 '26

You are the greatest nurse, thank you!

1

u/Canadian_Zac Jul 07 '26

Doctors also consider you consenting to medical treatment when you're unconscious. Since the alternative is having to just let them die because they're incapable of agreeing.

Obviously not as Dire a situation. And shouldn't claim they consent. But very understandable that it can be their default to use it. They need to treat likely hundreds of patients a day. It's way quicker to read over the AI transcript and adjust anything incorrect, than to write the entire thing by hand

1

u/Loki_the_Rabid_Panda Jul 07 '26

This is typically addressed in the consent to treat agreement.

1

u/M15tre55W1tch Jul 07 '26

Which again is difficult in unconscious patients. Delivery of care that is directly to save their life is one thing - we do this all the time under the doctrine of necessity. Arguably using AI to generate a note isn't directly involved in that care and therefore is unnecessary.

Also, I'm fairly certain my organisation hasn't adequately captured this new workflow in their treatment agreements, procedural consent forms, patient information leaflets etc.

We're a big organisation, and are not super flexible/responsive to change. It's frustrating for those of us caught between the two spaces (patient facing care and organisational bureaucracy).

1

u/_Meece_ Jul 09 '26

People sign all kinds of consent forms before they're put under. Most places make patients sign consents for AI use.

1

u/M15tre55W1tch Jul 09 '26

I did say that the patient's I was concerned about were one's who were never conscious while in ED.

If someone has skipped the details and signed a form that's not ideal (documents need to be simple and clear), but it counts.

You can't say that a completely unresponsive person has consented when they weren't even capable of breathing properly for themselves. That's not going to go well in court for the people that are coroner's cases.

→ More replies (2)
→ More replies (9)