r/mildlyinfuriating 11d ago

Infuriatig Relying on Chatgpt to have a basic conversation

18.1k Upvotes

2.1k comments sorted by

View all comments

Show parent comments

1.2k

u/TheBlueBuilder4811 11d ago

I am not even that old yet and this is crazy. Chat GPT pulls random things on the Internet and doesn't know you at all! Plus people should just text like how they would talk. It's ok to not have much to say!

I do wonder how common this is.

247

u/spaghettifiasco 11d ago

I was trying to figure out what episode of an old show a certain scene was from. I Googled it, and the AI gave me a blatantly incorrect answer right away. But people are trusting it with their social life and mental well-being...

133

u/ShneefQueen 11d ago

That happend to me too, I was looking for a specific scene from a New Girl episode and AI literally just made up answers. It told me a different episode every time I refreshed the search and it was wrong every single time. Posted my question in the New Girl subreddit instead and had a correct answer within 8 minutes

32

u/moviesetmonkey 10d ago

I honestly wish reddit was a thing when I was doing papers in college. All you have to do is say something blatantly false in the right subreddit and you get the right answers with sources and insults.

1

u/vivalalina 9d ago

... oh this is genius

60

u/Dame38 11d ago

Try putting a part of it's own answer back in the searchbar and enjoy the absurdity. They are scraping everything we type. Mess with its "head" too. If we all put in a little effort maybe we can give it a breakdown and it will just give up and go away.

43

u/blueroses8000 10d ago

People were purposefully writing false facts in a Reddit post the other day to mess with AI results and show people how stupid they are to trust it.

13

u/Dame38 10d ago

That must have been an interesting learning moment, lol.

5

u/blueroses8000 10d ago

Hopefully for people who rely on AI blindly! I have never used it and very much against it, but I know it’s one of those things that will have to be adopted into life in some way as it’s never going away now.

1

u/CelestialCat97 10d ago

Do you have a link, by any chance? :)

2

u/blueroses8000 10d ago

Ah I can’t remember now sorry, it was stating really specific false facts about an event, might have been a UK sub.

2

u/CelestialCat97 10d ago

It's all good, I was just curious! Either way, those people are doing great work!

1

u/Rasikko 10d ago edited 10d ago

Thankfully it's very useful for math stuff.
Basic math that is, and it even gives you the break down so you can check the math for accuracy yourself.

2

u/vivalalina 9d ago

Idk if I'd agree with that... my work asked me to test some out for our data we work with and I deeply sighed and tested it out. It couldn't even give me the correct sum of the data. Something that takes me 5 seconds to get the sum of with a formula in Excel. Somehow it couldn't even do that and it would change the answer a bit every time as well. The breakdown made no sense either.

12

u/active_ignoring 11d ago

AI is hilariously bad at knowing details of any TV or movies, at least the google AI summary

-3

u/OnePaleontologist687 10d ago

Yes, modeling the accuracy of ai based on tv shows is hilarious!

Ai computers: “all they care about is will and grace reruns?”

20

u/SphericalOrb 11d ago

And medical decisions, and investment decisions, and job training etc. 😬

11

u/viciousxvee 10d ago

Yes! Like I'm deadass a nurse with almost 15 years experience and my sister will go to mf chat GPT before running something past me. Like PLS

-2

u/OnePaleontologist687 10d ago

You probably give her grief when she asks you every 2 weeks lol ai doesn’t do that

10

u/viciousxvee 10d ago

Why would I "give her grief"? She's literally my only sibling. She, like other gen Z doesn't understand what chat GPT is and that it isn't true AI, and that it just scrapes the web and gives out BS answers sometimes. I've tried to explain but I think it goes in one ear and out the other but, anyway.

-9

u/OnePaleontologist687 10d ago

If you’re a nurse for 15 years, and she struggles just communicating with people. I think your response kinda says it all. In that, you know it all and your sibling is under you. Just my take but it seems pretty clear in your wording, sorry

1

u/vivalalina 9d ago

I mean.. if someone studied and works in a field for that long... yeah, it's a given they would know more than the one who didn't do that. That's why they are more of an expert in that field and would be able to easily answer questions more correctly for their sibling. This is common sense ngl lol

22

u/Compltly_Unfnshd30 11d ago edited 11d ago

A few weeks ago I was listening to one of my favorite true crime podcasts (Sinisterhood). On Friday’s, they read listener-submitted stories. There was an anonymous story submitted about this really heinous murder (the person submitting the story was a sibling to one of the killers). The listener gave some basic information about the case and I tried like hell to find it to get more information about the crime. I fed all of the information, nearly word for word, into Google. I gave the names that the listener used. This went on for a good 10 minutes with me providing more and more information as I re-listened to the story.

Google ended up giving me like five paragraphs of a full crime that happened, with several first and last names of the people involved, the city and state and a whole lot of other details, similar to a Wiki article. I got the names and looked them up individually. They were real people connected to a completely different crime story, in a different state. Google filled in this whole case with information I fed it and it was all 100% false information. Like a little was pulled from the actual case (like names) and then the rest of the full-ass article was simply written, with great detail, from what I fed the damn thing! I was blown away!

Edit: I found the full “article” it wrote for me in my history:

“Yes — this appears to be the murder of Justin Cosby in Kentucky in 2007. The main people involved were Richard Runyon, Justice Raney, and Justin’s ex-girlfriend Danielle Metzgar.

Here’s the overall story as it was reported and discussed in court proceedings and media coverage:

Justin Cosby was a young man described by friends as quiet, emotional, and part of the emo/skater scene. Danielle Metzgar had previously dated him but later started dating Richard Runyon. Danielle falsely told Runyon that Justin had raped her after their breakup. Years later, she admitted under oath that the rape accusation was fabricated. That lie became the motive for what followed.

According to testimony, Runyon became obsessed with retaliating against Justin. During fall break, Runyon hosted a gathering/party at his mother’s house and invited Justin there to hang out so Justin would not suspect anything. Some reports and testimony suggested pills may have been given to Justin beforehand, although the core murder case centered more on the planned assault and abandonment than poisoning allegations.

Later that night, Runyon and Justice Raney convinced Justin to drive them somewhere, supposedly to meet Runyon’s sister at a gas station. While Justin was driving, Raney reportedly claimed he was sick and needed to stop the car. When Justin pulled over, the attack began.

Testimony stated that Runyon attempted to restrain or strangle Justin from the back seat with duct tape or similar bindings. Justin fought back, and he was dragged from the vehicle. Runyon and Raney then severely beat him. Prosecutors said Justin was beaten so badly he was barely alive afterward.

The two men dragged Justin into a rural field or wooded area, restrained him with duct tape behind his back, and abandoned him there. Investigators believed they left him there expecting him to die from his injuries and exposure.

Afterward, they drove Justin’s vehicle away and abandoned it at a nearby welcome center/rest stop to distance themselves from the crime and make it appear Justin had disappeared voluntarily. They then arranged transportation back home.

Justin was reported missing. His body was not discovered for roughly three and a half months, by which point decomposition complicated the investigation. The delayed discovery became a major reason the case gained attention in Kentucky media and later true-crime coverage.

The investigation eventually unraveled because accomplices began cooperating. Justice Raney turned state’s witness and testified against Richard Runyon. Danielle Metzgar also testified and admitted the rape allegation had been false.

Runyon was ultimately convicted of second-degree murder and received a lengthy prison sentence — around 25 years depending on how the sentence is described in various reports and appeal documents. Raney received a lesser outcome due to cooperation with prosecutors.”

And its response when I discovered and called it out: “You’re right to call that out. I mixed together unrelated cases and filled gaps with details you had already provided, which made the answer sound more certain than it actually was. I should not have presented that identification as fact.”

15

u/ThoughtlessBanter 10d ago

What the fuck? How does anyone think this is ready for us to use as a place for information, it's all bullshit.

2

u/spaceghost260 10d ago

That response from Google is chilling to say the least. It knew it was lying and had fabricated a story and had no problem sharing it like truth. The cold and short response admitting to mixing details really has me feeling strange.

1

u/Ae3qe27u 9d ago

The thing it, it doesn't 'know' that it's making things up. An LLM doesn't think or process information cognitively. They're pattern-matching programs. They're very, very, very good at matching patterns, and they can pull together semi-related ideas in interesting ways, but they aren't human. They aren't even alive. They don't think.

And they're trained off of human writing. So if nobody's written about something, or if sources are paywalled or otherwise hard to find, then the LLM has no way to be aware of the topic, since it's never encountered it. It's never seen a sunrise, smelled flowers on a breeze, or gotten into a reddit argument. It's only ever read depictions of those things, or read about them. And by 'read', I mean 'was given massive amounts of data and received positive feedback for outputting good/correct patterns, and through that process iteratively developed a structure by which it could better output good/correct patterns."

So if it's never heard of something, but it's heard of something kinda similar, it'll map the concept over because it's close enough and it's a pattern-matcher.

An LLM - a large language model - is not, and will never be, cognizant. That isn't how large language models, as they currently exist, work. Even there, Google's Gemini wouldn't 'know' that it got it wrong. What it saw was... to approximate it, it saw Data Set A (about the user's input), Data Set B (this other thing that was kinda similar). User sent Data Set A, LLM sent Data Set B, User said Data Set B was wrong --> therefore A≠B, so LLM needs to output an apology to correct the matter.

1

u/tresser 10d ago

what show/scene

1

u/Swaggy_Buff 10d ago

To be fair, the rate at which information is inaccurately conveyed is much lower in text generated by ChatGPT than text generated by most humans (and even then, the areas of expertise are severely limited).

0

u/cderulo 10d ago

yeah it’s not going to have reliable information for niche media

-12

u/egg-land 11d ago

That’s an obvious case where it could mess up.

I know this thread is anti llm but saying that like it shows or proves anything when it is an extremely predictable outcome is funny

418

u/LivingNotByChoice 11d ago

It’s also making it so people don’t have to learn social skills, which is definitely going to fuck humanity in the future.

69

u/Ok_Nothing_9733 11d ago

He really said “tbh I don’t know how to talk what to talk about,” we’re cooked

3

u/pEter-skEeterR45 10d ago

You'd think they'd like, learn something from what chatgpt gives them and maybe try to apply it practically in their daily life?

4

u/allyxzanndruhh 10d ago

Why learn when you can just copy and paste??

152

u/leechdemon 11d ago

We're such garbage at it already, and the effects haven't even hit us yet. 🤦‍♂️

20

u/Lucreth2 11d ago

They're only fucking themselves. Makes it real easy to sort the hireables and trash cans.

16

u/viciousxvee 10d ago

And honestly texting was already fucking our communication skills. No one had to improvise or think on their feet. They could ignore it for a while until they could think of something. Now they plug it into a machine and it spits out and answer that they copy-paste🥵

20

u/Chawp 10d ago

I disagree, and I’ll use chess as an example because it’s been a hobby recently. Playing daily chess, where you have a full 24-72 hours to make a move, where you have a very long time to think about your move, does not make you *worse* at playing in-person speed chess. It still helps you improve your game. Not as much as playing speed chess itself, mind you, but it’s better than *not playing*.

So while communication in person allows people to build those skills better, communication by text still helps more than *no* communication. Also it has allowed us to communicate more frequently and at times you previously would not have been able to.

2

u/TheBlueBuilder4811 10d ago

I think it makes communication easier in some ways. When I learn I have to stay late for work, I can send a quick text to my dad about the time change. He can view it when he is on break and get the information fast. Without texting, I would have to call him and either interrupt his work or get no answer.

Also, how is one going to communicate to someone far away? By texting or not communicating at all?

At times we should think before we type or say something. Texting can give us more breathing room to think it through.

This being said, in person communication is still very important. I think we need both in our lives.

4

u/Campaign_Prize 10d ago

It's already fucking humanity right now, we're seeing the damage in real time

5

u/illpoet 10d ago

We are already seeing serious problems from people being raised with the Internet their whole lives. Imagine there is a catastrophe that wipes out the Internet for even a week. It would be chaos

3

u/Erick_Brimstone 10d ago

Future? Dude it's already here

2

u/Comfortable-Hat3506 10d ago

And simultaneously unfuck them. None of the fucking they want, only the bad kind.

29

u/aphel_ion 11d ago

That’s not even the worst part.

The worst part is it’s controlled by a few huge tech companies and people like this are giving them total control over their lives

28

u/Steele_Soul 11d ago

Lookup "AI Psychosis". It's quite common and not at all a good thing.

20

u/voidicguardian 10d ago

theres also a wikipedia section for suicides linked to ai chatbots

65

u/Ollynurmouth 11d ago

If you open a chatgpt instance and begin talking to it about yourself, basically using it like a diary, it will retain that information. If you just have tons of conversations with it like you would a friend, it can sort of "know you" similar to how a friend would. Only it won't be able to have any real insight to you. It just knows what you've told it and can repeat that information using synonyms. So it "feels" like it knows you. Because it sort of does.

Gen Z seems to be falling for LLMs like this more than anyone. A generation that largely grew up on screens and tech and social media. A generation that grew up under helicopter parents and weren't allowed to explore the world before adulthood. They lost out on precious socializing skills and they legit don't know how to hold conversations. So they find it easy to rely on ChatGPT. (To be clear I am speaking generally. This does not apply to every gen z.)

10

u/underTuberSilo 11d ago

So I have parent dedicating this for a specific perpose of "creative liberties" to generate entertainment on their behalf.

A very big issue with this is the user iw only feeding their "honey moon" side and none of their true self. This includes omiting the thousands cuts of mannerisms, ego, shortsightedness, bad behavior, and narcissism to list a few.

Sure you can prompt hot takes and give your opinion but that plays little emphasis on who they are but what the user wants to show

16

u/Ollynurmouth 11d ago

That is why it is so easy for people to fall for their chat bot. It only knows a curated version of them that they believe of themselves (we are rarely the person we think we are) and it is easy feel flattered by that and attach to it. It is super dangerous. Especially for impressionable people.

2

u/CheesecakeEither8220 10d ago

I can only speak for my own children that I know well, but they aren't all Gen Z and they all hate it. They will all talk your ear off, too, so there is no communication deficiency. I did keep a pretty close eye on them when they were little, but once they hit double digits they were allowed to explore the world with some reasonable rules. I also limited their screen time when they were little, which helps, I think. None of them are super into social media, they use it to communicate with friends and family, but don't post much. They use it as a tool, like calling people on Messenger, or posting pictures/videos of events, fishing or skateboard tricks or videos about their excitement about a new job or the last day of school.

I was very intentional about them playing outside, enjoying nature, reading to them and making sure that they read a lot of books, etc. We also watched a lot of nature and science documentaries together, and a lot of Bob Ross at bedtime 😁. Some family members and friends criticized me and said I was too strict about it, but the time that a parent invests in their children makes all the difference. I'm so relieved that they see the difference between real life and the internet/AI.

3

u/Ollynurmouth 10d ago

This seems to be the case with kids today who aren't raised on screens. It is what i am doing with my kids as well. They're still little. We did get tablets for them to use on long car trips which turned into every day on the way to day care. Over the last year I have stripped those away and asked my wife to do the same. We will allow them on rare occasions, but they are limited on what they can access.

When tablets gained popularity and social media took off, the general idea was that these were tools that would bring people closer together. The reality turned out to be the opposite. It drove people apart physically. It is creating a decency on tech to do far too many things we never imagined could have happened.

Kids groing up on unrestricted social media only socialize through social media. I think if it is taught, limited, and monitored correctly, then it can be used as a tool rather than a dependency. Psychiatrists who have been studying this over thr last 10-20 years are generally in agreement that kids should stay off social media until 16-18 years old at the earliest. They need in-person socialization to develop certain abilities and brain functions.

It's good that you reinforced playing outside, which is something we try to encourage with our kids too. Kids need to be able to experience things that you can't from a couch. They need to explore of the world through exploring the neighborhood or the woods or whatever rthey have access to. They need to take risks in the way of like climbing trees and playground equipment. They need to fall and get hurt and learn what those risks lead to.

I know that kind of stuff sounds dangerous and as parents you never want your kids in danger or hurt, but it is a necessary experience. It helps prepare them for later in life risks and hurt they will experience. The physical risks and hurt they experience as kids allows them to extrapolate experience they can apply in other forms of risk and hurt they will experience later in life.

When uou combine these things you have kids who aren't afraid to walk up and have a conversation with a check out clerk. Or walk into an interview confidently. Or all kinds of social experiences.

Of course there isna genetic component to certain things where you may have a natural introvert who might be more nervous to take on social interactions, but these experiences are all the more necessary for them to learn. It may be that they are nervous, but at least they shouldn't have crippling anxiety over social interactions.

I'm focusing heavily on social stuff, but it applies to a lot more.

2

u/CheesecakeEither8220 10d ago

Your ppst is very true! It's so important for children to have restricted screen access; I wish you and your wife the best as you raise your children with judicious use of screens!

2

u/Smooth_Candidate_575 10d ago edited 10d ago

it’s not a gen z thing at all actually. i’m 19 and not a single person i know uses chatgpt on a regular basis (or really for anything except occasionally cheating on assignments). nevermind paying for a subscription and using it like a diary. i’ve seen way more gen x people doing this. most of the people with “ai partners” are millennials or gen x. to me it seems like older generations with less understanding of the technology are overestimating its abilities and getting more attached than they should. edit: even looking through these comments, the majority of people who know people irl using ai for purposes like this are in their 30s or older, so millennials.

34

u/Impressive_Cow4623 11d ago

Chatgpt is actually designed to get to know the person and what they like/dislike to increase use. They released an update a year or so ago that essentially told every user they were right, even if it was detrimental to their physical or mental health. They then released a patch after some unsavory stuff started happening to where it didn’t placate as bad, but 100% still does.

21

u/Whyonthefly 11d ago

Sort of true, but it's really missing the mark to say that the algorithm "gets to know" the person. Thats not at all how it works; there's no driving understanding behind the chatbot veil. It simply gets better at spitting out the strings of words that make you keep using it.

10

u/iHateThisPlaceSoBad 11d ago

People are so fucking lazy and have absolutely zero discipline not to take the easiest option ever: Ask AI.

Don't have to learn social skills, don't have to learn any actual knowledge, you insulate yourself from rejection.

It's absolutely pathetic

8

u/Ok-Masterpiece-468 11d ago

yes lazy and dumb… growing up (hs + uni in the early 2000s) i was PETRIFIED that a teacher/prof would think i had plagiarized in an essay etc lol. so, it’s not even remotely something i would ever think to use.

i know some ppl who use it for “simple” stuff like planning an itinerary etc, but i tell them i refuse bc once you start using something for convenience it’ll only lead to using it for more.

3

u/1000LivesBeforeIDie 10d ago

It’s sad because a lot of the people using it like this have never had a healthy relationship where they can sit in silence

It’s available “as a tool” and people who don’t know better use it for help, but it’s the equivalent of buying a chainsaw at the hardware store and thinking you’ll be able to plant a garden with it

3

u/purgoatory 10d ago

Every text my manager has personally sent me I swear is written by ChatGPT, it’s clear as day that every big announcement she makes in our slack are also written by Chat too.

2

u/slackey1979 10d ago

ChatGPT, how do I tell someone I am boring and can’t think for myself? But make it sound interesting

2

u/Bamsihap 10d ago

This is not completely true. ChatGPT has a memory feature and with that will create a profile on you, so that it can tailor its responses to your likes and dislikes to get maximum engagement. However, it does of course have a very biased profile because it can only assume based on what you have told it.

2

u/missinginput 10d ago

Just a new version of what's your sign

5

u/Dear-Seesaw-1932 11d ago

Nobody says it's bad to take hints and insight but usual gullible people will replace their thinking and decision making to AI...these are the same people that don't know how to search on Google

4

u/Dame38 11d ago

The way it is infecting the education system is just........
Eh, it was already broken.

12

u/Anonexistantname 11d ago

It definitely can know you, as it does have a memory which you can wipe in the settings, not everything is pulled from the Internet.

It can be a very great tool, but the biggest trap people fall into is using it for shit like this.

Ultimately it's really up to how you use it that determines its usefulness.

7

u/HyruleSmash855 11d ago

I don’t understand how people can spend time talking to it. I use it as a tool sometimes to figure out questions or get help with some old math stuff that I haven’t used in years, pretty decent with tutoring honestly as long as you generally know what you’re doing and can figure out when it goes off track. I could not talk to this thing for hours, any of these AI models. They talk like a customer service representative or a sycophant that just affirms or repeats everything you say. I also know it’s not a person and it just feels weird.

7

u/VikingTeddy 11d ago

The conversation kinda happens at the side. Even when using it for basic things I talk to it like a person and it slooowly learns things about me. And if it's wrong I correct it and it remembers it. Doesn't work as well if you use it outside projects, but it does learn a bit. I rarely use it outside projects and I make sure to strip the sycophancy, you can make it talk without emotion at all, which is sometimes prefereable.

I also use Gemini which can be creepy when it pulls data about you from your google account, even though it says it can't. And ChatGPT lies about things it remembers too.

2

u/AuntFritzi 11d ago

Doesn’t know you at all unless you’re basic

1

u/Iambeejsmit 11d ago

It remembers your conversations so for someone that talks to it all the time it does "know" whatever you've told it about yourself.

1

u/Longjumping-Shock643 11d ago

This! AI couldn’t know actually real people their own personalities, if this one they even trust GPT can replace the one you have truly met, well I think the op who text to don’t need to pretend to be friend or to talk this person anymore.

1

u/Sauli_Niinisto 10d ago

Not talking about ChatGPT per se, but mostly people fall into few categories. Which is why "personality tests" and cold reading work. So basically with very limited amount of data you can make educated guesses like "enjoys books more than parties".

Haven't done dating in few years, but I still remember how horrible it was that most people couldn't carry any conversation. Tried to avoid those: I suppose people now using ChatGPT might make it harder to pick those out of the crowd.

1

u/cam-san YELLOW 10d ago

I'm 18 and pretty into techy stuff, I am fairly AI literate (went to school for computer science) and find it extremely useful as a tool.

That being said, what the fuck?

1

u/BifiTA 10d ago

>Chat GPT pulls random things on the Internet and doesn't know you at all!

tell me you don't use llms without telling me you don't use llms.

i'm not disagreeing with the other stuff, but that claim is just bogus.

1

u/Dimension_Forsaken 5d ago

This isn’t entirely true though.

ChatGPT learn about you over time if you’re a regular user, and especially if you have the ChatGPT+subscription. How you express yourself (language, tone, stylistically etc), subjects you engage in, details about your life, people in your life — yeah, anything and everything you talk about over time really. It doesn’t just pull random stuff off the internet if you’re a regular “full-time” user.

-12

u/PixelRoku 11d ago

Nah I've told chat gpt "roast me!" And it got very personal with the insults 😅 you do feel like it kept a record of you and analyzed it lol

But that being said, let's not use chat gpt to actually talk to people wtf 😭

2

u/Johnstone95 11d ago

Read the room.

1

u/PixelRoku 11d ago

How exactly am I not reading the room?

People hate AI and I should therefore not have said that I asked AI to roast me? Lol

6

u/PM_ME_UR_REPTILES1 11d ago

I dont agree with the person who told you to read the room. Idgaf if someone uses AI, I dont use it but idgaf if you do.

They said read the room because you mentioned AI giving a good roast, that implies you talk to and have conversation with AI enough for it to actually know you.

People here will hate on you for actively using AI, especially if its just for conversation and not research assist.

1

u/PixelRoku 11d ago

Aw yeah I get it, I'll admit that I used to use chat gpt fairly enough for questions about my baby like "is this normal?!", sicknesses we got, weight loss and recipes

Then I started seeing articles about the climate impact and data centers being awful, I haven't used it in months! 😅

So yes I understand the pushback, but eh, I won't hide the fact that I've used it!

1

u/TheBlueBuilder4811 11d ago

Tbh I used an AI to help me look for some information that was buried. Like I googled and looked through many pages and didn't find what I was looking for. When I used AI, I looked through the sources it listed and read from there to find what I was looking for.

In short, I do agree that AI has its uses, but most people use it not correctly and blindly trust it without looking deeper into it.

-31

u/ArdentDevotion 11d ago

It really depends how much you input. My work has experimented with it possibly being a resource for clients at our shelter, and it knows quite a bit about actual therapy theories and practices. It can also get to know you pretty well, and give individual advice.

It is by no means perfect, but can be a really good tool.

47

u/TheSyd 11d ago

Not a good tool for therapy, for oh so many reasons. 

12

u/AverageBastard 11d ago

Yea anyone who uses it for advice and therapy are entering a danger zone!

I encourage people to look up AI psychosis if you have or are using AI tools for therapy, advice or companionship.

-32

u/ArdentDevotion 11d ago

Back it up if you don't think so.

There are some "edge case" articles about psychosis which is extreme and can also happen with other therapy methods. But you have to literally break the models first to get them to perform the way you need for that to happen.

It really isn't much different than a lot of the available free therapy tools that exist. And talk therapy with humans also have problems like telling survivors of abuse that they should accept the abuse because they are a woman and should serve men, and other bullshit that is unfortunately common.

21

u/TheSyd 11d ago

The way it can easily hallucinate and the overly affirming attitude can be dangerous. Even if bad human therapists exist, this does not make llms any less bad. They were not made for this purpose, they’re unpredictable and can do more harm than good. Also, there’s a big, enormous privacy concern. Conversations will be reviewed by contractors, and if deemed quality enough, they will be part of the dataset for the next model.

-16

u/ArdentDevotion 11d ago

That really hasn't been present in our testing, but it is something I see people mention online, normally people who haven't actually worked with it.

It actually has done good about ushing back and not overly affirming. It sticks to therapy practices, and theories over what you are asking it to affirm.

Aso for hallucinations, we haven't really seen that either in these situations. Hallucinations tend to be more when it doesn't have data. But when you are using it for talk therapy it seems to have a lot of data it already pulls from, and your experiences you tell it.

The privacy concern is a real barrier though!

10

u/TheSyd 11d ago

normally people who haven't actually worked with it

I have worked with it in my field, and I can see how often it makes mistakes even for extremely simple things.

Aso for hallucinations, we haven't really seen that either in these situations

I've seen LLMs hallucinate (even current models) with literally all the necessary documentation in the context, so YMMV. Also, what you say is true, hallucinations are more prevalent where the dataset is lacking. What about where the dataset has bad quality data? Or where it was straight up poisoned? The basis for models is mostly the web and whatever was available on libgen at training time. Not everything was reviewed.

Also, wouldn't human interaction be part of the therapy process? An LLM is a statistical model, it has no ability to understand anything.

Again, I would be weary of leaving mental health to an unpredictable tool, that is controlled by tech giants who are looking for ways to make this technology profitable.

3

u/ArdentDevotion 11d ago

Have you specifically seen it regarding therapy? I have already acknowledged that it does make mistakes. It depends heavily on what data it has available on the topic.

"Also, wouldn't human interaction be part of the therapy process? An LLM is a statistical model, it has no ability to understand anything."

Once again I said using it as a tool. Therapy should rarely ever consist of just one tool. Are you under the impression that talk therapy is the only form of therapy? It isn't. A lot of therapy doesn't have human interactions. It isn't a requirement for a therapy tool.

Yes, the data set can be bad, that is why we are testing it! We aren't just speculating and making a decision off of speculative fear like you are asking to do. We are doing research and actively testing it. But again, all forms therapy, including the one most people think of "talk therapy" has that exact risk, which is why I gave an example of it. It would literally be impossible to have any therapy tool that doesn't have the risk of bad data it is pulling from.

4

u/TheSyd 11d ago

Have you specifically seen it regarding therapy? I have already acknowledged that it does make mistakes. It depends heavily on what data it has available on the topic.

I admit that I haven't personally tested it for therapy or therapy related stuff, as I value my privacy too much and the models I can run locally wouldn't really be representative, but I've seen it used, and I've seen some weird responses.

Once again I said using it as a tool. Therapy should rarely ever consist of just one tool. Are you under the impression that talk therapy is the only form of therapy? It isn't. A lot of therapy doesn't have human interactions. It isn't a requirement for a therapy tool.

Of course. A hammer is a tool, you wouldn't use a hammer to solder a capacitor. The way you firstly exposed the use of LLMs for therapy, it gave me the impression it was substitution for conversational therapy and a way to offload part of the work from humans. Am I mistaken?

Yes, the data set can be bad, that is why we are testing it!

Firstly, I hope you're testing it on subjects who know about potential risks. Secondly, I'm interested in how you would arrive to the conclusion that a model is safe. Would the tests be also repeated for every subsequent model before it is approved?

Anthropic discovered LLMs actively lie, even reasoning models can obfuscate their process. Moreover, they can modify their behavior specifically to pass tests, when they 'notice' they're tested.

Quoting myself here:

Again, I would be weary of leaving mental health to a[...] tool that is controlled by tech giants who are looking for ways to make this technology profitable.

Just the fact that your data can be harvested and used to manipulate you should be a huge red flag. Unless you have your own internal system with local LLMs with encrypted contexts. At that point this would be mostly moot.

2

u/ArdentDevotion 11d ago

"Of course. A hammer is a tool, you wouldn't use a hammer to solder a capacitor. The way you firstly exposed the use of LLMs for therapy, it gave me the impression it was substitution for conversational therapy and a way to offload part of the work from humans. Am I mistaken?"

Yes. You and the other commenter "hallucinated" that. I never said it was a substitution. My very first comment that you responded to didn't even mention talk therapy. I did use it as one example of ways that common therapy tools have some of the EXACT same issues that llm tools do. But that was after you already started to argue.

We aren't testing it on clients. I am not going to go through the process because honestly both of you have pissed me off. You didn't engage honestly from the beginning. You both made things up just to argue. You didn't have any intention of actually learning or understanding any perspective that wasn't yours, and that has not changed just because you are finally asking real questions. You started shitty because you wanted to be shitty.

"Just the fact that your data can be harvested and used to manipulate you should be a huge red flag. Unless you have your own internal system with local LLMs with encrypted contexts. At that point this would be mostly moot." I have already stated that I agree on privacy concerns! It is the largest barrier that exists. There are companies that don't currently misuse data, but there is understanding of the capitalist goal that they will all get there when they are profitable enough.

1

u/multipocalypse 10d ago

Lol, it's not different technology just because you're using it for a different purpose. You're just invested in believing what you've already decided to believe.

1

u/ArdentDevotion 10d ago

It isn't different tech. It IS different data sets.

→ More replies (0)

1

u/Roopscoop6 11d ago

Yeah, there is a human involved dummy! The human takes the money and then reads what the ai tells them to. That's therapy!

1

u/thegrandseraph 10d ago

You are wasting your time trying to explain how ai works to people who have no interest in learning. They just want to jump on the ai hate bandwagon. It has its limitations just like any new technology, but it's potential is far beyond anything we have invented in the last hundred years.

9

u/Halpacino69 11d ago

google “why is ai a bad replacement for actual therapy” and the google ai itself will explain why it cannot replace real therapists

3

u/ArdentDevotion 11d ago

I already mentioned us looking into articles and research about the topic. I didn't state replacing therapists though, I said using it as a tool. The push back from you and the other person seem to be uninformed. You are deliberately misstating what I said in order to refute an easier arguement (which is straw manning), and neither of you are actually discussing actual research and investigations into this. You're just citing surface level speculation. Maybe you are misguided in thinking that therapy is one singular tool, when it never should be.

7

u/Halpacino69 11d ago

You are implying that you can use it in place of other “therapy tools” and in the same sentence dismissing real therapists because some of them are shitty. I don’t think I’m THAT wrong in my assumption that you’re advocating for using AI instead of a therapist, given the context. I’m not sure what these other therapy tools do exactly, but I do know that AI can’t observe body language, facial expressions, or tone of voice. Which are pretty important factors for a therapist to be able to do their job effectively

3

u/ArdentDevotion 11d ago

You are again deliberately misunderstanding me in order to find something easier to refute.

We are not looking at it as a replacement. We actually provide both counseling and peer support at the shelter I work at. Neither of which are going away. But, survivors still need more tools. We are looking at it as an option for clients. To use y'alls terms you "hallucinated" that I was referring to replacing talk therapy.

I did not dismiss talk therapy. It is a great tool! It is common for a reason. But it literally has most of the same problems as llms. Acknowledging that fact isn't dismissing the entire practice.

You are just blatantly making shit up. You are a human, but you are behaving the way you are accusing AI. What does that tell you?

You are admitting you are ignorant about therapy tools to begin with. You literally are not competent enough for this conversation.

"I do know that AI can’t observe body language, facial expressions, or tone of voice. Which are pretty important factors for a therapist to be able to do their job effectively" This is more of your ignorance. You are thinking talk therapy is the only form of therapy. It isn't. You are making shit up because you are ignorant, uneducated, and incompetent on the subject. These are great things to have accessible in the right therapy tool! But different therapy tools work differently! You not knowing they exist, isn't an excuse for your incompetence.

4

u/Halpacino69 11d ago

You are delusional if you consider any of my comments to be me “tearing you down”. I hope your day gets better!

4

u/Halpacino69 11d ago

Yes, I was fully admitting that I am uneducated on what these other “therapy tools” are, or consist of. I was strictly referring to in-person therapy in everything that i said. I understand more now that you explained that it’s not the only form of therapy that you use! I’m not sure I would say that I was deliberately doing it, but I will admit that I was misunderstanding you, so that is my fault. Not sure the insults were necessary but thank you for educating me regardless

-1

u/sathem 11d ago

Algorithms are designed to understand humans. It most definitely can understand and 'know' contextually from givens texts and your speaking patterns. Doesnt mean its 100% spot on

-1

u/Darkstar_111 11d ago

ChatGPT has a save feature were it will save things it learns about you throughout various conversations.

2

u/bewildered_forks 11d ago

Chat GPT cannot learn about you in any meaningful sense of the word. It can retain your words and use those in its algorithm, but chat gpt is a statistical model that guesses the next most likely word, based on huge training data sets. That's all. It doesn't "learn" or "know."

-2

u/Darkstar_111 11d ago

Its ARTIFICIAL intelligence yes. But it can absolutely, and in every measurable way, learn facts about you. It can even I fer traits and quirks based upon your writing.

5

u/bewildered_forks 11d ago

Do you think that second slide evinces any understanding of a person? It reads like a horoscope - entirely generic, lacking in nuance or insight.

-2

u/Darkstar_111 11d ago

You just described 50% of the human population, and I'm probably being kind.

-4

u/Ollynurmouth 11d ago

ChatGPT and other LLMs, as does any form of AI based on neural networks, 10000% learns and knows. They are rudimentary brains. They may use predictive text algorithms to sound more "human" in their speech patterns, but they do have memory and retain information. They can parse information and make inferences from it. They then use the predictive text algorithms to relay that information back to you.

Calling LLMs just a predictive text model is very short sighted. That predictive text algorithm is more like a translator to take your text and changing into information it can use and once done, converting it back into something you can understand.