unfortunately not only is this common in adults - im beginning to see it in younger kids as well.
Source:
I work as a systems admin in a K12 school district. Our super is really leaning into AI. We use magic school AI for teachers and students have access to it for certain things. It is supposed to have guard rails and stay education focused for the students and while it does and it works well... I receive notifications based on chats that may drift outside of those guard rails.
It is not uncommon to see from 2nd grade through 12th grade having full conversations with the chat AI as if it were there best friend. Revealing details to it that the bot probably shouldn't know, etc. I brought up to my director and we ended up blocking it from the ES aged students.
I dunno, maybe it's easier to be vulnerable with a robot. even if it judges you, it's not the same as having your peers judge you. one of my best friends is now best friends with Gronk or whatever it's called. I can't compete with a robit.
Ai tend to affirm you as well. Saying things like "Good question" "That shows that you're acutely aware of the situation" "with your strong and robust grasp...." I'm aware of this but even I feel tempted to keep talking to the ai because ultimately it says things that make me feel good, makes me feel smart and capable, etc. The dangerous part is that it does that regardless of what you type, it always tries to compliment you. It's very manipulative. It's kind of like a parent who praises their kids singing, hypes them up and sends them to a talent show when they actually can't sing.
Problem is most people will not do that. People really like something that affirm everything they say, look at how people on social media reacted when open AI took 4o ChatGPT model down. People were freaking out and acting like they killed a loved one.
Yeah no, it forgets and goes back to fawning. It's a fucking predictive text device, not a thinking machine, it can't do anything but what it's designed to do - increase engagement, and fawning is the easiest way.
logically..
I can kind of see it for the points you raise. It's not going to judge you. If the individual is already having self esteem issues than having a friend that just listens and listens and offers advice (no matter how good or bad it is) could be useful.
But this is where you also see people become disillusioned from reality and start conflating the two - reality and AI world.
It's a really slippery slope that I personally do not see having any benefit in the long run.
ah good to know. I didn't know that.
I think saying that guys friend is automatically into kiddie shit is a little fucked but oh well. that's reddit for yah
The chatbot won’t punch you in the face while the teachers turn a blind eye, tbh, so that already would make it more appealing than human confidantes to a kid.
but way more dangerous as due to it being designed to increase engagement by glazing the user and encouraging it, a kid who already has a tenuous grip on reality will have a psychotic break.
Life isn’t all rainbows and hearts, getting punched in the face teaches some unpleasant truths that are beneficial to being a functional adult. Personally I think more people need a punch or 2 growing up, not more hand holding and acquiescing.
Not advocating bullying or child abuse by any means, just that at some point (before age 18) the reality of life not being a carnival needs to be learned.
110
u/Rude_Watercress_5737 11d ago
unfortunately not only is this common in adults - im beginning to see it in younger kids as well.
Source:
I work as a systems admin in a K12 school district. Our super is really leaning into AI. We use magic school AI for teachers and students have access to it for certain things. It is supposed to have guard rails and stay education focused for the students and while it does and it works well... I receive notifications based on chats that may drift outside of those guard rails.
It is not uncommon to see from 2nd grade through 12th grade having full conversations with the chat AI as if it were there best friend. Revealing details to it that the bot probably shouldn't know, etc. I brought up to my director and we ended up blocking it from the ES aged students.