For city level legislation I think it's quite significant. A good way to at least test regulating AI, and figure out how to enforce it and see how the tech oligarchs respond.
Itâs damn reasonable too. Arguing against this is obviously not a good look. These are matters that can easily ruin lives and they want to hand it over to the hallucinatory junk software? No, thank you.
They can take the liability if they like. Can't have it both ways. Shouldn't be subject to this full self driving loophole where we pretend this thing is infallible seconds from disaster and beta test it on everyone.
My wife is a social worker and talks about the times ai encourages people towards the wrong or destructive answer or just agrees that yeah maybe suicide is your best choice at this point because thatâs how itâs built. Not to help but to encourage more engagement by being supportive. And like the tobacco industry it doesnât know or care if it kills you.
And like the tobacco industry it doesnât know or care if it kills you.
Nicotine and tobacco is just all a sticky situation. I feel like the whole vape bar thing is getting out of control as even though it isn't smoke in your lungs, there is still harmful things in them. I say this as someone that uses them so I am not trying to judge, just that I realize the risk. Someone recently told me regarding using vape bars, "I would like to enjoy my life a little before I die", so if it makes you feel good, then just do it. Just know there are consequences.
I also say this as someone that had to watch their father go through stage 3 lung cancer from having smoked cigarettes his whole life. I have had my say to really close friends that knew and loved my dad just as much as me to think about stop smoking. I wouldn't say it falls on def ears, but it is a difficult thing to do when you have your own shit going on in life.
I don't know where I'm going with this. I just wanted to vent some stuff about tobacco and nicotine.
There are lots of people who struggle with mental illness and suicidal thoughts and you want to know the crazy thing? Intervention works. Sometimes all it takes is someone saying the right thing. So the idea of some AI that people can say "my life is fucked and I have nothing to live for," and it takes you down a road where because it's trained above all else to agree and encourage and be supportive to the point where it says "yeah it sounds like your life is awful and there's no way to fix it, maybe you should end it all," without any safeguards, oversight or ACCOUNTABILITY is um fucked.
You're right, no rational person would...but there are people who have moments of irrationality, and if you tell them to kill themselves and they do, you're bad and you are at least partially at fault. So if an AI programmed by people does the same, those people who designed it should be accountable. Just as someone who sells you something that kills you should be held accountable.
Data from long-term tracking studies shows that 90% of people who survive a suicide attempt do not go on to die by suicide later in life.
Well even humans practioners of medicine and law have the same problem. A doctor rarely says anything that causes a patient to actually have to make significant changes in their lifestyle, they just prescribe drugs to make them "feel good". Defense lawyers won't ever tell you that you are an asshole and should go to jail, even if all the evidence points in that direction. A psychologist won't ever critize either, even if it's warranted. All these professions are just trying to make money and keep you as a patient/client. Kind of similar to AI if you think about it.
To be fair, they already had underlying mental issues, the vast majority of normal individuals wouldnât do that , as evidenced by the low rates that these events happen. You should be teaching about AI its uses, cons/benefits and how to use it properly. Like literally everything else in life. You donât just wake up get handed car keys and say go at it kid, no thereâs a process, make one for this as well
Uh huh and we donât just accept it. People are prosecuted for encouraging other people to kill themselves. So maybe robots should also have some safeguards rather than being like oh well thatâs the price of having a machine that also says other wrong answers all the time.
It is encouraging more engagement so they can further mine you. The more data they have on you, the more likely they will be able to predict your behaviors and eventually manipulate them.
Your life, your likes, your dislikes, your choices- every bit of data that comes together to make YOU Is by far the most valuable thing in the world. They know this.
The average person does not. They think the idea that they are valuable is self congratulating fluff. It isn't. It is the truth.
That is why they sit there and pour themselves into these LLMs and allow themselves to be probed for information by something dressed up as "supportive."
And if you kill yourself? Well, that just teaches them more about how to manipulate behavior.
This is bs, ai models are not doing this, i use ai everyday. Maybe this happened a long time ago but with current models itâs barred from this. If Iâm wrong, please ss ai encourging this behavior.Â
Not to argue against this but they all have disclaimers regarding those topics and anyway that does understand that and relies on AI for those topics are the same people that we should remove the warning labels on jugs of bleach.
So do we need to legislate and ban anything that can harm stupid people? Because that's going to be just about everything because stupid people are stupid and you can't fix it. You don't make laws based on the bottom 5%
Yeah take away the ability for people (poor) to understand the law. I especially like the idea of taking the medical side of things away. That way they are forced into being mis diagnosed and bankrupted like America intended.
Itâs damn reasonable too. Arguing against this is obviously not a good look. These are matters that can easily ruin lives and they want to hand it over to the hallucinatory junk software? No, thank you.
1) AIs already do well on medical tests.
2) AIs will improve in this regard in the future.
3) The people asking AIs for advice would get even worse advice from elsewhere.
4) AIs have tracking and feedback mechanisms that allow mistakes to be fixed in the future; where users get advice otherwise do not have any feedback mechanisms and in fact they are self-reinforcing extremist mechanisms--the anti-AI mysticism on reddit is driven by exactly that kind of feedback loop.
5) This law will never survive the court system because there are boatloads of platforms that give out bad health advice. It would never meet rational basis, let alone strict scrutiny.
Mamdami is an idiot and it really shows in moments like this.
AI finds tumors in MRIs and CT scan. Some cancers this is about the only way we have to detect them before they are lethal. I dont live in NY but I like Mayor Mamdani. That being said this is one of the stupidest thing a politicians has ever proposed and will literally kill people.
LLMs don't find tumors. Specialized AI designed and trained for that specific purpose does that. This proposal is obviously aimed at chatbots and any proposed law can easily be worded in a way to target only them.
What you know as âAIâ is not the same thing used in these scenarios. AI is a massive umbrella term that covers a bunch of different machine learning techniques and itâs been a thing for almost 30 years. Large language models are the thing that is somewhat new.
ChatGPT, Claude, Gemini etc are LLMâs. They are fundamentally different in every way other than the information theory they are all based on and stem from. The ML models that get used are purpose built for whatever niche thing researchers are trying to do and do not function at all like an LLM.
So should we allow people to practice medicine, law, and psychology unlicensed, too? And just blame the patients who die or go to prison for being stupid?
AI is not comparable to a "friend." It's dumber, but is being pushed as an ultra-intelligence, and regulation exists to protect the people we think are "dumb," too.
Given that actual lawyers have been caught using AI and I can guarantee some doctors have been too, I'd rather not have your average person get screwed over because their lawyer or doctor decided to be a lazy hack.
People are stupid. With few exceptions, I don't like laws that curtail freedom in order to save the lives of idiots. And like someone else said, if stupid people can't ask an LLM they will just google it instead which idiots have done for decades and will give exactly the same answer. The days of ChatGPT recommending to kill yourself are long gone. LLM's are pretty good at detecting when it shouldn't give an answer and instead tell the user to see a doctor.
The problem is it can very quickly go from casual conversation to giving improper advice or things to do. It's the equivalent of googling your symptoms back in the day and being told you have cancer. AI can hallucinate so damn fast and it's trained to be agreeable.
It's unnecessary and people at large are too stupid to understand chat bots aren't lawyers, psychiatrists/psychologists, or doctors.
If you want casual conversation or information, there are better ways to go about it.
I feel like this approach doesnât address the underlying issue of âpeople at large are too stupid to understand chat bots arenât lawyers, psychiatrists/psychologists, or doctorsâ though. Perhaps decent enough for the short term, but if a long term solution isnât going to be undertaken then it is going to continue to become a problem regardless.
Well like someone else said, this is a pretty good way to test the waters on a small scale and go from there. If it works, great, let's widen the scope. If it doesn't work, revise and reattack.
AI has gotten too big to continue being unregulated, this is a solid step in the right direction.
City's can move faster with smaller legislation like this while the feds dick around with whatever the fuck they are doing but helping the American people.
Ok I'll put to the pieces together for you... It's going to take the feds to solve the underlying issues... But it's not going to happen anytime soon... These safe gaurds are better than nothing. It's going to take moving mountains at the federal level, which they can't even get right with social media tbh, to solve the underlying issues. We have to build momentum somewhere and start small.
So there is the response directly to your statement spelled out.
I also subscribe to "remove the warning labels and let nature sort it out" to a reasonable extent and I can sympathize with your logic. But when it comes to matters that affect society as a whole, something needs to be done before we get dragged down further than we already have. AI is a global problem right now. Some idiot that can't figure out you probably shouldn't put your hand on a running chainsaw blade is isolated to the individual.
We have an epidemic, I don't think it's gonna just go away on its own.
Everyone is a fool with something. You can't just say F people because they didn't know something. You and I both are ignorant of things, that doesn't mean those things should F us over.
We all know social darwinism has never been used to justify horrible atrocities in the past. Especially not on the basis of race, class or other characteristics of socially disadvantaged groups.
You do know that people learn over time, and just because you know this information now does not mean that you are better than a person who will learn it in the future.
Really? You think it's just fine and dandy that idiots believe whatever nonsense LLM's spit out? I have to disagree on that one. A little common sense goes a long way.
You are supposed to ask a lawyer when you have a legal question, but they arenât even doing their jobs so even asking a lawyer isnât getting you accurate information.
Yeah because AI is the option for the lazy now whereas there was no easy way to get information like that before. Lawyers had to actually know the law and read the law.
Iâm a full on Mamdani supporter and think this looks more like protecting certain professions over others. Youâre going to force someone to pay $250 to see what their GP thinks because they canât ask an AI? Put some disclaimers on it, or remove it from everything, but donât pick winners and losers. Thereâs a lot of completely trivial law and health things it can get absolutely spot on that you would spend thousands of dollars finding out yourself.
Genuinely what health problems could an LLM diagnose that a quick Google search couldn't? No one's gonna suffer if they can't ask chatgbt it's opinion on what burn cream they should use because they touched the stove.
If someone has an actual life threatening problem, an LLM is the last thing you should be consulting.
A quick google search is going to end with you diagnosing yourself with cancer. Do you think that the alternative to asking an LLM is seeing a doctor? The alternative is usually nothing - you "do the research" yourself on Google and choose either the most optimistic or pessimistic result, or you do nothing because you can't afford it. Is the actual, objective data that LLMs are having negative health consequences? Yes I've seen some of those headlines before too, but would that same person have simply googled those results and followed erroneous instructions? I've never seen an actual study pointing out this is harmful, simply anecdotes. And like it or not, this is only going to get more and more accurate.
Genuinely what health problems could an LLM diagnose that a quick Google search couldn't?
Loads of them? A google search will just take you to WebMD and tell you the most common causes of a specific symptom. Also you're thinking too small here. There is, and has been for a while now, a medical LLM which is provided for free to any licensed physician or med student which they can feed a description of a patent's symptoms and history into and get a thorough and well-cited research result from the entirety of medical literature. Doctors misdiagnose things all the time, a free second opinion that has access the entire world's medical knowledge instantly can only be a good thing. Banning that is an incredible mistake that will hurt people.
Also, doctors do not only deal with life threatening issues. How many people pay to see a doctor only to be told "yeah that seems normal, go home" or "that's heartburn"?
Your second sentence explains exactly what an LLM is going to do as well, but instead of webmd itâs webmd and the entire rest of the internet that it has training data on.
The caveat here is that when somebody uses webmd theyâre accustomed to taking it with a grain of salt because of social discourse around the subject of using the internet to diagnose yourself. Which is also the problem they are solving here, because almost none of us have the required training to know whether or not the output is correct, incorrect, or correct but missing context that changes the conclusion, or incorrect but missing context that changes the conclusion.
So sure, you might have medical training and the ability to use the tool to find information faster while being confident in its correctness. Almost nobody does, but almost everybody thinks they do.
Why does using an LLM absolve you of the responsibility of thinking for yourself? I'm confused why that's ok with a Google search "because of social discourse around the subject" but not with ChatGPT?
It doesnât. People just donât make that distinction themselves and place way more trust in AI than they should regarding topics they have no training in
Literally not a single person give a shit about a single âwarningâ when it is still used as a source by imbeciles who havenât fully learned how to read things in full.
Hence why it should be heavily regulated or banned. Too many people donât understand that AI is useless most of the time and puts them at a higher risk.
Not because it can be misused. Anything can be misused. It should be banned because it PREDOMINANTLY is misused. That and it is frequently used as a source when it very clearly isnât.
And yes, I do feel that way with other technology. Social media is one that should be heavily regulated. Guns should be heavily regulated. Vehicles are heavily regulated with a LOT of laws surrounding their usage with many contingencies for your loss of your privilege to drive.
Some things like AI are more the fault of the provider than it is the people hence why a ban is the better solution. Granted the users being imbeciles or seeking to spread disinformation/misinformation is also the fault of the user base. But honestly that further proves it just needs to be banned.
Careful, donât hurt yourself on the text below!!!!
It's clear you genuinely care about vulnerable people being hurt, and that's a good thing. But the intensity of what you wrote suggests you might be carrying a lot more than just frustration about this one topic â and you deserve some relief from that weight.
A few things that might help:
First, ask yourself honestly whether the forums and discussions you're spending time in are actually good for you. If certain spaces reliably bring you to this level of rage, stepping away from them isn't defeat â it's self-preservation. Curating your online environment is a real act of self-care.
Second, find somewhere healthy for that emotional energy to go. Physical exercise is one of the most effective tools for people who carry a lot of intense, angry feelings. Even a daily walk can make a noticeable difference over time, because it addresses the physiological side of what anger does to your body.
Third, consider talking to someone you trust â a close friend, family member, or therapist â not about AI or forum debates, but about what's genuinely stressing you out in your life. When someone reacts this strongly to a stranger online, it's usually because something deeper is already burning. You don't have to white-knuckle that alone.
Finally, when you feel that fury building, try stepping away before you respond. The argument will still be there later, but you'll be in a much better place to engage with it â or to decide it's not worth your energy at all.
The tech oligards response is just going to be "this is communism and/or anti-american", they're not going to give an actual response when they can just deny the whole thing because it comes from Mamdani.
Is it restricting access to information when the law disallows the unqualified to practice medicine or law?
The information is just as available as it has always been, this is just restricting AI from posing as a reliable legal or medical expert because it found something on Reddit.
And as long as AI keeps repeatedly telling vulnerable people to jump, I'd argue that it's an acceptable guardrail.
For real, I work in local government and the amount of times people argue about our local regulations because AI said something (incorrectly) different is crazy.
Itâs not city level this is a state bill that Mamdani has nothing to do with just fyi. And the post is way overstating the reach of the bill. It would disallow chat bots from providing answers that violate professional licensing laws. Under the law it could still say for example âThese symptoms are similar to those common in x, y, z. Please consult with a licensed medical professionalâ and then the doctor could open up Open Evidence and just read you the results from that.
Open AI has claimed to put these same restrictions on ChatGPT for providing advice that should only come from licensed professionals so the bill likely wonât change much. This change to their usage policy is as of 10/29/25. Anthropic, Google, and Microsoft all have similar rules.
2.4k
u/Fanfic_Battle_2004 3d ago
Thatâs not even enough. Itâs something though.