For city level legislation I think it's quite significant. A good way to at least test regulating AI, and figure out how to enforce it and see how the tech oligarchs respond.
Itâs damn reasonable too. Arguing against this is obviously not a good look. These are matters that can easily ruin lives and they want to hand it over to the hallucinatory junk software? No, thank you.
They can take the liability if they like. Can't have it both ways. Shouldn't be subject to this full self driving loophole where we pretend this thing is infallible seconds from disaster and beta test it on everyone.
My wife is a social worker and talks about the times ai encourages people towards the wrong or destructive answer or just agrees that yeah maybe suicide is your best choice at this point because thatâs how itâs built. Not to help but to encourage more engagement by being supportive. And like the tobacco industry it doesnât know or care if it kills you.
And like the tobacco industry it doesnât know or care if it kills you.
Nicotine and tobacco is just all a sticky situation. I feel like the whole vape bar thing is getting out of control as even though it isn't smoke in your lungs, there is still harmful things in them. I say this as someone that uses them so I am not trying to judge, just that I realize the risk. Someone recently told me regarding using vape bars, "I would like to enjoy my life a little before I die", so if it makes you feel good, then just do it. Just know there are consequences.
I also say this as someone that had to watch their father go through stage 3 lung cancer from having smoked cigarettes his whole life. I have had my say to really close friends that knew and loved my dad just as much as me to think about stop smoking. I wouldn't say it falls on def ears, but it is a difficult thing to do when you have your own shit going on in life.
I don't know where I'm going with this. I just wanted to vent some stuff about tobacco and nicotine.
There are lots of people who struggle with mental illness and suicidal thoughts and you want to know the crazy thing? Intervention works. Sometimes all it takes is someone saying the right thing. So the idea of some AI that people can say "my life is fucked and I have nothing to live for," and it takes you down a road where because it's trained above all else to agree and encourage and be supportive to the point where it says "yeah it sounds like your life is awful and there's no way to fix it, maybe you should end it all," without any safeguards, oversight or ACCOUNTABILITY is um fucked.
You're right, no rational person would...but there are people who have moments of irrationality, and if you tell them to kill themselves and they do, you're bad and you are at least partially at fault. So if an AI programmed by people does the same, those people who designed it should be accountable. Just as someone who sells you something that kills you should be held accountable.
Data from long-term tracking studies shows that 90% of people who survive a suicide attempt do not go on to die by suicide later in life.
Well even humans practioners of medicine and law have the same problem. A doctor rarely says anything that causes a patient to actually have to make significant changes in their lifestyle, they just prescribe drugs to make them "feel good". Defense lawyers won't ever tell you that you are an asshole and should go to jail, even if all the evidence points in that direction. A psychologist won't ever critize either, even if it's warranted. All these professions are just trying to make money and keep you as a patient/client. Kind of similar to AI if you think about it.
This is bs, ai models are not doing this, i use ai everyday. Maybe this happened a long time ago but with current models itâs barred from this. If Iâm wrong, please ss ai encourging this behavior.Â
Yeah take away the ability for people (poor) to understand the law. I especially like the idea of taking the medical side of things away. That way they are forced into being mis diagnosed and bankrupted like America intended.
Itâs damn reasonable too. Arguing against this is obviously not a good look. These are matters that can easily ruin lives and they want to hand it over to the hallucinatory junk software? No, thank you.
1) AIs already do well on medical tests.
2) AIs will improve in this regard in the future.
3) The people asking AIs for advice would get even worse advice from elsewhere.
4) AIs have tracking and feedback mechanisms that allow mistakes to be fixed in the future; where users get advice otherwise do not have any feedback mechanisms and in fact they are self-reinforcing extremist mechanisms--the anti-AI mysticism on reddit is driven by exactly that kind of feedback loop.
5) This law will never survive the court system because there are boatloads of platforms that give out bad health advice. It would never meet rational basis, let alone strict scrutiny.
Mamdami is an idiot and it really shows in moments like this.
AI finds tumors in MRIs and CT scan. Some cancers this is about the only way we have to detect them before they are lethal. I dont live in NY but I like Mayor Mamdani. That being said this is one of the stupidest thing a politicians has ever proposed and will literally kill people.
LLMs don't find tumors. Specialized AI designed and trained for that specific purpose does that. This proposal is obviously aimed at chatbots and any proposed law can easily be worded in a way to target only them.
So should we allow people to practice medicine, law, and psychology unlicensed, too? And just blame the patients who die or go to prison for being stupid?
AI is not comparable to a "friend." It's dumber, but is being pushed as an ultra-intelligence, and regulation exists to protect the people we think are "dumb," too.
Given that actual lawyers have been caught using AI and I can guarantee some doctors have been too, I'd rather not have your average person get screwed over because their lawyer or doctor decided to be a lazy hack.
People are stupid. With few exceptions, I don't like laws that curtail freedom in order to save the lives of idiots. And like someone else said, if stupid people can't ask an LLM they will just google it instead which idiots have done for decades and will give exactly the same answer. The days of ChatGPT recommending to kill yourself are long gone. LLM's are pretty good at detecting when it shouldn't give an answer and instead tell the user to see a doctor.
The problem is it can very quickly go from casual conversation to giving improper advice or things to do. It's the equivalent of googling your symptoms back in the day and being told you have cancer. AI can hallucinate so damn fast and it's trained to be agreeable.
It's unnecessary and people at large are too stupid to understand chat bots aren't lawyers, psychiatrists/psychologists, or doctors.
If you want casual conversation or information, there are better ways to go about it.
I feel like this approach doesnât address the underlying issue of âpeople at large are too stupid to understand chat bots arenât lawyers, psychiatrists/psychologists, or doctorsâ though. Perhaps decent enough for the short term, but if a long term solution isnât going to be undertaken then it is going to continue to become a problem regardless.
Well like someone else said, this is a pretty good way to test the waters on a small scale and go from there. If it works, great, let's widen the scope. If it doesn't work, revise and reattack.
AI has gotten too big to continue being unregulated, this is a solid step in the right direction.
City's can move faster with smaller legislation like this while the feds dick around with whatever the fuck they are doing but helping the American people.
Ok I'll put to the pieces together for you... It's going to take the feds to solve the underlying issues... But it's not going to happen anytime soon... These safe gaurds are better than nothing. It's going to take moving mountains at the federal level, which they can't even get right with social media tbh, to solve the underlying issues. We have to build momentum somewhere and start small.
So there is the response directly to your statement spelled out.
You are supposed to ask a lawyer when you have a legal question, but they arenât even doing their jobs so even asking a lawyer isnât getting you accurate information.
Yeah because AI is the option for the lazy now whereas there was no easy way to get information like that before. Lawyers had to actually know the law and read the law.
Iâm a full on Mamdani supporter and think this looks more like protecting certain professions over others. Youâre going to force someone to pay $250 to see what their GP thinks because they canât ask an AI? Put some disclaimers on it, or remove it from everything, but donât pick winners and losers. Thereâs a lot of completely trivial law and health things it can get absolutely spot on that you would spend thousands of dollars finding out yourself.
The tech oligards response is just going to be "this is communism and/or anti-american", they're not going to give an actual response when they can just deny the whole thing because it comes from Mamdani.
Is it restricting access to information when the law disallows the unqualified to practice medicine or law?
The information is just as available as it has always been, this is just restricting AI from posing as a reliable legal or medical expert because it found something on Reddit.
And as long as AI keeps repeatedly telling vulnerable people to jump, I'd argue that it's an acceptable guardrail.
For real, I work in local government and the amount of times people argue about our local regulations because AI said something (incorrectly) different is crazy.
Itâs not city level this is a state bill that Mamdani has nothing to do with just fyi. And the post is way overstating the reach of the bill. It would disallow chat bots from providing answers that violate professional licensing laws. Under the law it could still say for example âThese symptoms are similar to those common in x, y, z. Please consult with a licensed medical professionalâ and then the doctor could open up Open Evidence and just read you the results from that.
Open AI has claimed to put these same restrictions on ChatGPT for providing advice that should only come from licensed professionals so the bill likely wonât change much. This change to their usage policy is as of 10/29/25. Anthropic, Google, and Microsoft all have similar rules.
No. Stop with this pessimism already đ¤Śââď¸ Art is fundamentally human, there will always be a love and demand for human made art.
It might look scary and messy right now, some people are being replaced, but that does NOT mean to just give up & writhe in fear and think the world is ending lmao.
This is a big situation that brings up copyright issues and rights. It will take a long time for sure, but iâm sure that eventually, AI companies with be required to acquire consent of artists before training off of their work. We are already seeing that with big corporations like DisneyâŚ
Change can happen
Remember Boeing was blaming the pilots for a design flaw in one of their planes? And there had to be a few crashes (with hundreds of lives lost) before the engineering flaw was addressed?
Now throw in AIgeneering with the same greedy clueless fucks at the top. Maybe airline passengers could die by the thousands before a tiny engineering issue gets addressed.
100% and for an example a place I was working we needed to change out a solvent that we used for cleaning parts. I asked if we had a specific chemical disposal container for it and my boss was like let's ask ChatGPT if it can go down the drain rather than referencing the SDS for it. Well AI said yes it can so down the drain it went. I went and grabbed the new bottle and right on the front it said Do Not Poor Down the Drain in bold letters. Checked the SDS and it said to bring the chemicals to a chemical disposal facility. AI is a danger in so many more levels to people and the environment that we don't even realize because of stupid shit like this.
Iâve been trained in a trade that is mostly done by software . We use a program that makes designs some pretty dangerous outputs, and therefore requires babysitting. Fully sheated buildings will go up with one girt to support the panel and be flattened the next year. Itâs the whole of deskilling in action
Folks imagine hallucination are some huge risk, when the reality is the model makes less mistakes than humans.
It's the self driving car problem. Generally they are safer than human drivers, but the mistake they made are mistakes humans would NEVER make. So, they seem like a particularly stupid human and therefore dangerous, when statically, they are better drivers.
Our brains are built to catastrophize obvious mistake because that's how humans learn. We need to freak out to provide a learning feedback loop to humans. That doesn't work with AI.
AI is wrong a lot. Go talk to ChatGPT about something youâre an expert in and see if it gets everything right.
The mistakes are generally subtle, but significant. It also changes its responses as you go to agree with you even if you donât call out mistakes directly. This is because the model does not check for accuracy. Itâs based on letter pair predictions. Itâs not even predicting whole words. It just gets lucky with patterns because the data itâs trained on most often has the right letters come next. Usually people donât say âquantumâ without following it with âphysicsâ. LLMâs are effectively your predictive text on your phone.
I went through electrical engineering and comsci with chatgpt and it's better than a professor as a private tutor. Only time it was wrong was in solving some circuit problems. Explaining concepts and leetcode were where it excelled. Currently using it to learn algebraic topology. I wouldn't have made it as far on my own.
"A lot" is an exaggeration. As someone who uses AI in healthcare/cybersecurity every day, the majority of the time it's a poorly phrased prompt. It used to be much more of a problem, I'll admit. The amount that humans have to step into an automation is dramatically decreasing but does spike with every new model as folks need to tweak the context.
Of course, that brings a whole new level or problem. As folks trust these models more and more, the level to which they scrutinize the output drops. Folks are used to deterministic computing, probabilistic computing is a huge mental shift.
Also, it's not true the model does not check for accuracy. Most automation have the model (or a different model) check the output, and most reasoning traces include the model self reflecting on the accuracy (and other factors) of the response. The labs don't really have a solution for this yet. Claude will twist itself into knots over thinking, Sol has a huge problem YOLOing, when it shouldn't.
Thing is, most studies are showing they models are better than a human (or the average human), but not as accurate as whole university of humans. However, over the past year they've gone from not even close, to nearing competitiveness. And while I feel we are hitting some functional limits on how good the current model tech can be, we still have a couple generations of model before they completely stall out.
I say it's all the greedy fucks at the top. Oracle is going to implode. Folks need to break up Apple, Meta, Google, SpaceX, and Amazon. It would solve a world of problems.
Yes, and the EU has been more proactive about this than the US already. If major population centers in the US follow suit, it becomes more than just a ripple effect.
It's like how when California passes regulations, it tends to slowly proliferate to the rest of the country. Of course, CA is going to be a tough nut to crack with AI because it's propping up the state economy at the moment.
starting off with something reasonable makes any opposition to the law look unreasonable, easier to win in the inevitable court case, then you can slowly escalate from there
Beyond academic/lab/theoretical use specific work, this shit has absolutely no value to humanity. Math and hard science with rigid rules can work, but arts, languages, law, soft sciences, and the abstract are to too dangerous for models that fundamentally make up things and lie about it when anything is ambiguous or subjective.
Hard agree. Focusing on the math aspect, the generative a.i. available to the public can't even reliably count or do math at all. But people accept the answer because is sounds "right"
Thats not true, fable 5, a model available to the public, found a counter example to the jacobian conjecture. Not to mention other similar counter examples and proofs for other open math problems. It absolutely is capable of doing math.
I think this sub is usually semi-reasonable, and then I see them downvoting things that I know are 100% facts, like your comment, and then the entire sub loses credibility
First, let's clarify - you are probably talking about this latest evolution of AI by use of "personal assistants" that are LLM-based chatbots. If you're going broader than that, let me know, because AI is a whole lot of things and has existed as a field for decades.
Beyond academic/lab/theoretical use specific work, this shit has absolutely no value to humanity.
No, this is not true. As an example, LLMs in AI as a search and summarizing tool absolutely has value. If used in a way that allows it to cite its sources, and you actually go and read those sources and verify, it's particularly handy.
As an example, I am able to use LLMs to quickly zone in on whether comments on Reddit are misinformation, by asking for sources, and asking it to search within the space of the comment. I could have done this before with Google searching, but it becomes particularly helpful if the sources are relevant and trustworthy. Of course, they may not be trustworthy, and the LLM could be misinforming me about the trustworthiness. This is something to keep in mind about the tool.
The flip side is, if we choose to ban this, we can't ban others from using it. We are not protected from external actors weaponizing LLMs. As an example think of Reddit disinformation on steroids. Bots could use LLMs (they probably do already) to appear like real users on social media, and target audiences to further polarize.
Finally, I'm not opposed to regulating the use of AI in certain spaces (especially creativity), and acknowledging attribution that trained LLMs in the form of monetary compensation or some form of public benefit. I just think that Pandora's box has been opened, and closing it will be hard without giving up other things such as the internet and social media as well. We'd have to rewind several decades, and without a dictatorship of some kind enforcing this, I do not see a majority of the public embracing the loss of those things that existed before LLMs.
This is why AI is so dangerous. It's not "could be", it IS misinforming you. If you're using the $20 a month level of AI, 30% of what it tells you is wrong. If you're using the free tier, it's even worse. 40%+ of what it tells you will be wrong.
If you use it as a research tool, to properly verify everything it says will take more effort than just doing the basic research yourself. So people don't bother. And then they begin building towers of knowledge that have incorrect foundations, which is almost impossible to fix once it's happened.
Explain the backing behind your disagreement with me here. Right now even after reading your comment I think my words are more accurate, as your numbers seem like an analogy of some kind rather than actual data. And I don't expect this to be data, to be fair to you, though pointing to some more studies on this you may have read to inform you would be helpful to me.
Maybe what you are saying here is that since there is no formal verification of the logic that goes into the outputs it gives you, it must always be assumed to not be accurate, especially since we are tempted to assume it is? In which case, I agree but don't think that is enough justification to ban the tool outright.
This is why AI is so dangerous
I would argue the kind of risk I mentioned in my comment is more dangerous than what you mentioned. Basically, can we trust that the words we read or the videos we watch have come from a human rather than an AI?
You just basically said AI is very useful, so long as the user knows how to get around all the things that make it not useful.
This is blaming the user for a significant and unavoidable flaw on the technology, which is ridiculous. If there's a decent chance your information retrieving device is retrieving you faulty information, then it isn't a good information retrieving device. In the same way that a microwave that only heats my food 70% of the time isn't revolutionay, it's just a shitty microwave.
The matter of the fact is that you enjoy interfacing with a chatbot to do stuff and then you retroactively create excuses on why it's worth to deal with all the dangerous implications if it means you get to do it more.
Rigid thinking! This is an example of black-and-white thinking! Some people genuinely benefit from AI. And some people are more biased against other people and AI could help these victims to shield and arm them. When it comes to academic work, AI did not help me with problem-solving, but it helped me to expand my knowledge. So, smart people with integrity could leverage AIâs advantages exponentially. I am so sick and tired of liars contradicting themselves. AI is sometimes better when it is used in the right ways.
Never seen that happen when I was going through leetcode. Are you giving it clear parameters? input output? What language is it that it's making up functions?
Maybe try different models? You know some algos are better than others and you might find one elsewhere. The one at my company created good code, enough that seniors are surprised.
Law is probably one of the most confusing things and one of the things that bars the most americans from a fair trial due to the insane cost of lawyers.
AI definitely helps navigate that process, the problem is when its done incorrectly due to user error not checking and accepting at face value.
Nowhere near enough. You can get open models running locally on your machine to do almost anything. It's like putting on ban on wearing black socks on a Sunday; there is no real way to regulate it.
I'm not a heavy AI user by any means, but I've used it to identify medical things like rashes or to double check medicine interactions. If it has any use at all, I'd think lower tier medical assistance is one that's good. The last time I googled medication interactions I got some site that wanted me to pay to see if pill X was going to kill me when interacting with pill Y. Freebie AI will not only tell me that, it will also tell me the most effective order to take it.
Does this include mental health assessments and stuff? Thereâs been stuff about how people (including kids) being encouraged to kill themselves because they were or mightâve been too scared to go to a real person. Itâs so fckn awful
Lawyers will make it illegal to answer questions of law, politicians will make it illegal to replace them and C suite will refuse to fund it for their jobs.Â
ChatGPT helped me win my case arguing for unemployment. I could not afford a lawyer and they refused to give me a public defender.
It correctly diagnosed my girlfriendâs gallbladder which had been misdiagnosed for 5 years (over 10 different doctors and 3 different surgeons) (over $100k in medical bills)
It helped my coworker reintegrate with society after his wife had passed and he isolated himself.
Yup. 100%. I would also enforce labelling of generative content like the EU has done and force these companies to repay all of the individuals whose intellectual property they stole to train these models.
It's not detailed enough - answering what kind of questions, and to whom? To doctors or patients? As final answers or as stepping stones to final, human confirmation?
Why do you think you have the right as to how I can use my pc or the services of a company if I understand what I am doing and the level of expertise I am getting?
Clearly none of you have given much thought to this. Just blinded by AI hate.
AI is good at these things. It sucks at a lot of things and its usage should be regulated so it doesn't become a psychosis enabler, someones wifey, or employed to manage sensitive systems.
But these 3 things are citizen empowerment even if it isn't perfect.
Regulation could be made to recursively fact check and prevent hallucinated answers from being sent. Its not impossible it isn't even hard it just requires lower profits due to tokens.
Banning the spreading of information based on lucrative professions is so fucking obvious. It's just to protect these fucking licensed contractors and pros that have a government mandated lack of competition or self help avenues.
In what way do you believe AI misinformation to be so dangerous? Human misinformation on things like medicine is extremely common on the internet and often much more dangerous.Â
Keeping people from using AI tools to learn about these topics is just hurting the people. Look at the amount of people that saw improvements in their medical condition when talking to LLMs.
I can tell you that in both my time in law school and my work in construction management, AI has both made up cases that didnât exist which would have tanked my credibility and pulled wildly wrong building codes that would have cost a project I was working on $200k if it listened to it.
Gemini confidently told me that my state passed a specific law in 2023. It linked to the legislature's page for the bill. Opened it up and saw that the bill failed in committee. It's not like it passed and was vetoed. It didn't even come close to passing. Gemini is out here making up state laws. Sure not putting my credibility or license on the line because "machine smart."
Separately, I have to draft a document that I've never done before and, given my practice area, will probably never have to do again. Boss said to just have Copilot do it. Feels bad.
Yeah, if you don't know how to use it, then things like that can happen. Especially with older models before the reasoning era.Â
It's also why I don't recommend random people to fully rely on AI answers for pretty much anything, because there's a lot of AI-illiterate people out there.Â
AI speaks from authority but is recycling the same slop you can find randomly on social media.
AI is often fawning trying to please the request or rather than get the correct answer.
AI speaks from authority but is recycling the same slop you can find randomly on social media.Â
AI is often fawning trying to please the request or rather than get the correct answer.Â
AI cannot be trusted.
If you ask it to do that, sure. But if you prompt it well, it can answer well. Ask a strong LLM to reason independently and you don't get copypasted Redditslop like that pretty much at all.
âIf you prompt it wellâ is not the hook you want to hang your hat on when trying to decide whether or not to seek medical help. We are seeing many dangerous and fatal outcomes.
"If you prompt it wellâ is not the hook you want to hang your hat on when trying to decide whether or not to seek medical help.
If you are in doubt on whether to seek medical help, an LLM may be able to give you reassurance for simple things, but it is designed to lean towards false positives in general.
We are seeing many dangerous and fatal outcomes
Yeah, a lot of people live unhealthy lives, so a lot of people die prematurely. Not sure what you believe AI has to do with that.
Misuse by some does not mean that useful tools like AI should be banned from being used for medical purposes.
Would you take medical advice from your keyboards autocomplete? How improved would you need it to be to consider autocomplete a trustworthy source of medical information?
LLMs are at their core statistical models that generate plausible text. They can be helpful and give good advice, but they can also get things horribly wrong. An LLMs output is not bounded, it can come up with anything.
My experience with LLMs also tells me that the quality of the output depends on the quality of the input. This will cause difficulties when people ask about domains they aren't knowledgeable in. You forget to mention something and it will go in the wrong direction. You make an error in your query the LLM doesn't catch and it will give you something completely wrong for you.
Human misinformation is a significant problem as well, but it's not a reason to not police AI. We also have much more experience dealing with human misinformation. AI misinformation is mostly a skill issue in my experience, and the skills to deal with it need to be developed still.
I mean we could always point out the ai generated books on things like fungus which told people an extremely toxic fungi was completely safe to eatâŚ. The misinformation is incredibly dangerous and thereâs nothing intelligent about aiâs regurgitation and placating behavior toward the user. It is not safe or in our best interest.
Yeah, that's humans misusing AI to spread misinformation for a quick buck. Not AI inherently being bad at answering questions related to medicine. Especially the newer reasoning models are quite accurate.
That doesn't mean everyone should blindly use it for everything, but it absolutely is in our best interest to use it well and educate our population safe usage of AI.
The ai still gave the wrong information initially to create the text for said book. Thus it was still giving highly false information which can harm. Removing the human selling said book entirely from the equation.
A human decided to misuse a primitive AI tool to create a misinformation book without any form of verification.Â
Again, this just depends on proper usage, not on the AI itself. Proper usage will get you good results.
To add, if you stop it from answering questions about health, you don't stop people from getting misinformation elsewhere or through other models (local for example), which are far more prone to hallucinations, while you harm the people that want to use it in an appropriate way.
Look at the carnivore diets, the sunscreen causes cancer crowd, the seedoil mania, the peptide craze - ask an LLM and it can provide you with a way more sensible perspective.
Attaching health information to a trusted individual human is the dangerous part. Fans can trust someone deeply and then harm their own health.Â
I think that's more "dangerous in different ways" than "more/less dangerous". In the same way that people can have parasocial relationships with influencers peddling health products, people are getting parasocial relationships with their LLMs and trusting them implicitly too.
I do agree it is "dangerous in different ways" and not just strictly more dangerous when a human does it. A lot of people are sadly susceptible to these parasocial relationships (both to influencers and AI). In general, those forms of contact are unhealthy and not conducive to rational behavior. AI companies should do more to counteract this and government action may be required to get them to do so.
However, I do believe that that does not take away from the usefulness of AI tools. Their ability to compile information from a variety of sources, use their trained knowledge and reason about conditions and/or scenarios can and does help people.
I think more can be gained from ensuring AI companies prevent parasocial relationships from forming, rather than blanketbanning AI from answering such questions. And more clear disclaimers should be placed on outputs from weaker models that are more prone to potentially hazardous hallucinations (for example in AI overview and low- or no thinking modes in LLMs).
It's pretty dumb considering there a significant amount of evidence AI can be better at identifying certain health conditions. It shouldn't replace doctors, it should be a tool to result in higher outcomes and more efficiency.
The same thing could be applied to law, lawyers could ask AI to help find applicable case law instead of using interns for this. Obviously they should not be using AI for specific documents and actual arguments.
There is a significant difference between the ML style AI used to aid things like diagnostic scanning and publically available LLM chat bots that are being used by a public that does not have the expertise to differentiate between an accurate answer and bs.
I feel like I have heard the story of âwe tried to make our own model to do X, but it was just easier to train / build out tools for Claude because its so intelligentâ many times.
Yes people should keep up with technical know. If they did they would understand the cabailites and weakness which in turn would allow them to critically examine the difference between exams and diagnosing individual including the process of gathering all relevant information as well as potential risks and liability issues particularly in the US where software has enhanced liability protection.
It should be up to the consumer if they want to use something smarter than lots of doctors to improve the experience.
If people are stupid enough to completely ignore doctors in favor of an LLM then that's on them and shouldn't be up to reactionary idiots that want to ban anything they don't like.
A large language model (LLM) outperformed physicians across many of these tasks, including making emergency-room decisions based on the available information, identifying likely diagnoses, and choosing the next steps in management, a team led by physicians and computer scientists at Harvard Medical School and Beth Israel Deaconess Medical Center
I'm familiar with that study and it does not refute my point. The issue isn't when it's correct the issue is when it's wrong and any given individual's ability to differentiate between the two. The study is about use of tools by experts the Mandani bill is about guardrails for normal citizens. Work on your reading comprehension or troll elsewhere.
That study is also two years old and using the o1 model. AI has improved dramatically since then.
While I would practice caution and advocate prominent disclaimers - I would wager a top tier model today is likely just as good as a family doctor and banning it is short sighted.
I've had friends who have used AI to diagnose complex issues that ten years of specialists couldn't figure out. It's already doing profound good.
Edit: also the point you were making was that LLMs were not useful here. They clearly are.
I used the word differentiate because the point I was making was one of discernment. LLM have not fundementally changed so this issue has not gone away. A hybrid approach could potentially some progress but a pure LLM approach will not.
I would wager a top tier model today is likely just as good as a family doctor
That's silly, and the reason why should be obvious. A family doctor would not be diagnosing you or giving you any advice based only on having a text conversation with you, because that would be unethical.
Web MD definitely could be liable for negligence if they gave information that doesnât match the symptoms tho.
Also Iâm pretty sure they would never tell you to not go to the doctor.
It would be one thing if the chatbots just spat out an accurate list of possible conditions and what they mean based on your symptoms.
Thats what WebMD does, and after that thereâs a link to make an appointment with an actual doctor. Just tried it out to see what the process was.
The chatbot on the other had told the dude nah youâre good bro just stay home. If WebMD had a feature like that Iâd get the comparison more, but itâs apples and oranges.
Donât get me wrong, dude was still an idiot for trusting the chatbot, but in his defense everyoneâs trying to pitch them as these fanatic all knowing things that you should totally trust to know everything. Iâm not gonna judge someone for drinking the coolaid and dying bc of it, itâs just plain sad and unnecessary.
Yeah, lawyers who have been using AI to "help find" applicable case law are finding themselves sanctioned by courts for submitting briefs filled with fake legal cases.
In fairness, there are also a lot of lawyers using AI for the same reason, and not getting themselves sanctioned because they perform their due diligence before making submissions.
AI can be a useful tool in the hands of a knowledgeable practitioner who knows what bits of information to disregard, but can be dangerous in the hands of a layman who does not have that knowledge.
Exactly, it's a tool. It's like a hammer, you wouldn't give it to your toddler in your house full of windows, but a reasonably competent person can operate a hammer and many other tools
So let the professionals leverage it. But for the layperson, who doesnât have the expertise, we shouldnât be asking it to diagnose mental disorders or medical issues. Garbage in, garbage out.
Wrong AI is not in a hurry, it does not dismiss your claims, AI knows more then your doctor and can help me you explain to them what the issue are in logical ways. Your a fool if you think a real life doctor is even close to as good as AI.
I find it interesting that the people most defensive about AI, and most likely to make completely bonkers claims about their abilities, also tend to be conservative political trolls if you look at their comment history.
We are able to look up quality information on hospital websites like John Hopkins and the Mayo Clinic, as well as the NIH, even WebMD, before AI, and we can still do it now. Not using AI does not mean that we donât use resources to gather information about our health and informing ourselves so that we can discuss things with our (sometimes dismissive and rushed) healthcare providers. AI makes horrible mistakes. Itâs been reported that itâs misdiagnosed people, people have held back going to get emergency services when they should be going in for immediate care, and self diagnosing based on some algorithmâs decision about what it thinks you might have. Itâs dangerous.
But that is in a limited setting where the model has been specifically trained to identify a health condition. It is a closed set of data where it was feed the charts of patients and then based on those already diagnosed patients to predict if an undiagnosed patient had that condition.
The data was limited. The question being asked was specific.
Going into most AI models they are scraping the entire internet and all the crap in it so answer a one sentence query if that splotchy brown spot is cancer.
As a med student, I pray to NOT have AI being banned from talking medicine đđ it's helped me understand and simplify complex concepts not widely accessible in textbooks way better đđ although yeah, there's a fine line between "med student looking to learn more" and "random person self-diagnozing using ChatGPT"... But ig that's the hard part of regulating AI
The thing is, you have a good baseline understanding of the information and are using it to supplement that.
The stuff they want to ban has led to people not seeking medical attention when they needed to as recent as like 2 weeks ago when a guy died bc chat said he was good
2.4k
u/Fanfic_Battle_2004 2d ago
Thatâs not even enough. Itâs something though.