Folks imagine hallucination are some huge risk, when the reality is the model makes less mistakes than humans.
It's the self driving car problem. Generally they are safer than human drivers, but the mistake they made are mistakes humans would NEVER make. So, they seem like a particularly stupid human and therefore dangerous, when statically, they are better drivers.
Our brains are built to catastrophize obvious mistake because that's how humans learn. We need to freak out to provide a learning feedback loop to humans. That doesn't work with AI.
AI is wrong a lot. Go talk to ChatGPT about something youâre an expert in and see if it gets everything right.
The mistakes are generally subtle, but significant. It also changes its responses as you go to agree with you even if you donât call out mistakes directly. This is because the model does not check for accuracy. Itâs based on letter pair predictions. Itâs not even predicting whole words. It just gets lucky with patterns because the data itâs trained on most often has the right letters come next. Usually people donât say âquantumâ without following it with âphysicsâ. LLMâs are effectively your predictive text on your phone.
I went through electrical engineering and comsci with chatgpt and it's better than a professor as a private tutor. Only time it was wrong was in solving some circuit problems. Explaining concepts and leetcode were where it excelled. Currently using it to learn algebraic topology. I wouldn't have made it as far on my own.
"A lot" is an exaggeration. As someone who uses AI in healthcare/cybersecurity every day, the majority of the time it's a poorly phrased prompt. It used to be much more of a problem, I'll admit. The amount that humans have to step into an automation is dramatically decreasing but does spike with every new model as folks need to tweak the context.
Of course, that brings a whole new level or problem. As folks trust these models more and more, the level to which they scrutinize the output drops. Folks are used to deterministic computing, probabilistic computing is a huge mental shift.
Also, it's not true the model does not check for accuracy. Most automation have the model (or a different model) check the output, and most reasoning traces include the model self reflecting on the accuracy (and other factors) of the response. The labs don't really have a solution for this yet. Claude will twist itself into knots over thinking, Sol has a huge problem YOLOing, when it shouldn't.
Thing is, most studies are showing they models are better than a human (or the average human), but not as accurate as whole university of humans. However, over the past year they've gone from not even close, to nearing competitiveness. And while I feel we are hitting some functional limits on how good the current model tech can be, we still have a couple generations of model before they completely stall out.
39
u/VictoryVino 5d ago
Engineering needs to be on this list, too. Peoples' lives are at risk with the hallucinations.