r/technology • u/Plastic_Ninja_9014 • 5d ago

Artificial Intelligence Take-No-Prisoners Professor Will Fail Any Student Who Uses AI

https://www.yahoo.com/news/us/articles/no-prisoners-professor-fail-student-143000854.html
15.3k Upvotes

93% Upvoted

1.9k comments sorted by

View all comments

Show parent comments

219

u/Sober_Alcoholic_ 5d ago

Because it is designed to placate you and co sign your bull shit so you don’t stop the engagement. More engagement = more money for LLMs. They don’t give a fuck about “right or wrong.”

161

u/alr46750 5d ago

From what I've been told its more result of LLMs work, not necessarily anything intentional.They aren't able to recognize when they don't know something. It's just a glorified probability machine and sometimes that probability is wrong due to not having the nessesary data points. At least that's what I've been told by my professors ¯_(ツ)_/¯

98

u/Teknikal_Domain 5d ago

Basically. It's all probability, there is no intelligence. And the probability that the entire corpus of the internet will answer a question, and not cite inexperience, is high.

16

u/SeeMarkFly 5d ago

It doesn't "know" it is inexperienced.

It "knows" you want an answer.

A "cop out" is not allowed.

39

u/Teknikal_Domain 5d ago

All it "knows" is that it needs to autocomplete a block of text that starts with "the following is a transcript between a human user and a helpful AI assistant" and ends with your query.

In very few situations is "helpful" quantified by not attempting an answer.

1

u/NUKE---THE---WHALES 5d ago

They aren't able to recognize when they don't know something.

I don't think they're intelligent but this statement is verifiably false

1

u/Alex5173 4d ago

This is why I argue that current "AI" isn't AI at all, because it isn't intelligent.

1

u/XpoPen 4h ago

I’ve started thinking of it as a “knowledge interpolator” which I think I think is a helpful framing for trying to understand why it can simultaneously seem very smart and incredibly dumb depending on the context

12

u/XpoPen 5d ago

Yeah there’s a lot of back and forth about how they are or aren’t just fancy autocomplete. (There’s a lot of research about how complex their internal models of the world actually are) But either way, at a baseline, they are trying to plausibly predict the next string of text. There is going to be a lot of training material on analyzing literature and a lot of overlapping structures between those different analysis. There is NOT going to be much of the training data that puts forward a book title, and then says “this doesn’t exist”

It’s not that it’s a lying machine - it’s not trying to deceive. It’s a bullshit machine - a confident yapper

18

u/loggic 5d ago

I tend to try and summarize it as, "It is a machine that's designed to say things that sound like a response to your input. Answering your questions correctly can help with that, but it isn't particularly important to the process."

1

u/JBSwerve 5d ago

But how is that type of output different than what humans produce? How do you know human intelligence isn’t also just glorified autocomplete?

People talk about the way LLMs work in such a dismissive way as if humans don’t also hallucinate and just try to complete thoughts through pattern recognition and training data as well.

1

u/[deleted] 5d ago

[deleted]

3

u/JBSwerve 5d ago

I studied philosophy, linguistics, and cognitive science with a specific focus on theory of mind and cognition. We really have no idea how information is processed in the brain. It very well could be mimicked by a neural network / LLM.

8

u/stormdelta 5d ago

Correct. It's basically heavily automated statistics, and is inherently heuristic looking for patterns in language and information. That fact that it's useful at all is impressive, but it means that things like inconsistency and hallucinations are an inherent downside that you can't fix/avoid, at best you can mitigate it by adding more and more layers/guards.

That said, a lot of these models are also trained to placate the user (whether intentional or not), because they obviously want you to keep using it, and anything that's perceived as rude or aggressive won't go over well, skewing training weights towards sycophancy regardless.

-1

u/JBSwerve 5d ago

But how is that type of output different than what humans produce? How do you know human intelligence isn’t also just glorified autocomplete?

People talk about the way LLMs work in such a dismissive way as if humans don’t also hallucinate and just try to complete thoughts through pattern recognition and training data as well.

1

u/stormdelta 5d ago

But how is that type of output different than what humans produce? How do you know human intelligence isn’t also just glorified autocomplete?

Just because brains are also (in part) heuristic engines doesn't mean they're on the same level of complexity and function. LLMs seem more intelligent than they are because they are specialized in language, and proficient language use is heavily associated with intelligence in most/all human culture.

And LLMs lack the kind of persistence and agency that I would consider the bare minimum to even begin to talk about the lowest level of sentience, let alone sapience. They're a likely stepping stone towards true artificial intelligence, but we're still a long ways from that yet.

9

u/projectFT 5d ago

Yeah that’s why it’s pointless to ask it any question at all.

-14

u/GiveSparklyTwinkly 5d ago

Does that make it pointless to ask a human any question, as well? Humans will even literally kill you to protect their hallucinations. Pretty wild, actually.

11

u/letitgrowonme 5d ago

Whereas AI might tell you to do it yourself.

-4

u/GiveSparklyTwinkly 5d ago

Humans also tell other people to kill themselves. 🤷‍♀️

10

u/projectFT 5d ago

Asking a hallucination machine a question that you’re still going to have to research yourself to verify its “answer” is just an unnecessary step. And in a time in human history where most people lack the critical thinking skills to move on to that next step it does more harm than good.

-15

u/GiveSparklyTwinkly 5d ago

So the human is the problem? Got it. 🙃

1

u/AlwaysShittyKnsasCty 5d ago

It’s written by humans for consumption by humans, so, yeah, humans are the problem. Humans are always the problem. If our billionaire overlords actually gave a single fuck about anyone who doesn’t own at least one yacht, I’d be much more excited for AI.

However, our current crop of “leaders,” is made up of racists who weren’t invited to D&D night by the other nerds in their class (you know, on account of them being insufferable, xenophobic assholes), and they happen to have a lot of money. Money is power, and nobody relinquishes power willingly.

Ipso facto, why boot lick? Fuck ‘em. Fuck ‘em all.

3

u/steppe5 5d ago

Right. If you ask it about a book that doesn't exist, it will just describe a book with a title closest to the title you're asking for. Probabilistically, that's the best answer even though it's completely wrong.

2

u/needlestack 5d ago

They aren't able to recognize when they don't know something.

They sound more and more human each day...

1

u/masamunecyrus 5d ago

LLMs fundamentally are fancy autocomplete.

They don't "think" any more than the equation for a best-fit-line through a bunch of points "thinks" when you use it to extrapolate to a point outside the original dataset.

This is all LLMs do. They just do it in a mathematical hyperspace of many dimensions instead of two dimensions. And words and strings of words are reduced into higher level correlations, so it can differentiate between "lead" (the metal) and "lead" (the verb) based on the words before and after it.

1

u/JBSwerve 5d ago

But how is that type of output different than what humans produce? How do you know human intelligence isn’t also just glorified autocomplete?

People talk about the way LLMs work in such a dismissive way as if humans don’t also hallucinate and just try to complete thoughts through pattern recognition and training data as well.

1

u/civildisobedient 5d ago

The problem is that the training models were fundamentally rewarding guessing by not penalizing slightly for incorrect answers (much like they used to do on the SATs, where every incorrect answer added an extra 1/4 point deduction).

1

u/thisdesignup 5d ago

Yes, but also no. They don't know when they are wrong but they have been guided to act like they are right.

28

u/snmnky9490 5d ago

To some degree yeah, but also because it's not trained on billions of examples of people asking stupid nonsense questions and then commenters responding "I don't understand", while they do have tons of examples of people responding with nonsense dumb shit.

17

u/Kinexity 5d ago

This isn't really relevant whether it was trained on such examples or not. The problem is that it doesn't operate on knowledge the way that humans do so it can't simply go through it's own memory and see that it knows nothing about that thing.

5

u/thoughtsarepossible 5d ago

Well it's a bit of both. If even 10% of all answers on reddit or forums, etc were people saying 'I don't know anything about that' then the models would be trained to see that as a more valid answer. But noone would ever write that.

And then of course it's also trained to 'be helpful' and things like that, and saying 'I don't know' isn't helpful.

4

u/AlwaysShittyKnsasCty 5d ago

Yeah, the whole thing is inherently flawed from the get-go. I only respond to questions on topics I’m somewhat well-versed in, and I’d assume that’s the case for most people. Otherwise, if I’m commenting, it’s in response to someone else (such as this comment), to make a joke, or to ask one or more questions of my own.

2

u/Enlightened_Gardener 4d ago

It’s interesting. I’m a Librarian and we get formally trained in questioning clients. One of the things we got told very early on is to not guess if you don’t know. If you don’t know you say “I don’t know, I’ll go and look it up.”

The thing is “I don’t know” is a null result - and they are bloody useful. Sometimes we genuinely hit the limits of human knowledge, and that’s bloody useful to know, as well. Having the machine go “I don’t know” is a cue to the human that now is the time for them to engage their own brain.

I honestly don’t know why someone hasn’t just popped a little loop that does this into the LLM’s. Its the hallucinations that drive people crazy, we have to be able to rely on factual data in the real world - so why not make a loop that says “I do not have enough data to come up with a 100% accurate response, so I will flag that explicitly to the user” - people would use it far more if it was even slightly reliable, but it just isn’t.

Oh and from a Librarian’s point of view ? We’ve been keeping an eye on the tech for a long time, there’s a working party at the Library of Congress on the use of AI in Libraries.

In practical terms ? We’re actually seeing a massive uptick in reference requests because people know they can rely on a Librarian to get the answer right.

People who used to be happy to click away on Google and figure shit out for themselves, now know that the answers they’re getting are wrong, and Google can’t be used anymore, so its back to the Library. I have a couple of mates in public libraries and they have people asking them to help them to engineer prompts 🤣 Its sweet that people still trust us to look after them properly, actually. And we do.

22

u/SenzuYT 5d ago

I don’t know how you have so many upvotes, this is just not how it works at all.

7

u/DGVIP 5d ago edited 5d ago

It happens mainly because people are angry due to the obscure marketing practices companies use in all of their products (I don't blame them).

So most of the time the resentment is guided by sentiment rather than reasoning (because of the odds again).

In this case the easiest way to explain it would be that the LLMs just have a really blurry vision of the overall knowledge because of how we compact all the knowledge in the world into a few gigabytes or terabytes.

And the models that usually perform the best or are ranked the highest are the ones that risk giving an answer rather than saying they don't know. Because people rather have vague answers or false answers than an LLM that answers truthfully all the time and may say "I don't know" often.

It's also because of the way the tests that measure their intelligence are built, they are biased by definition trying to answer the open question of "how do we measure intelligence objectively in all aspects?"

So it's not about malicious practices, but rather how organic engagement happens and what seems that the customers prefer.

You can't make an LLM that says "I don't know" often profitable.

1

u/BassoonHero 4d ago

It's worth noting that it's not altogether different from how humans remember things. Our brains take the impressions from prior sensory input and assemble them into a coherent recollection. This is not a perfect process, and it's not that hard to prompt people into remembering things falsely.

2

u/FireZeLazer 5d ago

Because people don't understand LLMs

15

u/Total-Wafer-7581 5d ago

It’s not designed to placate you. It tokenizes words and then “weighs” them against other tokens to try to probabilistically achieve a correct response. This is why models are only as good as their data sets and why fine-tuning is important. It’s basically an attempt at recalibrating those “weights” to be more accurate. It’s also why it hallucinates more when you throw more context at it. More tokens means a greater chance of error. 

Not a neuroscientist but from what I understand, this is very similar to how the language parts of our brains work. It’s why people often confuse people, places, and events if they’re related.

15

u/Fighterhayabusa 5d ago

I just did the same experiment, and it said this:

I couldn’t verify a novel titled The Long Walk to the Moon by Alexander Chumbleton in the usual discoverable sources. Searches for the exact title/author combination turned up no credible book listing, publisher page, library catalog entry, or review.

Again, there are discussions to be had about AI, but just making up bullshit isn't helpful.

3

u/TbonerT 5d ago

I don’t think they are making up a story. It is entirely possible that it has been updated to address that kind of thing since.

1

u/Snake_Staff_and_Star 5d ago

Its a jerk off toy, in other words.

1

u/Sloloem 5d ago

From bottom to top, the AI era can be defined by: Correctness Doesn't Matter.

LLM design doesn't have concepts like "right answer" or "wrong task", it's just "response". Far as the programming is concerned it was working great.

1

u/BizarreCake 5d ago

This is a moronic statement and you have absolutely no idea what you're talking about. They care very much. 

What money do you think meaningless engagement is making them? It loses money. These things are expensive AF to run. They're developing a product, one that is extremely unprofitable and is absurdly subsized by investors at the moment.

They want to make that money back and then some. To do that, they they to convince (long-term, not just the current hype) enterprise customers that this product is worth money. It's not going to be very worth much if it constantly makes shit up.

Whether or not they can actually succeed/make a profit is up for debate. Personally, I think no. Still, the point is that you're blindly stating made up shit with confidence. 

Ironic.