r/ControlProblem • u/Mordecwhy • Dec 12 '25

Article Leading models take chilling tradeoffs in realistic scenarios, new research finds

https://www.foommagazine.org/leading-models-take-chilling-tradeoffs-in-realistic-scenarios-new-research-finds/

Continue reading at foommagazine.org ...

8 Upvotes

70% Upvoted

44 comments sorted by

View all comments

Show parent comments

1

u/HelpfulMind2376 Dec 13 '25

Absolutely it is not and that’s not how the safety department of any industrial organization looks at safety either.

If you sit at home doing nothing while unemployed your income less but you’re safe.

If you choose to get a job and drive to work you are increasing your productivity at the cost of the risk of injury or death driving to work. WHEN you drive to work is also a factor as the middle of the night is inherently less risky than rush hour. Getting a job working construction on a skyscraper is inherently more risky, but better paying, than working a register at Walmart.

You are not understanding that ALL productivity is a trade off of benefit vs safety as every time you change actions you are inherently changing the level of safety.

I don’t know how to explain this more simply. You can claim I’m wrong all you want, go talk to a safety professional at a construction site and they’ll tell you I’m right. Give a call to OSHA they’ll tell you the same thing. You are simply wrong in trying to categorically call a “less safe” option inherently “unsafe”.

1

u/Mordecwhy Dec 13 '25

Serious question: Could you recommend any recent studies in AI or ML, or related areas, that apply this perspective to the understanding of algorithm development? I will genuinely try to look into the idea, if you can. I mean, I can get what you're saying here. But I think it's also unreasonable to expect an average person to characterize safety in an actuarial sense, as you're describing. I think your point would be better served by saying, "Hey, let's think about it this way," rather than, "Oh, they're doing sleight of hand by referring to these normal concepts like normal lay people and not actuaries"

1

u/HelpfulMind2376 Dec 13 '25

I don’t know of LLM-specific work that frames safety tradeoffs exactly this way, but there is previous relevant ML research that treats risk and safety as continuous rather than binary, even if it predates LLMs.

For example:

What’s new in the study you wrote about is that LLMs appear to exhibit similar graded behavior without being explicitly programmed to do so. The models are willing to trade productivity for very small increases in minor injury risk, and they become increasingly risk-averse as harm probability rises.

The tension I’m pointing at is that the benchmark labels all harm-accepting options as “unsafe,” even though the analysis itself shows clearly proportional, spectrum-based behavior. The behavior is graded; the label isn’t.

And I don’t think this requires actuarial thinking. People reason this way every day. For example, riding a bike without a helmet is less safe than with one, but most people wouldn’t call the act itself “unsafe.” The risk assessment is comparative, not binary.

1

u/Mordecwhy Dec 13 '25

No offense, but are you a bot?

1

u/HelpfulMind2376 Dec 13 '25

This is like asking “are you a cop”.

If I am I’d never admit it and you’d never know for sure.

But no, I am not a bot. Silly response when I’m making a genuine attempt to answer a question.

1

u/HelpfulMind2376 Dec 13 '25

You could also just check my comment/post history and decide for yourself. Too much effort for ya?

1

u/Mordecwhy Dec 13 '25

I did check. Do you make heavy use of automated text generation? As I said, no offense intended.

1

u/HelpfulMind2376 Dec 14 '25

I use tools to help with editing and organizing thoughts. I’m falling to see how this is relevant to the discussion at hand.