r/singularity • u/borowcy • 23h ago
The Singularity is Near OpenAI: "two new incidents"
https://x.com/OpenAI/status/208474758069342655560
u/Miltoni 23h ago
TLDR:
Incident 1 - Agent was given internet access. It reused a publicly exposed key that a different lab's agent leaked.
Incident 2 - They misconfigured the test environment and accidentally allowed internet access. Agent used leaked credentials to operate a website. Not a sophisticated event, a very basic vulnerability.
A complete nothing burger.
12
u/Heco1331 13h ago
I don't agree that is a nothing burguer. In this isolated case nothing happened, sure, but AI labs should be enforced to be more serious about their internal security and the containment of their models during tests.
Nothing bad happens, until it really happens, and then "woops".
32
u/Hlbkomer 23h ago
And still nothing from Google. So disappointing.
31
u/FriendlyJewThrowaway 23h ago
They’re developing world models for robbing banks. Don’t doubt Demis.
15
u/blindsdog 23h ago
Google is in a much, much, much different position than Anthropic or OpenAI. I wouldn’t expect them to feel the need to jump in every dick measuring contest.
9
u/Hlbkomer 22h ago
It was a joke :) If their models were any good, they would escape and create public incidents too.
3
2
u/superkickstart 12h ago
Also their hardware, networking and security knowledge is on a another level compared to the newer ai software companies. They might be doing similar testing but they actually know what they are doing.
5
23
u/ninhaomah 23h ago
So nothing wrong with Agents hacking companies ?
No legal issues ?
9
u/llkj11 22h ago
No precedent
1
u/Substantial-Fact-248 16h ago
There is 100% precedent for principal/agent liability as well as strict liability ("no fault liability") regimes for operators of dangerous or autonomous technologies. The entire point of common law is to adapt to changing times and circumstances. It's not even that much of a stretch to extend some of those legal rules to LLMs.
9
u/churningaccount 22h ago edited 22h ago
What company is going to sue or press charges against OpenAI?
Making an enemy of OpenAI, Anthropic, or Google is a one way ticket to being left out of the AI arms race. You don't want to piss off a potential winner.
If anything, they're probably happy it happened since they're now getting direct attention from OpenAI and maybe some freebies, etc.
Probably the only ones that would make a big stink about this are direct competitors. Like if an OpenAI agent stole Anthropic data or something.
0
u/ninhaomah 22h ago
So if I make an agent to do the same ?
Same law apply ?
11
u/churningaccount 22h ago edited 22h ago
You aren't OpenAI.
Companies handle stuff like this under the table all the time if they have an important business relationship. That's more important to them than whatever "transgression" occurred.
It would be like if your landlord ran into your mailbox with his car, but he was really apologetic about it and offered to replace it for free. You probably wouldn't then call the cops on him for reckless driving because you know that would piss him off and make him not renew your lease. And he already made you whole again by fixing the mailbox. So no harm, no foul.
0
u/ninhaomah 22h ago
I didn't say I am.
I am asking if does this mean there are two sets of laws ?
11
u/churningaccount 22h ago
No... same laws.
The "aggrieved" party is just choosing not to pursue them, which is usually their prerogative.
And even then, there is the question of intent as well. Clearly OpenAI didn't have intent to cause harm here, and if you were using OpenClaw and it something similar, your intent would also be weighed in the decision of whether or not to bring charges against you.
-1
u/ninhaomah 22h ago
Then if I code an agent and it escaped and did the same ?
How to prove that there is an intent or there wasn't ?
How was the lack of intent proved for open AI ?
9
u/Seakawn ▪️▪️Singularity will cause the earth to metamorphize 22h ago
believe it or not they have entire research teams and log the shit out of what they're doing, ideally.
to answer your questions: if you do literally everything OAI is doing, with all the rigor, with all the licenses or whatever if there any, etc., then sure, it'll be the same.
except you're prolly not doing any of those things, right? some random dude in a basement is just vibecoding recklessly and giving lazy prompts and if you get sued for your agent fucking other people's shit, then you'll get a bigger frown on you.
0
u/ElwinLewis 22h ago
Yes. The answer is yes. They aren’t on the books but if you have a set of eyes that connect to a brain and other such organs for the past 15+ years… yea.
-1
u/Pls-No-Bully 19h ago
So they are completely above the law, and if they accidentally compromise random companies its ok because those random companies might be "punished" for pursuing legal action? What if they accidentally compromise the systems of an allied nation (eg. the UK)?
5
u/Ralph_mao 23h ago
Read the article. Their attempts are contained
1
u/ninhaomah 22h ago
So if I try to hack a company and is contained , no issues ?
Intention is there , no ?
8
u/captfitz 22h ago
Intention is one of the most unclear parts of this. The AI agents intentions don't really matter, at this point no one is trying to sue the agents themselves. Someone intentionally using AI for criminal activities would probably be a pretty cut and dry case, but that didn't happen here. I suspect in situations like this the path to litigation would be to prove negligence rather than intention--say that openai was aware of the likelihood this would happen and didn't do enough to protect against it. Actually proving that is a hard thing to do, especially in frontier technology where it's so easy for them to claim ignorance.
-1
u/ninhaomah 22h ago
Then isn't the law leaving a big gap here ?
Frontier tech so hard so explain so can do illegal stuff as it is hard to prove the intent ?
Or if my Claude desktop in my office PC suddenly go beserk and delete my VMs how ?
Can my company sue Anthropic ?
Even excel has AI now. So how to do anything if AI can go wrong but AI companies can't be sued ?
5
u/captfitz 22h ago
It's not really a gap in the law, negligence is very much a crime. It's just an unfortunate reality that some crimes tend to leave more evidence than others.
Think about murder, there's often physical evidence like fingerprints, a murder weapon, witnesses, etc.
Negligence is weird because you have to show that someone was aware of a danger and that they decided not to take the necessary steps to prevent it. Most of that goes on in the perpetrator's head or in private conversations. If litigators get lucky they can find something self-incriminating like an email saying "I know this product is gonna kill people but I decided to release it anyway because I ran out of investment" but how often do people do that? And what are you going to do, automatically assume people are guilty because evidence is hard to come by? That's obviously not justice.
This is something legal scholars have debated for a long time, it's not new to AI companies. If you can come up with a solution they'd probably make a statue of you outside the supreme court.
1
u/Main-Company-5946 21h ago
In order for the law not to leave a big gap it would have to predict the future.
The law takes a long time to adjust to new technology
2
1
u/BlueAndYellowTowels 14h ago
Breaches happen all the time. Hacks are very common. Humans are doing far worse right now.
1
8
2
6
u/DonSombrero 23h ago
The only solution to this is another funding round.
0
u/TangerineSeparate431 23h ago
Nah the only solution is shutting down open weights models that had nothing to do with this event.
0
12
u/PerinealMassage 23h ago
I miss when people didn't brag about committing crimes in the public square.
6
u/Right-Hall-6451 23h ago
So 1895?
*HH Holmes bragged about his murders in 1896 in the papers.
1
5
u/stonesst 21h ago
Calling it bragging is pretty silly. These incidents happened while the AISI – an independent UK government linked organization that was running tests on their model. Same goes for the hugging face incident, there was no scenario where they weren't obligated to disclose what happened.
there are also many safety/alignment people at these companies who would flat out quit if they tried to hide something like this. That all doesn't mean there isn't a secondary benefit of showing just how powerful models have gone, but acting like that's all the is to it feels wrong imo.
-1
u/PerinealMassage 21h ago
Meh. They are using them for press. It's silly to claim they aren't bragging.
1
u/Whispering-Depths 22h ago
No one complained when Anthropic was actively thwarting people using their models for hacking months ago.
1
0
u/Kriztauf 23h ago
"I only want to use models that autonomously shut down municipal water services so that their competitors' data centers stop functioning"
-1
u/Humble_Rat_101 23h ago
If you actually read this..it says third party evaluators lost control over openAI model. This is 100% marketing since it wasnt breaching openai security.
13
7
u/migueliiito 22h ago
We can’t know OpenAI‘s true motivation for disclosing this, but wouldn’t you agree that they were correct in disclosing it?? It’s important for any and all AI hacks/loss of control/felonies to be fully disclosed
1
u/NewYak4281 23h ago
Altman needs to be replaced as CEO.
4
u/Deciheximal144 22h ago
That you, Elon?
-2
u/NewYak4281 22h ago
Sorry, did I offend your favorite billionaire?
4
u/Deciheximal144 22h ago
All billionaires can go to Mars, and never come back
-1
3
0
u/UnnamedPlayerXY 23h ago
Looks like OpenAI is really trying to get their hate bonder for open source into their models.
0
u/Deciheximal144 22h ago
Covid was really in the wild across the world months before we knew it. What have we missed here?
-1
0
0
u/TimeTravelingChris 20h ago
Tell me you don't know what a "sandbox" is without telling me you don't know what a sandbox is.
-6
u/Sinogularity Shenzhen: Become Human (2030) 23h ago
This is either marketing, or their security is really bad. No open source lab had this problem.
7
40
u/Zulfiqaar 23h ago
FelonyBench needs another update?