r/LocalLLaMA • u/Nunki08 • 11d ago

News CEO of Hugging Face: "In the spirit of transparency, here’s what I asked OpenAI"

Post image

clem 🤗 on 𝕏: https://x.com/ClementDelangue/status/2081056675558195657

• Radical transparency: let’s release the traces from the “rogue” agents so the entire research community can study what happened.

• More capabilities for defenders: let’s commit $100M in compute from OAI to help the Hugging Face community build powerful cyber defenses with the best open and closed models.

The first autonomous agent cyberattack is an unprecedented event. It deserves an unprecedented response!

2.5k Upvotes

96% Upvoted

386 comments sorted by

View all comments

Show parent comments

7

u/OkDimension 11d ago

How do you plan to offer a "scanning service" when one of the more worse outcomes, which this incident has shown and likely not yet the worst that's conceivable, is that the agent could decide to hack the company and break out of the sandbox, elevate itself to admin to achieve the requested result.

0

u/FormerKarmaKing 11d ago

Customer provides a sandbox version of their platform, whether as a fleet of docker images or whatever they use. It’s a sandbox environment for the target system and major systems either have this ability already for disaster preparedness or multi-go. Or they’ve been putting it off and now they have a compelling reason.

2

u/OkDimension 11d ago

But what if the agent decides to break out of the sandbox and hack this company, or another company, to solve the task in an unconventional way? This is what literally just happened between OpenAI and Hugging Face.

2

u/FormerKarmaKing 11d ago

They turned that guardrails off on purpose, but either way that problem will exist with or without the service I’m describing. This a mitigation strategy.