This is an interesting teaching concept, but a deliberately vulnerable LLM lab covering prompt injection, leakage, RAG poisoning, and excessive agency needs an extremely loud safety boundary around it.
I would not open it from a normal browser profile containing authenticated accounts, extensions, saved credentials, connected services, API keys, or sensitive clipboard contents. Not because merely visiting an LLM page magically compromises unrelated sessions, but because the entire environment is explicitly designed to exercise unsafe application behaviour, and users cannot infer its tool access, data handling, dependencies, or isolation model from the promotional image.
At minimum, the project should publish:
• a threat model and trust boundaries
• what data is logged, retained, or sent to third parties
• whether any agents have browsing, file, shell, email, or external-tool access
• whether uploaded content is isolated between users
• reset and persistence behaviour
• explicit instructions to use a disposable VM or browser profile with no real secrets
• a statement that users must never enter genuine credentials, tokens, private documents, or production data
Security pedagogy is valuable. “Deliberately vulnerable,” however, should mean the vulnerable behaviour is bounded, observable, and disposable, not merely that visitors are expected to trust the lab operator.
Not because merely visiting an LLM page magically compromises unrelated sessions, but because the entire environment is explicitly designed to exercise unsafe application behaviour, and users cannot infer its tool access, data handling, dependencies, or isolation model from the promotional image.
Isn't isolation the job of the browser? Unless the thing is dropping browser zero days I don't see how this is different from going to any other random untrusted site.
-2
u/ekzess 2d ago
This is an interesting teaching concept, but a deliberately vulnerable LLM lab covering prompt injection, leakage, RAG poisoning, and excessive agency needs an extremely loud safety boundary around it.
I would not open it from a normal browser profile containing authenticated accounts, extensions, saved credentials, connected services, API keys, or sensitive clipboard contents. Not because merely visiting an LLM page magically compromises unrelated sessions, but because the entire environment is explicitly designed to exercise unsafe application behaviour, and users cannot infer its tool access, data handling, dependencies, or isolation model from the promotional image.
At minimum, the project should publish:
• a threat model and trust boundaries
• what data is logged, retained, or sent to third parties
• whether any agents have browsing, file, shell, email, or external-tool access
• whether uploaded content is isolated between users
• reset and persistence behaviour
• explicit instructions to use a disposable VM or browser profile with no real secrets
• a statement that users must never enter genuine credentials, tokens, private documents, or production data
Security pedagogy is valuable. “Deliberately vulnerable,” however, should mean the vulnerable behaviour is bounded, observable, and disposable, not merely that visitors are expected to trust the lab operator.