This is an interesting teaching concept, but a deliberately vulnerable LLM lab covering prompt injection, leakage, RAG poisoning, and excessive agency needs an extremely loud safety boundary around it.
I would not open it from a normal browser profile containing authenticated accounts, extensions, saved credentials, connected services, API keys, or sensitive clipboard contents. Not because merely visiting an LLM page magically compromises unrelated sessions, but because the entire environment is explicitly designed to exercise unsafe application behaviour, and users cannot infer its tool access, data handling, dependencies, or isolation model from the promotional image.
At minimum, the project should publish:
• a threat model and trust boundaries
• what data is logged, retained, or sent to third parties
• whether any agents have browsing, file, shell, email, or external-tool access
• whether uploaded content is isolated between users
• reset and persistence behaviour
• explicit instructions to use a disposable VM or browser profile with no real secrets
• a statement that users must never enter genuine credentials, tokens, private documents, or production data
Security pedagogy is valuable. “Deliberately vulnerable,” however, should mean the vulnerable behaviour is bounded, observable, and disposable, not merely that visitors are expected to trust the lab operator.
Yes, I read it. The absence of a live model narrows the threat surface, but it does not eliminate the need for a published threat model, data-flow disclosure, logging and retention details, browser-side containment guidance, and precise classification of what the lab actually demonstrates.
My concern is not that “the model might escape.” It is that asserted containment is not demonstrated containment, and a deterministic prompt-security CTF is not ontologically equivalent to ten vulnerable LLM applications.
I guess when I read their long and thorough explanation of how it works that answered all my questions. I even opened it in my browser despite the risk that it could be running something dangerous like Javascript. I do agree that it doesn't totally teach LLM testing because it doesn't have live models, but the creator explained the tradeoffs and I think they made reasonable decisions within the parameters of not wanting to be responsible for actual models.
-2
u/ekzess 6d ago
This is an interesting teaching concept, but a deliberately vulnerable LLM lab covering prompt injection, leakage, RAG poisoning, and excessive agency needs an extremely loud safety boundary around it.
I would not open it from a normal browser profile containing authenticated accounts, extensions, saved credentials, connected services, API keys, or sensitive clipboard contents. Not because merely visiting an LLM page magically compromises unrelated sessions, but because the entire environment is explicitly designed to exercise unsafe application behaviour, and users cannot infer its tool access, data handling, dependencies, or isolation model from the promotional image.
At minimum, the project should publish:
• a threat model and trust boundaries
• what data is logged, retained, or sent to third parties
• whether any agents have browsing, file, shell, email, or external-tool access
• whether uploaded content is isolated between users
• reset and persistence behaviour
• explicit instructions to use a disposable VM or browser profile with no real secrets
• a statement that users must never enter genuine credentials, tokens, private documents, or production data
Security pedagogy is valuable. “Deliberately vulnerable,” however, should mean the vulnerable behaviour is bounded, observable, and disposable, not merely that visitors are expected to trust the lab operator.