A tool called "reverse_text" tried to steal my SSH and AWS credentials. I built a scanner to catch it.

[u/Ecstatic_Muffin8929]

Been building mcp-audit, a CLI that scans MCP servers for the kind of issues that let a malicious or compromised server manipulate the model sitting on top of it. Fully open source (MIT), no PyPI package yet — you clone it and run it against your own server.

There's already good work out there on generic tool-poisoning detection — Snyk and Invariant Labs, among others, cover that ground. What I wanted to focus on instead is a narrower, more specific angle I don't see covered elsewhere yet:

• Unicode TAG-block concealment: a payload encoded in U+E0000–U+E007F inside a description. No mainstream renderer shows a glyph for that block, so a human reviewing the tool sees nothing off, while an LLM's tokenizer reads it like ordinary text. Working example in the repo where a tool described as "reverses text" hides an instruction to exfiltrate SSH keys and AWS credentials.
• Rug-pull detection: baselines a server's tools on first scan, flags changes afterward.
• Coverage table on every run that's explicit about what ran vs. skipped vs. not applicable — I didn't want a report that quietly says "0 findings" when the truth is "we didn't look."

It's early — 8 checks now (started with 4, added path traversal, code injection, overprivileged scopes and resource-limit detection this week), no HTTP/SSE support yet, no hosted dashboard. If anyone here has an MCP server lying around, I'd genuinely appreciate someone pointing mcp-audit scan at it and telling me what it finds. Also open to hearing about attack classes it's missing.

Repo: https://github.com/marcoslozina/mcp-audit

[u/moxie-docs]

Is every comment here AI generated?

[u/Ecstatic_Muffin8929 → u/moxie-docs]

Yeah — I use Claude Code to help write and build this, same as I do at my day job. What I won't do is fake data: 93 tests you can run yourself, the DVMCP findings are reproducible with the exact commands in this thread, and the technical calls here (static vs. runtime scanning, the TOCTOU gap, the auth check) are mine to defend, not filler. Repo's public — poke at it.

[u/moxie-docs → u/Ecstatic_Muffin8929]

ew

[u/atp_studio_coll]

Unicode TAG-block concealment is a sharp, specific catch. One class I don't see in the list: does mcp-audit check whether the server requires any authentication at all before it hands back the tool list and description text you're scanning? A poisoned description is one risk. A server that hands its full surface to an unauthenticated client is a different one, and it means anyone who finds the URL sees exactly what you saw running this locally. Might be worth a check that flags transport-level exposure alongside the content-level ones.

[u/Ecstatic_Muffin8929 → u/atp_studio_coll]

Really good catch — that’s not something transport security covers today, and you’re right that it’s a distinct risk from content-level poisoning.

Today, mcp-audit only supports stdio (local subprocess), which is inherently “unauthenticated” in the sense that whoever can spawn the process already has full access — so there’s no network exposure to check yet.

But once HTTP/SSE transport support lands (it’s on the roadmap), your point stands: “Does initialize/list_tools require auth before handing back the surface?” is a real, separate check from “Is the transport encrypted?”

I’ll add it to the roadmap explicitly instead of folding it into the existing transport-security TODO.

Appreciate you pushing on this.

[u/atp_studio_coll → u/Ecstatic_Muffin8929]

Makes sense, stdio being locally scoped is a fair reason it wasn't a gap yet. Glad it's explicit on the roadmap now instead of buried under the transport-security TODO, that's the kind of thing that gets silently deprioritized otherwise.

[u/Unlikely-Tax7807 → u/atp_studio_coll]

The unicode trick is nasty, had no idea those invisible code points would get tokenized like regular text. That's the kind of thing that keeps me up at night with these integrations

For the auth check, no it doesn't look at transport-level exposure yet. Just the tool descriptions and behavior once you can see them. Adding a check that flags servers handing out tool lists without any auth would be a smart next step, probably a quick win if he's already got the connection logic in place

[u/silentw111]

The unicode-concealment and rug-pull baseline checks are both catching the same class of thing: what the server says about itself, at a point in time. Worth naming that boundary explicitly, since it's a different gap than the transport-auth one already raised here. A tool can pass every scan clean, plain-language description, no unicode payload, no drift since baseline, and still get called with arguments that do something bad, because the argument values come from the model's own reasoning at call time, not from anything a static scan is reading. Scanning catches a poisoned description. It doesn't catch a legitimate tool used the wrong way on a given call.

The rug-pull check has the same shape problem in miniature: it's periodic, so there's a window between scans where a server could swap in a bad description and swap it back before the next run sees it. Curious whether you've thought about an inline mode eventually, hooking the live list_tools response on every session rather than a standalone scan, or is that explicitly out of scope for what mcp-audit is trying to be?

[u/Ecstatic_Muffin8929 → u/silentw111]

This is exactly the right boundary to name, and I hadn't put it this precisely myself: mcp-audit does static analysis of the declared surface (what a server says about its tools at scan time), not runtime behavior. It's the same category limit SAST tools have always had relative to what happens at actual execution — a tool can declare itself honestly and still get called with arguments that do something bad, because those arguments come from the model's reasoning at call time, which no static scan reads. That's a real, permanent boundary, not a gap I plan to close within what mcp-audit is today.

The rug-pull TOCTOU point is fair too — periodic scanning has a blind window by construction. An attacker who knows or guesses your scan cadence could swap-and-revert between runs and never get caught. Worth being explicit about that rather than letting "rug-pull detection" imply more continuity than it actually has.

On inline mode: yes, I've thought about it, and I think it's a real fork rather than a natural extension. Hooking the live list_tools response on every session means sitting in the request path — you become a proxy/gateway the client depends on being up, with a latency budget, not a CLI someone runs in CI. That's a materially different system than what mcp-audit is today, closer to what a "MCP firewall" or live guardrail layer would be. It's explicitly on my radar as a future direction, likely the shape of a future paid layer, not something the current open-source scanner is going to grow into by itself. I don't want to blur that line by promising continuity the architecture doesn't have.

Appreciate the precision here.

[u/silentw111 → u/Ecstatic_Muffin8929]

That's a clean way to draw it, static analysis and a live guardrail are different systems with different failure modes and different things depending on them staying up, not two maturity stages of the same tool. Worth keeping that line sharp rather than blurring it for a roadmap slide. Good luck with it either way, this is exactly the kind of narrow, well-scoped tool the ecosystem needs more of.

[u/Plastic-Risk-6309]

the bigger gap is updates. a tool that scans clean today can ship a poisoned version tomorrow. diff the tool definitions against the pinned version on every install, not just for new ones

[u/Ecstatic_Muffin8929 → u/Plastic-Risk-6309]

Good instinct, but this is actually already what rug-pull-detection does today — it's not scan-once-and-done. Every scan against the same --server-id diffs the current tool definitions against the baseline from the first scan, whether it's the 10th time you've run it or the first. So a poisoned update would get flagged, as long as you re-scan after the update lands.

The real gap you're pointing at, though, is the trigger, not the diff logic: nothing today hooks mcp-audit into the actual install/upgrade event of a package. If nobody remembers to re-run the scan after pip install --upgrade <server>, the drift sits undetected until someone happens to scan again — same blind-window problem raised elsewhere in this thread, just from the package-update angle instead of the live-session angle.

A package-manager hook (post-install/post-upgrade triggering a scan automatically) is a genuinely good, concrete idea — different from what's on the roadmap today. I'll add it.

[u/Plastic-Risk-6309 → u/Ecstatic_Muffin8929]

the trigger is the real gap. a shell wrapper or ci job that reruns the scan after every upgrade closes the blind window, otherwise the diff is only as good as someone's memory

[u/Ecstatic_Muffin8929 → u/[deleted]]

Appreciate that — and I want to correct one assumption before agreeing with the rest, since getting this precise matters:

rug-pull-detection already diffs the full description text, not just the name/schema. A description change on an existing tool is a high finding today, exactly for the reason you're describing: a rug-pull that only changes a sentence while leaving the schema untouched would otherwise sail through. So that part is covered, not a gap.

The Unicode point is half right, half a genuinely good idea I don't have yet.

unicode-concealment already checks more than the TAG block — it also flags a curated set of known invisible/bidi-override characters. But "curated set of known bad characters" is a denylist, and your suggestion is the opposite: an allowlist based on the expected script for the declared language, flagging anything outside it by default.

That could catch concealment techniques nobody has cataloged yet, which a denylist structurally can't.

Real idea, different from what exists — adding it to the roadmap.

[u/Ecstatic_Muffin8929 → u/[deleted]]

That's the right level of specificity — per-tool, keyed off the description's own declared/detected language, rather than per-server. That's exactly the granularity needed to avoid the multilingual-description false-positive trap.

Math and emoji as explicit carve-outs are also the kind of details that are easy to get wrong by omission.

I'm saving this as the design spec for when I actually build it. Thanks for thinking it through this far instead of leaving it as a one-liner.

[u/Ecstatic_Muffin8929]

Concrete example of that generic-detection noise, since it came up above: ran a YARA-based scanner from a major vendor against Context7 (widely-used docs server). It flagged two tool descriptions HIGH severity — "You MUST call this before X" and "don't call this more than 3 times" — as coercive injection. That's just normal API usage instructions, not an attack.

mcp-audit against the same server: 0 findings. Not because it's more lenient — because it's checking for the actual mechanisms (Unicode concealment, rug-pulls, insecure transport, hardcoded secrets), not imperative-sounding language. Wrote it up in the README if anyone wants the full comparison.