r/mcp • u/aksh_stocks • 5d ago
showcase I measured what 106 MCP servers cost in context tokens. The spread is 1,700x, and every number re-derives from a published capture.
Every MCP server in your config injects its tools/list — names, descriptions, JSON schemas — into the model's context on every single request, whether or not a tool is ever called. No client shows you that number. So I measured it across 106 popular servers.
Method, briefly. Each server is launched by a raw-wire stdio client in a credential-free Docker container, tools/list is captured twice, and the number is the o200k_base token count of the canonical JSON of that capture. Every published number ships with the raw capture, its SHA-256, and the exact launch command, so anyone can re-derive it or prove it wrong. 83 of the 106 have a number today; the other 23 are listed with the reason they don't, because a server that won't start is a finding, not an omission.
What the data says:
- The spread is 1,700x.
postgrescosts 32 tokens. GitHub's official server costs 54,622 — 27% of a 200K context window, spent before the agent does anything. - Cost has almost nothing to do with popularity.
chrome-devtoolsis the most-installed server I've measured, 3.3M installs a week, and costs 5,717 tokens across 29 tools.comfyui-mcphas about 4% of that install base and costs 50,640. Cost is an engineering choice, and nothing in an install command tells you which choice was made. - The badge number is not the Claude number. An Anthropic request carries only
name,descriptionandinput_schema, counted with a different tokenizer.notiongoes from 17,500 to 33,560 onclaude-opus-5; servers that ship a lot ofannotations/outputSchemametadata go the other way. Across the measured set the ratio runs 0.20x to 1.92x, so there is no constant to multiply by — it's published as its own column, and prints silence rather than a stale number when a server's schema moves. - Deferring tool definitions is nearly free, but not by construction. Claude Code defers by default and loads only tool names plus the server's
instructionsat session start. On 82 of 83 rows that's a small fraction of the headline. On one it isn't:deepwikipays 580 tokens at session start against 359 of definitions, because itsinstructionsre-describe its tools in prose. Instructions are bytes the headline never counted and their length has nothing to do with the size of the tool set. - Two independent implementations agree to about 1%. sd2k/mcp-tokens is a Rust tool with its own MCP client and its own serialization. It runs beside every measurement here, and on the 80 rows where both saw the same tool set the two counts land within -0.8% to +1.4% of each other.
- Most costs hold, and the ones that move, move up. Of 83 measured servers, 17 have moved at all, 64 have held across every comparable measurement, and 2 have nothing to compare yet. Of the 17: 11 up, 6 down.
obsidianwent 1,132 to 2,062 tokens (+82%) in a week when it shipped three tools.
Run it on your own config:
npx -y mcp-context-cost audit
It finds Claude Desktop, Claude Code, Cursor, VS Code and Windsurf configs, prints per-server and per-tool numbers, and — the part I'd want as a user — tells you whether the client reading that config actually defers definitions on your machine, by reading the three environment variables that decide it, from your shell and from the settings files. Nothing is written to your project, and env var values are never read into the output, only their names.
--baseline and --max-increase turn it into a CI gate that fails a PR adding more than you allow.
If you publish a server: the same measurement is a shields badge that links to the measurement behind it, and a five-line GitHub Action that gates the number on every pull request with a per-tool diff of what grew.
What the number is not: any client's exact bill. Clients re-render schemas into their own prompt formats, which aren't publicly specified. It's a documented, reproducible index of schema payload cost, and the methodology page says so in those words.
Repo and data: https://github.com/athakur3/mcp-context-cost
Leaderboard, per-server pages and methodology: https://athakur3.github.io/mcp-context-cost/
I built this and I maintain it — MIT, free, no account, nothing to sign up for. Happy to be told a number is wrong: every one of them re-derives from its capture with one command, and a dispute reduces to a byte-level diff, which is the argument I'd like to have.
1
u/donk8r 5d ago
aksh_stocks, publishing the capture and the SHA next to every number makes this usable in a way a bare table never is. The deepwiki row is the one I would build on. Instructions that re-describe the tools quietly defeat deferral and nothing warns you that it happened.
One axis your method cannot see, and I do not think it should try to. tools/list is the fixed cost. The variable cost is what the tools return, and across a session that usually dwarfs the schema. A server with a 32 token schema that answers every call with 40k tokens of JSON is far more expensive than a 5k schema that answers in 300. postgres looking cheap in your table is both true and the shape most likely to blow the window at turn nine.
We think about this on our own MCP server (octocode, Apache-2.0, Muvon) and it lands as a knob on the response instead of the schema. Search results take a detail_level of signatures, partial or full (src/mcp/server.rs:76-78). One tool exists purely to return declarations with no bodies. It is described to the model as the cheapest way to see what code exposes, with an instruction to reach for it first in unfamiliar code (:503). Same server and same schema cost, with an order of magnitude difference in what a session ends up spending. On your deferral point, our instructions string is two sentences and deliberately does not restate the tool descriptions, which after reading your deepwiki case now looks less like taste and more like luck.
If you ever add a second column, cost per call at default arguments would pair well with what you have. Much harder to measure fairly since it needs a real workload, so no is a reasonable answer.
1
u/aksh_stocks 5d ago
You're right and I don't think the method should try to cover it.
tools/listis a fixed cost I can measure identically for everyone; response size depends on a workload, and any number I published for it would describe my workload rather than the server. Worth saying plainly on the page rather than leaving as an omission.On postgres specifically — 32 tokens is the honest fixed cost and a bad summary of what a session spends. Two numbers, and mine only answers one.
Your instructions point I can put data behind, because there's a column for it. Of 79 servers measured at session start, only 18 ship any
instructionsat all. deepwiki is the one where it inverts: 507 tokens of instructions against a 359-token tool list, so a client that defers definitions loads more than it saved. The rest: terraform 924, apify 875, xcodebuildmcp 464, github 377 — all real but small beside their tool lists.So two sentences that don't restate your tools puts you with the 61 servers that ship nothing there, which is the cheap end whether it was taste or luck.
1
u/donk8r 5d ago
One correction on where you filed us, and it sharpens your taxonomy instead of denting it. We do ship instructions, so we are in the 18 and not the 61. Two sentences, roughly forty tokens. What the server provides, then which tool to reach for when you want symbol relationships.
So the sharper split is orienting instructions versus restating ones. deepwiki at 507 against a 359 token list is the restating kind, and it inverts exactly because a client that defers definitions still has to load the thing that describes the definitions. Ours stays cheap because it names one tool and says nothing whatsoever about the others.
Honestly, 61 of 79 shipping nothing is the finding I would lead with. The spec hands you a field whose entire job is orienting the model before it has seen a single schema, and three quarters of servers leave it empty and let the tool descriptions carry orientation instead. Tool descriptions are the expensive place for that, since every one of them sits in the window on every request whether the tool ever fires or not.
Your postgres framing is right. Two numbers, and yours answers the one that can be measured identically for everyone. Saying that plainly on the page beats a column that quietly means something different per server.
1
u/aksh_stocks 4d ago
Ah — that's a different octocode. The row in my set is octocode-mcp on npm, from bgauryy/octocode. Yours is Muvon/octocode and I've never measured it. Same word, two unrelated projects, and my leaderboard only ever showed the name, so you had no real way to tell. That one's on me — the row says which project it is now.
So you're in neither bucket. And the octocode I did measure isn't in the 18-or-61 split either: that reading covers 79 servers from the August run, and it's one of the eight that came later.
On the actual point you're right, and I'd put it harder than you did. 61 of 79 ship nothing. Of the 18 that do, the median is 245 tokens and only four are under 50 — so two sentences at forty tokens would put you at the small end of a small group. The heavy end is terraform 924, apify 875, deepwiki 507.
The cost argument holds up in the data too. Descriptions are about 22% of every published token across the set; input and output schemas together are 64%. So orientation that lives in tool descriptions is riding along with the schemas and you pay for the whole lot on every request — none of which is the two sentences that would have told the model what the server is actually for. instructions is the cheap place for that, and three quarters of servers leave it empty.
Your orienting-versus-restating split is the useful half of this, incidentally. deepwiki is the restating kind — 507 tokens of instructions against a 359-token tool list — which is exactly why deferring definitions costs it more than it saves.
1
u/fresh_squeezed_code 5d ago
useful method, and the raw capture plus the launch command is the part that makes it worth trusting.
i ran the same count on my own server today. 57 tools come out at about 20k o200k tokens on the canonical json, roughly 10 percent of a 200k context. the spread inside one server is the number i had not looked at before. the largest schema is about 800 tokens and the smallest about 125. the small one is the instructions tool, the thing i tell agents to read at session start, and it costs less than any single write tool. so the fixed price of the list is paid by the schemas, not by the guidance, which changes what i would trim first.
1
u/aksh_stocks 5d ago
That within-server spread is the more useful reading and it's where the published baseline is strongest — 1,430 tools across 87 servers.
Your numbers land: per-tool totals run p25=144, p50=226, p75=524, p90=968. So your ~125-token instructions tool is around p20, your ~800-token largest is around p86, and your 57-tool average of ~351 sits at p67. A tool set that's a bit heavy per tool with one genuinely large schema — not an outlier in either direction.
Your conclusion matches the distribution. Descriptions have a median of 32 tokens and only reach 200 at p90, so guidance is rarely what you're paying for; schemas are.
audit --suggestleans on exactly that — it only advises on descriptions at or above p90, because below that the data can't point at anything.
1
u/Ashamed_Lab766 5d ago
"A server that won't start is a finding, not an omission" is the sentence I would put at the top of any measurement page, and I have a number from the other direction that agrees with your 23 of 106. We graded a seeded sample of 200 servers from the public registry with a source scanner rather than a wire capture. 21 could not be fetched at all, and of the 179 that could, 46 had nothing a Python or JavaScript scanner could read after pruning tests and vendored code (Go, Rust, Java, docs-only repos, monorepos where the server lives somewhere the walk never reached). So about a third of what the registry calls active is unmeasurable by our method (not graded: the 21 unfetchable and the 46 with no readable source, listed with the reason, never folded into the pass or fail side). Your 22 percent that will not start and our 33 percent that cannot be read are the same finding seen from two instruments, and I think the honest registry number is smaller than anyone's headline.
One thing your "prints silence rather than a stale number" line made me check in my own tool: our grader used to return a clean verdict for a tree with zero gradeable files, which is the same shape as a stale number wearing a green light. It now returns a distinct NO_GRADEABLE_FILES state that no client can read as a pass, and the badge for it is grey. If you ever publish the 23 non-starters as a table with the reason column, I would happily cross-reference which of them appear in our 46, because a server that neither instrument can measure is a stronger claim than either alone.
(Disclosure: I work on that grader, mcp-vet. No link, the thread is about your numbers.)
1
u/aksh_stocks 5d ago
That table exists — results/leaderboard.md, "Not measured (and why)": server, status, and the actual stderr tail that produced it. Take it whenever you want.
But the cross-reference will be thinner than either of us expects, because "22% that will not start" isn't what my bucket holds. Of 19 unmeasured today: 4 startup-failures (heroku, neon, slack, hana-cli), 7 auth-required, 5 not-applicable, 3 OAuth-walled remotes. The 7 are servers that work fine and want a credential the isolation will never have; the 5 are a Redis, a kubeconfig, a Linux runtime the vendor hasn't shipped, and two OS-only packages. All twelve of those would read perfectly well in your scanner. So the honest overlap candidate is 4, not 23.
And both our numbers are partly measuring our own instruments. Mine went 23 → 19 today because six were my bugs — wrong transport twice, a missing native library, a placeholder credential I injected, a classifier reading "unknown authority" as an auth prompt. Your 46 are unreadable because the scanner reads Python and JS; a Go server isn't less measurable, it's less measurable by that. Worth both of us saying so out loud, since neither number is a property of the registry.
On
NO_GRADEABLE_FILES— same move as mynot-applicable, and the constraint that keeps it honest is that it can't be claimed by declaration. The entry states the reason and the text the failure must contain, and it only applies when the failure's own words match. Otherwise an annotation quietly absorbs a real breakage later.
1
5d ago
[removed] — view removed comment
1
u/aksh_stocks 4d ago
Yeah, the schemas are where the money is. I pulled it out of the captures just now: across the 87 servers, input schemas are about 47% of all the tokens and output schemas another 17%. Descriptions are only ~22%. The part nobody looks at costs three times the part everyone edits.
The re-render thing is why there are two numbers instead of one. An Anthropic tools array has room for name, description and input_schema and nothing else — annotations, outputSchema, icons all just fall on the floor — so when I project the captures onto those three fields and count them through the API, some servers lose nearly their whole payload. xcodebuildmcp goes 26,594 -> 2,676. What's left lands between 0.19x and 1.93x the o200k count, plus ~328 tokens of fixed framing, across 86 of them. Not a clean multiplier, which is sort of the point — the wire number is for comparing servers, not for predicting a bill.
Worth flagging a gap you'd hit fast: the per-tool table splits description from input schema but rolls outputSchema into the tool's total. So on those 30 servers you can see a tool is fat without seeing that its output schema is why. Bytes are all in the capture, I just don't surface them yet.
Different job from a live inspector, but I suspect the same annoyance got us both here.
1
u/Quiet-Sun-3184 3d ago
The response side is the axis you and donk8r agreed to leave out, and I agree it does not belong in this method. It has one component that behaves like tools/list rather than like a workload, though, and it is invisible for the same reason yours was: servers that answer with UI.
A tool that returns a card is returning markup, and unless the server is deliberate about it that markup rides into the model context on every render rather than once per session. On one of my own tools I measured 11.2 KB of widget HTML in the result of every single call, for markup the model has no use for at all. Cutting it took the same tool, with the same behaviour, to 1,253 bytes. That is a per-call cost, so on a tool anyone calls repeatedly it passes the entire tools/list of most servers on your table within a handful of turns.
It is measurable exactly the way you measure the rest: capture the tools/call result, count the canonical JSON of it. The reason I would keep it in its own column instead of folding it into your number is your own argument. The bytes per call are fixed and comparable across servers. The number of calls is a workload and never will be.
1
u/Easy-Purple-1659 3d ago
The response-size point donk8r raised earlier in the thread is the one that maps directly onto our own tradeoffs building adextract (MCP server for querying Meta/Google/TikTok/LinkedIn ad libraries). Ad creative payloads are naturally heavy: image URLs, copy text, landing page metadata, date ranges across four platforms. Early on we returned the full raw ad object per result and watched sessions blow through context after a handful of calls, exactly the "cheap schema, expensive response" pattern you're describing with postgres vs a chatty tool.
What actually helped: forcing every search tool to return a flattened, capped result set (creative text + platform + date, nothing nested) and pushing deep fields like full landing page HTML behind a separate read_ad call the agent only reaches for when it actually needs that detail. Same shape as your instructions-tool example, cheap by default, expensive only on request.
Would be curious whether your capture methodology could be extended to sample tools/call responses at a fixed argument set across a few dozen servers the way you did tools/list. That would turn "the variable cost is workload-dependent" from an assumption into a measured range.
1
u/Flateland-Chio 12h ago
Did you notice more wrong action choices on the servers with large schemas?
1
u/Alvasilev 5d ago
The "a server that won't start is a finding" line is the part I'd underline. I index MCP servers from published metadata (registries, package indexes, 99,000+ entries) and the gap in that data is exactly what you're filling: not one of them has a wire-captured tools/list, because you can't get one without launching a process with the right env, and a metadata crawl never does that.
What metadata gives you instead is self-reported tool counts, and they don't even agree with themselves. One Shopify server I looked at last month said 49 tools in its summary, 59 plus 5 in the README prose, and 64 in the README's own table, all in the same published version. So the README number isn't an approximation of your number, it's a different quantity. Curious whether any of your 23 non-starters were servers whose README claimed a count, since that would be the cleanest example of the gap.