ToolStream: Cutting MCP Definition Overhead by 98.5%

In the measured setup, ToolStream reduced the serialized definitions sent to a model from 94,677 bytes for 112 upstream tools to 1,393 bytes for four meta-tools. That is a 98.5 percent reduction in tool-definition bytes per turn while preserving on-demand access to the full tool set.

The measurement is deliberately narrow. It demonstrates schema-byte reduction for one recorded configuration. It does not assume a universal token ratio, model price, latency result, or client-side caching behavior.

AI clients often load every connected tool definition before work begins. A useful collection of GitHub, Jira, Confluence, file, and knowledge tools can therefore consume substantial input context even when a conversation needs only a few of them.

The product question was not how to remove capability. It was how to keep every tool callable while making the default interface much smaller and giving the system evidence about routing quality.

I placed a local proxy between the AI client and upstream MCP servers. The client sees four stable operations: discover servers, discover tools, execute a selected tool, and reconnect an unavailable server. ToolStream owns discovery and routing behind that small interface.

This choice keeps capability on demand instead of preloading every schema. It also creates one point where routing decisions, execution, health, and measurement can be observed without changing each upstream service.

ToolStream is written in TypeScript for Node.js and implements the MCP 2025-06-18 interface through the official SDK. Local ONNX embeddings provide semantic routing without an API call, while SQLite in WAL mode stores tools, sessions, usage events, co-occurrence data, and route traces.

Session context boosts tools already relevant to the conversation. Popularity data improves cold starts. A deterministic strategy selector supports routing experiments, and an oracle compares surfaced tools with tools actually called to calculate rolling Precision at K.

The project measured 72 Atlassian tools, 26 GitHub tools, and 14 Obsidian tools in its SQLite catalog. Their descriptions and input schemas totaled 94,677 bytes. The four ToolStream meta-tool definitions totaled 1,393 bytes using the same serialization basis.

The ratio was calculated directly from the recorded catalog and source definitions. Token and cost examples in project documentation are estimates because tokenization, model pricing, compression, and caching differ by client. This case publishes the byte result only.

The proxy includes per-server timeouts, bounded parallel execution, idle-session reaping, reconnect behavior, health monitoring, and degraded pass-through behavior when semantic routing is unavailable. RouteTrace records explain which tools surfaced, why they surfaced, and how long routing took.

These controls make routing inspectable and failure modes explicit. They are engineering features, not proof of production scale or a security certification.

Fresh local verification on July 31, 2026 passed all 230 automated tests and TypeScript compilation. The latest main-branch GitHub Actions run also passed.

The repository remains private, so this case describes the product and measured evidence without linking source. I claim the verified tool counts, byte counts, ratio, test result, and build result. I do not claim universal token savings, customer adoption, hosted uptime, or a production deployment.