Cutting Agent Tool Tokens by ~80% at 99% Solvability Parity: 5 Lessons from Refactoring Verbose APIs
Why 1:1 API tool dumps quietly burn context budgets, the counter-intuitive list-vs-hydrate call-count tradeoff, deterministic Decimal aggregation, and how brittle eval assertions fake regressions.
- Consolidating sprawling CRUD surfaces into intent-aligned Python @tool_spec overlays (e.g., 60 email endpoints -> 18 tools) and stripping blocked schema fields cut static tool schemas by ~50-82%.
- Replacing fat list responses with summary projections intentionally increased follow-up hydrate calls (+56%) while slashing net response payload tokens by ~78-83% (and up to 88-99% on mutating endpoints).
- Moving pagination + financial math into deterministic Python Decimal tools on Google ADK SkillToolset cut multi-turn prompt tokens by ~32%, while state-based eval assertions proved ~99% solvability parity (0.0pp accuracy loss).
Measured Token & Solvability Impact Across Refactored Agent Tool Suites
| Optimization Layer | Mechanism | Token Impact | Accuracy / Parity |
|---|---|---|---|
| Static Tool Schema (Email) | 60 V1 endpoints -> 18 flat @tool_spec tools | -82% schema tokens | ~99% solvability parity |
| Combined Email + Calendar Suite | Intent-aligned action overlays + flat validators | -66% schema tokens | 0.0pp true accuracy drop |
| Backwards-Compatible V1 Strip | Declarative FD blocked_fields config | -23% to -34% schema | 100% V1 signature preserved |
| Search + Mutator Payloads | 4-field summary projection + thin mutator ack | -78% to -83% net (-99% mutators) | +56% calls, 5x leaner net context |
| Cross-Service Ledger Math | Google ADK SkillToolset + Python Decimal tool | -32% multi-turn tokens | Eliminated float/sum drift |
Verbose V1 API Surfaces vs Ergonomic Toolset Overlays
// Ergonomic @tool_spec Overlay: 18 workflow-aligned tools
// Summary-first search (-83% per-call) + concise mutators (-88%):
search_emails({ query: "is:unread billing", max_results: 10 }) -> [
{
id: "1962a8f",
sender: "billing@vendor.io",
subject: "Invoice #8841 adjustment",
snippet: "Updated credit memo attached..." // 87 tok/call
}
]
get_email({ id: "1962a8f" }) -> { body_text: "...", attachments: [] }
trash_email({ id: "1962a90" }) -> { id: "1962a90", status: "trashed" }Per-pattern numbers above are illustrative clean-room scenarios. Across 75+ production tool simulations the average was ~80% fewer response payload tokens (78-83% range) at ~99% solvability parity.
1. Consolidating 60 Endpoints into 18 Workflow Tools (& FD Schema Stripping)
Auto-exposing an entire legacy API catalog directly to an LLM taxes every single turn of an agent trajectory. Public tool-use benchmarks such as LongFuncEval (arXiv:2505.10570) and Meta-Harness (arXiv:2603.28052) show sharp reasoning degradation once tool catalogs exceed ~20 tools, nest inputs past 2 levels, or flood context windows (where 4x leaner context yields up to +7.7pp higher accuracy).
To optimize multi-service tool catalogs without breaking deterministic execution, we combined two complementary mechanisms depending on backwards-compatibility constraints:
- Action-oriented @tool_spec overlays (StrictBaseModel, extra='forbid'): Consolidating 60 raw email endpoints (26 workflow-relevant functions merged/wrapped + 34 rarely used admin, POP/IMAP, and S/MIME endpoints excluded) into 18 flat, intent-aligned tools cut full static schema tokens by ~82% (and ~66% across combined email + calendar suites).
- Function Definition (FD) schema stripping: Where existing V1 function names and runtime return shapes had to remain 100% untouched, declarative FD configs stripped unused blocked_fields and rewrote verbose parameter descriptions at schema-generation time, saving ~23% to 34% of schema tokens with zero runtime code changes.
- All-or-nothing flat group validators: Instead of deeply nested union objects, flat optional scalar parameters validated via Pydantic v2 @model_validator gave models clear, self-correcting error messages on invalid combinations.
2. The Real List-vs-Hydrate Tradeoff: Fat Lists -> Thin Summaries -> Selective Projection
Conventional wisdom assumes fewer tool calls always means fewer tokens. Production trace telemetry proved the exact opposite on search-heavy workflows.
Our baseline V1 list endpoint was not too thin - it was far too fat, returning full message objects with 12+ fields, nested MIME headers, bodies, and base64 raw payloads on every search (~516 tokens per call), while mutating calls like send or trash echoed entire 6.8 KB resource snapshots. We iterated through two measured stages:
- Stage 1 (Summary-first search + on-demand hydrate): Stripping search results down to 4 lightweight fields (id, sender, subject, snippet at ~87 tokens/call) paired with an on-demand get_message tool intentionally increased total tool calls by ~56% - yet still cut total runtime response tokens across trajectories by ~78% to 83% (and cut mutating trash/send outputs by 88% to 99%).
- Stage 2 (Trace-driven selective projection): Inspecting follow-up hydrate traces revealed that models almost exclusively fetched messages just to read plain-text body and timestamp. Selectively projecting those two clean fields onto search items eliminated 95%+ of follow-up get calls without bringing back MIME or base64 bloat.
from pydantic import BaseModel, ConfigDict, Field, model_validator
class StrictToolModel(BaseModel):
model_config = ConfigDict(extra="forbid")
class SearchMessagesInput(StrictToolModel):
query: str = Field(..., description="Keyword or sender/status filter.")
max_results: int = Field(10, ge=1, le=50, description="Max items returned.")
after_iso: str | None = Field(None, description="Inclusive start ISO-8601 date.")
before_iso: str | None = Field(None, description="Exclusive end ISO-8601 date.")
@model_validator(mode="after")
def check_date_window(self) -> "SearchMessagesInput":
if self.after_iso and self.before_iso and self.after_iso >= self.before_iso:
raise ValueError("after_iso must be earlier than before_iso.")
return self
class MessageSummaryItem(StrictToolModel):
id: str
sender: str
subject: str
received_at: str
snippet: str3. Google ADK Progressive Skill Disclosure & Deterministic Decimal Compute
On cross-service commerce and finance workflows (spanning orders, calendars, email, and SQLite ledgers), dumping paginated raw order JSON into the prompt and asking an LLM to sum net revenue across partial refunds and cancelled line items fails twice: it burns massive cumulative prompt context and introduces subtle floating-point / status-filtering hallucinations.
Using Google Agent Development Kit (ADK SkillToolset) for progressive skill disclosure alongside MCP connectors, we pushed pagination and financial math into deterministic Python Decimal aggregation tools (such as summarizing countable orders and ledger balances server-side). Across multi-turn cross-source scenarios, progressive skill loading + server-side aggregation reduced cumulative prompt tokens by ~32% and eliminated customer-facing math errors.
4. The Evaluation Harness Trap: Why Tool-Name Assertions Fake a -5pp Regression
When we first benchmarked the consolidated ergonomic toolset against the V1 baseline across thousands of multi-hop tasks, automated graders flagged an alarming ~5 percentage point drop in solvability.
Before rolling anything back, we audited the failing trajectories one by one and uncovered a classic evaluation anti-pattern: ~90% of the 'failed' tasks had actually achieved the exact right user outcome in the database, but legacy grader scripts contained brittle assertions checking hardcoded V1 function names in the call log! Replacing call-trace assertions with deterministic final-state assertions proved 0.0pp true accuracy degradation (~99% solvability parity) while cutting schema and payload tokens by ~66% to 82%.