The Efficiency Trap. How Do You Estimate and Control Token Costs?

Abstract

Per token, AI has never been cheaper. Per outcome, most organisations are paying more, because the bill is now written at runtime by agents that decide for themselves what to read, retry and delegate. This article explains why that bill became non-deterministic and why most of it goes on I/O rather than reasoning. It then offers six questions to ask in order, a six-layer toolkit with a primary and a secondary tool for each layer, twenty-one initiatives scored by impact against effort, a reference architecture in which the gateway is the only mandatory layer, and a ninety-day plan to put a number on the bill before the invoice arrives. The thesis fits in one line: you cannot budget a behaviour, you can only bound it.

This piece serves as a follow-up to “The Discipline Gap. Is Spec-Driven Development the Answer?” [1]. While that article explored how software generation was automated without the underlying engineering judgement, this article addresses how the same shift disrupted financial planning.

The question I did not take seriously

Nearly three years ago a client asked me how he would control what AI was going to cost him. Frontier models felt almost free then, a $10 or $20 seat with no meter attached, and the honest answer felt like « it barely costs anything yet ». He asked anyway. He had lived through the same film with cloud a decade earlier and recognised the opening scene.

He was early. Today two questions arrive in almost every client conversation before anything else is discussed. How do I estimate what this will cost? And how do I stop it running away? They used to come from procurement, late, after a decision. Now they come from CTOs and CFOs, early, as a condition of starting.

For me personally, there is an ironic twist here. More than a decade ago I taught software estimation and argued, building on Steve McConnell « whose estimation course I had the privilege of attending in person », that the shotgun beats the rifle: a spread of imperfect estimates you can defend beats one precise-looking number you cannot [2]. I did not expect that old discipline to be the most useful thing I brought to AI. It is, because the answer has not changed:

When you cannot produce one honest number, you produce a defensible range and you bound the tail.

Why doesn't a cheaper model mean a smaller bill?

The reassurance I hear most often is « the models keep getting cheaper, so this solves itself ». It is a reasonable belief. It is also, historically, the most expensive assumption in the room.

In 1865 William Stanley Jevons published The Coal Question. Watt's engine used a fraction of the coal Newcomen's had needed for the same work, and every sensible person concluded that national consumption would fall. Jevons read the ledgers. Consumption had risen. Sharply. “The very contrary is the truth,” he wrote [3]. When the engine got cheaper to run, the engine went everywhere.

We are those factory owners now. Stanford's AI Index measured the price of GPT-3.5-level performance falling from $20 to $0.07 per million tokens between November 2022 and October 2024, more than 280-fold [4]. Andreessen Horowitz put the broader trend bluntly: “The cost of LLM inference has dropped by a factor of 1,000 in 3 years” [5]. Notice what is being measured. A fixed level of capability got cheaper. The flagship model your team actually uses did not. Meanwhile consumption grew faster than prices fell. Across more than 100 trillion tokens on OpenRouter, the average prompt grew roughly fourfold, from about 1,500 to over 6,000 tokens, and programming requests routinely exceed 20,000 input tokens [6]. Uber exhausted its entire 2026 AI coding budget by April, across roughly 5,000 engineers; its CTO, Praveen Neppalli Naga, said the company was “back to the drawing board”, and it now caps each engineer at $1,500 per tool per month [7].

Gartner projects that by 2028 AI coding costs will surpass the average developer's salary [8]. The cheapest thing in the stack is on its way to overtaking the most expensive person on the team.

Per token, everything got cheaper. Per outcome, most organisations are paying more.

The non-deterministic bill

Software cost has passed through three eras, each defined by a different scarce resource. In the datacentre, cost was a purchase and we optimised utilisation. In the cloud, it became a consequence of architecture and we optimised elasticity, which is why FinOps was born. In the agentic era it is a consequence of behaviour, and what we need to optimise is predictability.

FinOps works because the same infrastructure produces the same cost. Agents break that assumption on every run. In the agentic era the bill is written at runtime, by a system deciding for itself how often to think, what to re-read, which tools to call and whether to try again. Two engineers, same repository, same task, can differ by an order of magnitude in spend. One pastes the failing test and a sentence of intent into the chat. The other types « the build is broken, fix it », and the agent goes looking, which is a polite way of saying it reads the repository at frontier rates until something matches. The discipline gap and the cost gap are the same gap, measured with a different instrument.

The closest analogy is a taxi meter. A fixed fare can be approved in advance. With a meter, the only real controls are the route, a ceiling agreed before setting off, and watching the display. Most organisations are still negotiating the fare on a journey that is already metered. I call this the non-deterministic bill.

Where does the money actually go?

An agent loop spends tokens on four things that have nothing to do with the quality of the model: re-reading context it already read, carrying raw tool output through the window, retrying work it got wrong, and fanning out to sub-agents that hand back their entire transcript.

It opens twelve files to locate one method. It writes a twenty-first test file that mirrors the previous twenty. All of it is billed at frontier rates, and most of it twice: once as expensive output, then again as input when it comes back to be reviewed. Herbert Simon warned in 1971 that “a wealth of information creates a poverty of attention” [9]. Though he was describing human executives, his observation applies just as directly to context windows.

A coding agent spends most of its time on I/O. Very little of it goes on thinking.

That changes what efficiency means. Compressing a 20,000-token command result into 5,000 tokens is useful. Not running the command is better. Reading five relevant symbols instead of forty files is better still. Token efficiency is not primarily a compression problem. It is a work-selection problem, and the order of the questions matters more than any single percentage:

1. Avoid. Does this operation need to happen at all?
2. Retrieve. Can I fetch only what is required?
3. Compress. Can I shrink it before the frontier model sees it?
4. Execute. Can a cheaper tier do the mechanical work?
5. Verify. Can I check only what changed?
6. Measure. Did cost per successful outcome actually improve?

The toolkit, layer by layer

The image below turns those questions into six layers a team can actually install, from Avoid work to Route & attribute. Each layer names a primary tool and a secondary alternative. « These are the ones I have seen in practice, which is a different thing from a ranking ». Retrieval maps to the second layer, compression and reuse to the third and fourth, cheaper execution and measurement to the sixth.

1. Avoid work. Primary tool: Ponytail. Secondary: SDD rules for YAGNI (You Ain't Gonna Need It) and KISS (Keep It Simple S*@#). Ponytail is a skill that makes the agent stop at the first level that holds: the library that already exists, the line that never needed writing. Its author, Dietrich Gebert, reports 80% to 94% less code on Claude models, and he also published the run where the gain disappeared on a small local model [10]. The secondary option needs no install: a specification with explicit non-goals, read by an agent told to keep it simple and build only what was asked.

2. Know the system. Primary tool: Graphify. Secondary: CodeGraph. Graphify maps code, documentation and specifications into one queryable graph, so the agent traverses relationships instead of grepping files. On Claude Code it installs a hook that redirects search calls to the graph [11]. CodeGraph is the lighter choice when the problem is code alone: fast, focused graphs of symbols, callers and dependencies. While both rely on deterministic Tree-sitter AST parsing to map code structures, saving substantial tokens during repository navigation, Graphify extends this by leveraging LLMs to produce rich contextual documentation.

3. Optimize context. Primary tool: Headroom. Secondary: RTK (Rust Tokens Killer). Headroom is a local proxy that compresses tool results, retrieved documents and conversation history, and keeps the originals locally so the model can retrieve them on demand; its authors claim 60% to 95% fewer tokens [13]. RTK works one step earlier and more narrowly, filtering test, build and git output before it reaches the context; its authors claim 60% to 90% on common commands [12].

4. Reuse context. Primary tool: prompt caching. Secondary: context compaction. At Anthropic, cached input is billed at a tenth of the standard rate, and other providers offer similar discounts [14]. One documented team moved from a 7% to an 84% cache hit rate and cut its spend by 59% [15]. The gap between those numbers was not technology. It was whether the stable part of the prompt was structured as a default or left to habit.

On Claude, almost every block of a request can be cached: tool definitions, the system prompt, messages, images, documents and tool results [14]. The cache covers the prefix in a fixed order: tools, then system, then messages. So put what never changes at the top and what changes every turn at the bottom, because editing anything above a cache marker invalidates everything after it. Prompts below a model-specific minimum, between 1,024 and 4,096 tokens, are not cached, and no error says so. Entries last five minutes by default, or an hour at a higher write cost. « The usage fields in each response are the only proof the cache was hit ». The rule we all learned, do not re-send context, turns out to be shorthand for do not re-send context to the expensive model.

Compaction covers what caching cannot: it replaces a long session history with a smaller summary before every new turn pays for it again.

5. Say less. Primary: Caveman. Secondary: Concise output format. Both strip preambles and wrapper prose before they become someone else’s input. Caveman installs as a skill or plugin in Claude Code, Codex, GitHub Copilot and most other agents; its author reports about 65% fewer output tokens on average, it does not touch input or reasoning tokens, and it adds around a thousand input tokens per turn [16]. Concise is Claude Code’s built-in output style: it leads with the result but keeps error reports, security warnings and destructive-action confirmations complete. Outside Claude Code, the same effect is three lines in AGENTS.md or Copilot’s instruction file (e.g. {"outputStyle": "Concise"}). An independent test of Caveman found most tokens in real coding sessions go to thinking and code generation, which neither touches.

6. Route & attribute. Primary tool: LiteLLM. Secondary: Portkey. Consolidate all model calls under a single endpoint, assign team-level budgets, tag every request, and switch to a more cost-effective model via simple configuration changes. LiteLLM handles routing, usage tracking, and cost management, while Portkey provides a managed gateway equipped with guardrails.

A newer pattern is worth watching: decision models such as TypeSafe’s Jev, which return a typed choice instead of text, can route a request or gate a tool call for a fraction of an LLM’s price. Jev Router picks the model and reasoning effort per request (used by OpenRouter). Laya and OpenJev offer local, open-weight versions [23]. « They are weeks old as I write, so treat them as a hypothesis to test, not a default ».

Underneath all six sits a spec-driven framework, OpenSpec, BMAD or GitHub Spec Kit, the discipline the previous article argued for [1]. Here it earns its place for a second reason.

A specification is compressed future context.

Defining intent, constraints, and acceptance criteria upfront turns a specification into compressed context. Reusing these decisions for implementation and verification avoids repeatedly rediscovering information from raw source files. It is also what the fifth question needs, something explicit to verify against.

And none of the six layers is tied to a harness: the same tools work with Claude Code, Codex or GitHub Copilot, on any model, local or cloud.

The layers add up; they do not compete. Each acts on a different source of tokens, so comparing their headline percentages is meaningless. And every figure above comes from the tool's own authors, so treat each one as a hypothesis until it has run on your own repository.

A prompt is a request, not a wall

Most of those tools are instructions. Dimitri Mazmanov at Spotify published the cleanest demonstration I know of why that matters [17]. His first version relied on instructions telling Claude when to hand bulk reading to a cheaper worker model. In his words, it “sort of worked”: the agent followed the instructions when it noticed them. The version he shipped adds a pre-tool-use hook that checks every file read before it happens and sends large ones to the worker, which returns a short answer. If the agent ignores the instructions, “the hook still blocks the expensive read”. The expensive model gets the answer and never sees the corpus, and the Claude-side tokens fell by around 90% in his read-heavy tests. Identical intent, expressed across two distinct layers, yet only one proved resilient. He is also candid about the limits: the worker missed a subtle thread-safety bug, and each delegation costs a round trip of ten to thirty seconds.

A saving you have to remember is not a saving. Only enforced defaults compound.

Money obeys the same rule. A spending limit written into a system prompt is a suggestion the model may ignore under sufficiently interesting conditions. The same limit at the gateway is a wall. Many organisations I have worked with got burned by a runaway agent with the limit written down somewhere, in the wrong layer. Gartner reached the same conclusion from the other side of the table: “Token discipline will not emerge through developer choice alone” [8]. « I now put this sentence on the first slide of any agentic AI business case, before the savings estimate, not after it. »

Two bills, two owners

The primary reason most cost-control initiatives fail is organizational: they attempt to manage AI expenditures as a single metric under a single leader. In reality, AI spend consists of two distinct bills with separate owners.

Engineering owns the first bill. Product and engineering own the second, jointly. The ceiling is the difference that matters. Build spend has a circuit breaker made of a human being who eventually goes home « unless, as may soon become routine, a fleet of autonomous agents runs continuous operations 24/7, waking her overnight whenever sign-off is required ». Run spend has nothing of the kind. An in-product agent runs at three in the morning, on a holiday, in a loop, for a customer who has already left the chat. The unit economics look excellent on paper. Gartner benchmarks an agent-assisted contact at around $13.50 against $1.84 for self-service, yet its own survey found that only 14% of customer service issues are fully resolved in self-service [18]. The saving is real only if “resolved” does not drift into “deflected”.

Twenty-one initiatives, one order

Although the toolkit offers the underlying mechanisms, establishing the operational sequence remains a CTO's responsibility. To structure this, we track twenty-one initiatives (thirteen tailored for build environments and eight for production runs) ranked by balancing implementation complexity against business value. Sharing these strategic assessments invites valuable dialogue, as a client questioning a specific score often highlights critical operational constraints. Appendix A outlines every initiative alongside its immediate first step and verifying metric.

Quick Wins via Configuration: The highest-impact initiatives require minimal implementation effort. Implementing prompt caching (S01) only entails reordering prompts and enabling a flag, while bounded loops (P1) simply sets an iteration cap. Both can be deployed immediately.

Tactical vs. Strategic Focus: Financial return resides in the left half of the matrix, whereas long-term strategy occupies the right half. A frequent failure mode is prioritizing strategic bets prematurely, leaving teams with architectural plans but no impact on their invoice at quarter-end.

The Missing Low-Impact Column: The bottom half is nearly empty, a signal for caution rather than comfort. Low-impact ideas were excluded during shortlisting.

Where each piece lives

While a ranked list outlines immediate priorities, the architect inside us will inevitably inquire about where each component belongs within the system.

Build and run mirror each other in the top four layers and share the bottom two. In our experience, constructing duplicate gateways and caching backbones is among the most frequent and costly redundancies organizations commit. Layer five is the only mandatory one. It is the one place a budget is enforced rather than requested, and the one place every other saving becomes measurable. Skip GraphRAG, self-hosting and prompt optimisation and you still have a defensible cost position. Skip the gateway and nothing else on the diagram can be proven.

Starting from Scratch: The 3 Waves Plan

To defend a strategy before a board, sequence matters as much as timing. This roadmap unfolds across three structured waves, with each phase requiring concrete evidence of completion before transitioning to the next. Refer to the diagram below for a clear mapping connecting each wave to its corresponding initiative codes.

Wave 1: make the bill legible and bounded. One gateway in front of every call, with per-team budgets. Prompt caching wherever a system prompt or repository context repeats. Cycle caps on every production agent, tested until you have seen them fire. A work-avoidance hook in the coding harness. And the denominators, cost per merged change and cost per resolved outcome, agreed in writing. Nothing here needs a new vendor or a model decision.

Wave 2: make the cheap path the default. Route the routine 60% to 80% of coding traffic to a cheaper tier. Delegate bulk reading to cheap workers, and write the exclusion list: editing and genuine reasoning stay with the frontier model. Measure tokens by source and apply the matching toolkit layer, from Know the system to Reuse context. Batch the non-interactive work, stack provider, exact and semantic caches on the product side, and put guardrails and tracing around every production agent before launch rather than after the first incident.

Wave 3: change the behaviour. Pilot spec-driven development on one team and measure rework before and after. Pilot programmatic tool calls on the most tool-heavy workflow. On the product side, introduce small-model routing and evaluation gates that block a change which doubles tokens per task without anyone noticing.

Everything else, prompt optimisation, full GraphRAG, repository splits and self-hosted hardware, waits for a threshold: spend above $500 per developer per month, a cache hit rate below 20%, fifty or more developers with steady usage.

And four rules govern all three waves:

• attribution before optimisation,
• one team before any mandate,
• a before-and-after on your own workload for every change,
• and a named owner who gets paged.

Uber even ranked its engineers on an internal usage leaderboard [7]. It had visibility and still lost a year's budget by April, because nothing was capped and nobody owned the alarm.

Measure the tail, not the mean

Three measurements matter more than any headline number.

1. Cost per successful unit of work. Evaluating cost per token merely measures unit pricing, while tracking cost per call incentivizes unnecessary verbosity. Real control comes from tracking cost per merged change in build environments and cost per resolved outcome in production, backed by defensible denominators. For benchmarking, Uber observed average spending between $150 and $250 per engineer monthly, with power users nearing $2,000 [7], whereas Gartner Peer Insights indicates that 23% of engineering leadership sees spending between $200 and $500 per developer [8]. Use these figures as baseline sanity checks and validate them against current usage.

2. Tokens by source. A high or increasing tokens-per-task metric signals an issue, but fails to identify the root cause. By categorizing and breaking down usage: separating repository reads, static context, tool outputs, sub-agent returns, retrieved payloads, retries, and conversation history. Each distinct line item illuminates a specific layer of your toolkit.

3. The tail. A lesser-known variant of the Robin Hood tale « as I imagined it » tells of the willing participant balancing an apple on his head who is struck by the arrow and perishes. His death was not caused because Robin was a poor shot. He died because Robin was, on average, an excellent one. « Averages bring comfort to everyone except the man standing under the tail of the distribution ». What breaks the budget isn't the typical case, but the p99 session, the runaway execution loop or the sudden afternoon fan-out. Averages offer false security; the long tail is where true budget limits belong.

An average is not a budget. The tail is the budget.

Where this argument is weakest

This may seem premature, and on some engagements may be the case. Indeed, a three-day proof of concept hardly justifies a full FinOps framework. Beyond that, four key considerations help put this argument in perspective.

• Savings are not productivity. METR's randomised trial found experienced developers 19% slower with AI assistance while believing they had been 20% faster [19]. The sample was small and the result is contested, but the direction of the error matters. A cost programme that promises a productivity number it never measured is building its own credibility problem.
• Vendor numbers are best cases. Anthropic's famous 98.7% token reduction for code execution over MCP came from one favourable workflow, roughly 150,000 tokens down to 2,000 [20]. Cloudflare's own Code Mode demo, reported by WorkOS, measured 32% on a simple task and 81% on a complex batch [21]. Same idea, different workloads, a threefold spread. The same discount applies to every tool in the toolkit. Quote the range, never the headline.
• Every lever has a floor. Delegating a small file read to a cheap worker costs more in round trips than it saves. An optimisation applied indiscriminately stops being an optimisation, and a 60% cut on 3% of spend makes a better slide than a saving.
• The strongest counterargument. If the price of a given capability falls tenfold a year, patience is the cheapest optimisation. For raw price exposure, that is largely true, although Gartner's own analyst expects infrastructure and profitability pressure to “push model pricing higher” [8]. It stops being true the moment spend is driven by loops per task, because loop counts do not fall on a vendor's roadmap. Gartner expects over 40% of agentic AI projects to be cancelled by the end of 2027, citing “escalating costs, unclear business value or inadequate risk controls” [22]. Those failures will come from controls, not from prices.

We run it on ourselves first

ParserAI is our internal agentic platform: provider-agnostic, running across several frontier models, where our people build skills, agents and scheduled sub-agents that reach internal tooling through MCP across talent, staffing, sales, presales and account management. One front door across many providers turned out to be the one place to see spending by team, set budgets, cache aggressively and move a workload to a cheaper model without asking anyone to change how they work. The earlier waves were, more or less, the order in which we built it. « I hesitate to advocate for a practice whose price I have not personally absorbed. When an uncomfortable course of action is suggested here, it is typically because it was tested against our own budget first. »

The invoice moves up a layer

The history of software cost is the bill moving one layer up. From machines to allocations. From allocations to architectures. From architectures to behaviours. Each step made the previous era's controls look outdated, and each arrived when capability outran our ability to govern it.

Jevons was not arguing against efficiency. He admired the Watt engine. He was arguing against a comfortable assumption: that a cheaper unit produces a smaller bill. It never has. What produces a smaller bill is deciding, deliberately and in infrastructure, how much of the newly cheap thing you are willing to consume. Six verbs, in that order: avoid, retrieve, compress, execute, verify, measure. The cheapest of them is the first.

Organisations will not compete on the price of their tokens. They will compete on how few they need to spend in order to be right.

The non-deterministic bill will not become deterministic. Models will keep getting cheaper and agents will keep finding new work to do with them, which is exactly what we want.

The discipline is not there to make the machine think less. It serves a single purpose: quantifying expenditure prior to receipt of the invoice.

I write to understand, to summarise, and not to forget. The teacher in me never quite left « I still carry a professor's title from the Universidad Nacional de Córdoba », although in pieces like this one the main student is me. I am an electronic engineer by training, and I approach AI the way engineers approach every shiny new object: fascinated, curious, and suspicious of anything I cannot measure. As CTO at Parser, I get to test that curiosity against reality every week, alongside our clients and friends.

Appendix A. The twenty-one initiatives

Priority scores serve strictly as a sequencing guide rather than an absolute benchmark. Each entry lists where the saving comes from, the first step that fits in one sprint, and the number that proves it worked.

Build

• S01 · Prompt and context caching. Saving: cached input at about 10% of the standard rate. First step: put system prompt, standards and repo map in one stable prefix. Proof: cache hit rate on the invoice.
• S02 · LLM gateway and routing. Saving: one endpoint for budgets, attribution, model swaps by configuration. First step: point every IDE, CLI and CI bot at it; per-team keys. Proof: spend by team and tool, last 30 days.
• S03 · Harness-level work avoidance. Saving: expensive reads blocked or redirected before they happen. First step: one pre-tool-use hook with a size threshold. Proof: files read per task.
• S04 · Open-weight models on cheap inference. Saving: routine work at a fraction of frontier price. First step: route test generation and mechanical refactors to one cheap tier. Proof: share of traffic on that tier; its rework rate.
• S05 · Context and tool-output compression. Saving: every admitted token is billed again on later turns. First step: CLI output filter in one repository; compaction on long sessions. Proof: tool-output tokens per task.
• S06 · Cross-model delegation. Saving: I/O-bound work done by a cheap worker that returns a summary. First step: one bulk-reader worker plus a written exclusion list. Proof: frontier input tokens per task.
• S07 · Spec-driven development. Saving: no regeneration cycles; decisions reused, not rediscovered. First step: one team, one framework, specs with acceptance criteria. Proof: rework rate before and after.
• S08 · MCP code execution. Saving: intermediate payloads stay out of the model. First step: rewrite the most tool-heavy workflow as one script. Proof: tokens per run, conservative range.
• S09 · Batch APIs and FinOps governance. Saving: about 50% off non-interactive work. First step: move nightly docs, embeddings and test generation to batch. Proof: share of tokens at batch price.
• S10 · Reflective prompt optimisation. Saving: a small model made reliable for one repeated task. First step: evolve one prompt against a few hundred labelled runs. Proof: small-model accuracy on a held-out set.
• S11 · Repository knowledge graphs. Saving: traversal instead of broad file reads. First step: index one service; symbol lookups before file reads. Proof: files read per task; answer quality.
• S12 · Modular architecture and repo scoping. Saving: a UI task stops loading the backend. First step: per-module agent instructions for two modules. Proof: context size at session start.
• S13 · Local or self-hosted inference. Saving: zero marginal token cost after capital spend. First step: model the payback with finance first. Proof: cost per token at real utilisation.

Run

• P1 · Bounded loops and cycle limits. Saving: an unbounded worst case becomes a known ceiling. First step: cycle cap plus per-conversation ceiling, tested. Proof: p99 cost per conversation.
• P2 · Guardrails and policy. Saving: closes a cost hole and a compliance hole in one layer. First step: allow-list actions; authorise irreversible steps. Proof: blocked actions per thousand conversations.
• P3 · Outcome-based unit economics. Saving: makes “is it worth it” answerable. First step: agree in writing what counts as resolved. Proof: cost per resolved outcome, weekly.
• P4 · Observability and decision tracing. Saving: regressions caught in staging, not on the invoice. First step: trace every production run with cost per step. Proof: minutes to locate a spend spike.
• P5 · Cost attribution and chargeback. Saving: per-feature and per-customer budgets. First step: tag every call with feature, team and customer. Proof: share of spend attributed.
• P6 · Small-model routing and cascades. Saving: small model by default, escalate on doubt. First step: route the lowest-risk intent first. Proof: escalation rate; CSAT on routed traffic.
• P7 · Automated evals and deployment gates. Saving: blocks the change that doubles tokens per task. First step: tokens-per-task ceiling next to the quality floor in CI. Proof: releases blocked for cost.
• P8 · Runtime tiering and semantic caching. Saving: repeat questions never reach a model. First step: stack provider, exact and semantic caches. Proof: share of traffic served from cache.

Appendix B. Where to find the tools

This compilation is neither definitive nor an endorsement, and it will inevitably date more quickly than the main text. Treat every choice as reversible: pin versions, pin the source, and keep the gateway abstraction so that replacing any row costs a configuration change rather than a migration. Two examples from writing this appendix: installing RTK from the default Rust registry fetches an unrelated package with the same name, and GSD moved its maintained home to a new organisation in 2026 after ownership concerns.

• 1. Avoid work: Ponytail · SDD (YAGNI/KISS) · neighbours: Claude Code hooks; Cursor and Windsurf rules; CI gates. github.com/DietrichGebert/ponytail
• 2. Know the system: Graphify · CodeGraph · neighbours: Serena; Aider repo map. Graphify, CodeGraph, Serena, Aider
• 3. Optimize context: Headroom · RTK · neighbours: LLMLingua; sqz. Headroom, RTK, LLMLingua, sqz
• 4. Reuse context: Prompt caching · Context compaction · neighbours: GPTCache; RedisVL. Provider features inside Claude Code, Codex, GitHub Copilot; GPTCache, RedisVL
• 5. Say less: Caveman · Concise (Claude Code output style). Caveman, Claude Code output styles; or inside AGENTS.md / .github/copilot-instructions.md
• 6. Route & attribute: LiteLLM · Portkey · neighbours: Bifrost; OpenRouter; Kong or Cloudflare AI Gateway; Jev Router (TypeSafe) inside OpenRouter. LiteLLM, Portkey, Bifrost, TypeSafe on OpenRouter
• SDD frameworks: OpenSpec · BMAD · Spec Kit · neighbours: Kiro; GSD. OpenSpec, BMAD, Spec Kit, GSD
• Coding agents: Claude Code · Codex · GitHub Copilot. Every layer above is harness-agnostic.
• Observability and evals: Langfuse; LangSmith; Arize Phoenix; Helicone; OpenLLMetry; DeepEval; Braintrust. Langfuse, Phoenix, Helicone, OpenLLMetry, DeepEval, Braintrust
• Guardrails: NeMo Guardrails; Guardrails AI; LLM Guard. NeMo Guardrails, Guardrails AI, LLM Guard
• Serving and optimisation: vLLM; SGLang; Ollama; DSPy with GEPA. vLLM, SGLang, Ollama, DSPy

References

[1] Martin Miceli, Parser. “The Discipline Gap. Is Spec-Driven Development the Answer?” July 2026. parserdigital.com/insights/the-discipline-gap-is-spec-driven-development-the-answer

[2] Steve McConnell. Software Estimation: Demystifying the Black Art. Microsoft Press, 2006. microsoftpressstore.com. See also Martin Miceli, “AI-Powered Software Estimation”, April 2026. linkedin.com

[3] William Stanley Jevons. The Coal Question. Macmillan, 1865, Chapter VII. oll.libertyfund.org/titles/jevons-the-coal-question

[4] Stanford HAI. AI Index Report 2025: inference cost for GPT-3.5-level performance, November 2022 to October 2024. hai.stanford.edu

[5] Guido Appenzeller, Andreessen Horowitz. “Welcome to LLMflation: LLM inference cost is going down fast.” November 12, 2024. a16z.com/llmflation-llm-inference-cost

[6] OpenRouter and Andreessen Horowitz. “State of AI: An Empirical 100 Trillion Token Study with OpenRouter.” arXiv:2601.10088, January 2026. arxiv.org/abs/2601.10088

[7] The Information, April 2026, interview with Uber CTO Praveen Neppalli Naga. theinformation.com; Bloomberg, June 2026, on Uber's $1,500 per-tool monthly cap.

[8] Gartner. “Gartner Predicts AI Coding Costs Will Surpass Average Developer's Salary by 2028 as Token Consumption Surges.” Press release, June 24, 2026, quoting Nitish Tyagi. gartner.com

[9] Herbert A. Simon. “Designing Organizations for an Information-Rich World.” In Computers, Communication, and the Public Interest, Johns Hopkins Press, 1971. veryinteractive.net

[10] Dietrich Gebert. ponytail. GitHub, 2026 (project-reported benchmark, including a negative result on a 3B model). github.com/DietrichGebert/ponytail

[11] Graphify Labs. Graphify. GitHub, 2026. github.com/Graphify-Labs/graphify

[12] RTK, Rust Token Killer. GitHub, 2026 (project-reported savings). github.com/rtk-ai/rtk

[13] Headroom Labs. headroom: the context compression layer for AI agents. GitHub, 2026 (project-reported savings). github.com/headroomlabs-ai/headroom

[14] Anthropic. Claude API pricing for prompt caching: cache reads at 10% of base input; Batch API at 50%. platform.claude.com/docs/en/build-with-claude/prompt-caching

[15] ProjectDiscovery and Parth Malhotra. “How We Cut LLM Costs by 59% With Prompt Caching.” 2026. projectdiscovery.io

[16] Julius Brussee. caveman. GitHub, 2026 (project-reported benchmark). github.com/JuliusBrussee/caveman

[17] Dimitri Mazmanov, Spotify Engineering. “Portal by Spotify cut my Claude Code token usage by 90%.” September 2026. engineering.atspotify.com

[18] Gartner customer service benchmarks. Gartner survey, August 2024. gartner.com/en/documents/5164231

[19] Joel Becker et al., METR. “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity.” July 2025. arxiv.org/abs/2507.09089

[20] Adam Jones and Conor Kelly, Anthropic Engineering. “Code execution with MCP: Building more efficient agents.” November 4, 2025. anthropic.com/engineering/code-execution-with-mcp

[21] WorkOS. “Cloudflare: Code Mode Cuts Token Usage by 81%.” Report on Rita Kozlov's demo at MCP Night, 2025. workos.com

[22] Gartner. “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027.” Press release, June 25, 2025. gartner.com

[23] TypeSafe. Jev and Jev Router, OpenRouter model listings, Sep. 2026. openrouter.ai/~typesafe. For local, open-weight alternatives, see the Laya semantic-operators package. pypi.org/project/semantic-operators