Cost Engineering

Cost-engineering a multi-agent system

July 18, 2026 · 5 min read

The cost of a multi-agent system is a design property. It is decided when you choose which model runs which step, what happens on failure, and what your infrastructure does while nobody is using it. Discovering the cost afterwards, on the invoice, means the design skipped a requirement.

Building Proposal Architect for the Google for Startups AI Agents Challenge 2026 forced the question early, because a public demo has the worst cost profile there is: bursty, unauthenticated traffic against an expensive inference backend. These are the levers that mattered.

Route models by where quality compounds

A multi-agent pipeline does not need a premium model call at every step. Most steps (extraction, grounding, composition) run well on fast, inexpensive models. We reserved the premium tier for exactly one step, the architect agent, because architecture quality propagates into everything downstream: the estimate, the price, the narrative. One expensive call that improves the cheap ones around it is a good trade; a pipeline of expensive calls is a subsidy to nobody.

Bound the worst case, not the average

Retries are where budgets die quietly. A pipeline that retries everything on any failure has a worst-case cost several multiples of its average. Two bounds change that: a hard deadline per model call and per run (a hung call burns time, not budget), and stage-level retry that reuses upstream state, so a composer failure costs one composer call, not a full pipeline re-run.

Drop idle compute to zero and make peak cost finite

Serverless inference frontends earn their keep on demos and early products: scale-to-zero means the inference frontend's compute drops to zero while idle, and an instance cap means a traffic spike (or an abuse attempt) has a known ceiling. Add an application-level throttle in front of the model calls and the three limits compose: idle compute is free, bursts are bounded, and a single actor cannot monopolize the budget.

Treat billing alerts as part of the architecture

A budget alert is not an ops afterthought; it is the system's smoke detector, and it belongs in the design review next to the timeout table. If the alert ever fires, some bound above has failed, and that is an engineering signal, not an accounting one.

What a run actually costs

The honest answer is an order of magnitude, not a number: a bounded run is a handful of model calls, most of them on inexpensive tiers, with exactly one premium call. The precise figure moves with models, prompt sizes and output lengths, and any decimal-point precision we could print here would be false. What the design guarantees is the shape: per-run cost is bounded, known in advance, and dominated by the one call you chose to make expensive.

The takeaway

Cost engineering for agents is ordinary engineering: route by value, bound the failure modes, cap the infrastructure, and instrument the money. Teams that treat cost as an emergent property get surprised. Teams that treat it as a requirement do not.