← AI Labs

AI Lab · Technical experiment · 2026 · 3 min read

Multi-agent router

Can low-cost models do most of the work if a frontier model only plans and reviews? A personal tool, and an experiment in trading cost against quality.

Skip to the decisions

Problem

Using a frontier model for every step is expensive and slow. Most steps are routine and do not need it.

What I did

Built a three-layer agent setup: frontier models only plan, review and sign off; low-cost models do the work in parallel.

Status

Working on my own projects. No benchmark result yet: the comparison against a Claude-only setup is running.

Role
Solo: framing, build, evaluation
Type
Technical experiment
Models
Claude directs; lead models own streams; DeepSeek and Gemini Flash work
Status
Working (Oct 2026)
Skills
AI evals, cost vs quality vs latency

Problem

One model for everything is expensive

I was using a frontier model for every step of my AI work. Most of those steps did not need it.

  • Drafting boilerplate code, reformatting text and summarising files are routine tasks.
  • Planning, judging quality and fixing hard bugs need the strongest model.
  • Paying top price for both meant costs grew with every project.

How it works

Plan at the top, work at the bottom

LAYER 1 · FRONTIER MODELDirectorClaude · plans, signs offLAYER 2 · LEADSEngineering leadbreaks down, reviewsContent leadbreaks down, reviewsResearch leadbreaks down, reviewsLAYER 3 · WORKERSCoderlow-cost modelTesterlow-cost modelWriterlow-cost modelEditorlow-cost modelResearcherlow-cost modelSummariserlow-cost modelSTAFF ROLES · SERVE EVERY LAYERDispatchercheapest capable modelQA reviewerchecks every outputBudget keepercost cap per taskEscalationone retry, then up
Three layers. Frontier capacity sits only where judgment is needed; the volume of work happens in Layer 3 at low-cost prices.
  • Layer 1, Director (Claude): owns the goal, splits it into streams, gives the final sign-off.
  • Layer 2, Leads: one per stream. Each breaks its stream into steps and reviews the result.
  • Layer 3, Workers: low-cost models with one job each, run in parallel.
  • Every output is reviewed. A failed step gets one retry, then moves up a layer.

Key insight

Key insight

The cheap model doesn't need to be right every time. The system does.

A low-cost worker that is right most of the time, inside an org that reviews every output and escalates misses, can match a frontier-only setup at a fraction of the cost. The structure, and especially the reviewer, is the product.

Goals and metrics

Cost, quality and time, as one decision

North StarCost per accepted taskNorth Star. Cheap output that fails review is not a saving.
Quality pass rateShare of tasks that pass a simple rubric
Time per taskWall-clock time, with parallel steps
Escalation rateShare of steps that end up back with Claude

Guardrail: pass rate no more than 5 points below the Claude-only baseline. That is the trade I accept for at least 40% lower cost: a failed step is caught in review and retried, so a small quality drop costs little. Past 5 points, rework eats the savings.

Decisions and trade-offs

Three calls, and what each one cost

01Orchestrator

What I chose Chosen

A frontier model plans and reviews

What I gave up

The lowest possible cost

Why. Planning and judging are where cheap models fail most.

02Review every output

What I chose Chosen

Claude checks worker results

What I gave up

Some of the savings

Why. Unchecked cheap output creates rework, which costs more.

03Escalation cap

What I chose Chosen

One retry, then escalate

What I gave up

Maximum savings on hard tasks

Why. Endless retries waste time and money.

Proof

Benchmark against a Claude-only baseline

MetricTarget vs Claude onlyStatus
Cost per taskAt least 40% lowerBenchmark running
Quality pass rateNo more than 5 points lowerBenchmark running
Time per taskFaster on multi-step tasksBenchmark running
Escalation rateUnder 20% of stepsBenchmark running

How it runs: 20 to 30 real tasks across code, writing and summaries, each through both setups, scored with a simple pass or fail rubric, noting where the router failed.

Where it breaks

Known failure modes

  • Data privacy: sending work to some third-party providers may break company policy. Fix: an approved-provider list per team.
  • Reviewer cost: if Claude has to redo too much, savings disappear. Watch the escalation rate.
  • Silent errors: a confident wrong answer can pass review. Fix: stricter checks on high-risk tasks, such as code that touches data.

What I learned

Evaluation is the real work

  • Routing logic is easy to write and hard to trust without a benchmark.

Next

What happens next

  • Finish the 20 to 30 task benchmark and publish the table above with real numbers, including one real failure and its fix.