Skip to content
HONTRAN.DEVCreative Development

· 11 min read

Claude Code ModelRouting: What IBenched, What It CostClaude Code model routing with real numbers: Opus vs Sonnet and medium vs high effort, benched on review, build and creative work, with every cost.

Claude Code Model Routing: What I Benched, What It Cost
Studio note 089 — coverAll notes

If your team runs Claude Code every day, the bill is not decided by how much you use it. It is decided by which model and which effort level each piece of work runs on. That is Claude Code model routing, and most teams leave it on the default, where a five-line config edit runs on the same expensive setting as an architecture review. I run 40+ role agents across my own delivery work, each pinned to its own model and effort. Before I pinned any of them, I benched the choice. This post is the data: what moved quality, what only moved cost, and the one place where my own benchmark was wrong.

Why Claude Code model routing matters more than usage

First I measured where the money went. Across 159 of my own logged Claude Code sessions, my original rule ("the main session's tokens are the expensive ones, so delegate away from it") turned out to be a category error. Tokens are priced by model, not by role. An Opus subagent costs exactly what an Opus main session costs.

What the telemetry did show:

  • The main transcript is re-read on every turn. Across the logged sessions, cache reads ran about 370× total output. Every edit-and-check loop the main session runs itself makes the thing that gets re-read longer, for the rest of the session.
  • Long hand-run sessions are the leak. Sessions with 1–3 main-session edits were 3% of all output tokens (20 work sessions). Sessions with 31+ main-session edits were 58% (33 work sessions).
  • Delegation was happening, but sideways. Subagents already carried 22.9% of all output tokens. But about 52% of subagent spend ran on Opus, the same tier as the main session, so only 9.3% of all output landed on a cheaper model.
  • Fix rounds climbed back up. Sessions that delegated still averaged ~36 source edits on the main session, more than self-only sessions at ~29, because the build went down to a subagent and the fix round came back up to the expensive main session.

So the lever is not "use AI less" and not "delegate more". It is delegate down a tier for work that is already decided, and keep the expensive model for judgment.

How a routing decision is pinned

Claude Code lets each subagent carry its own model and effort in its frontmatter. Per the official subagent docs, a subagent without a model falls back to the main conversation's model, and a subagent without effort runs at the session's effort level. That inheritance is the trap: a team lead who raises effort to high for one hard problem silently raises every build their session dispatches.

A pinned builder looks like this:

yaml

---
name: spec-builder
description: Implementer for settled behavior with acceptance checks.
model: sonnet
effort: medium
---

I checked that the pin really changes the run. Same build task, dispatched from a session sitting at high: the pinned Sonnet/medium builder used 65.6k tokens, 122 s, 17 tool calls and scored 19.5/23 on hidden tests. An unpinned general-purpose Sonnet agent, which inherited high, used 84.0k tokens, 190 s, 22 tool calls for 19/23. That is −22% tokens and −36% time at the same quality, from four lines of frontmatter.

The bench: one recipe for every routing question

Every row in my routing matrix comes from the same A/B recipe. Each arm runs headless with an identical prompt; only the model or effort flag differs, and hooks are off so nothing adds turns:

bash

claude -p "$(cat prompt.txt)" --model opus --effort high \
  --settings '{"disableAllHooks":true}' --output-format json

The JSON output carries total_cost_usd, duration_ms and num_turns. Three rules make the numbers worth anything:

  • Score against ground truth, not vibes. Reviews run against a commit with 9 known, seeded defects. Builds are scored by a hidden test suite (23 tests) the builder never sees.
  • At least 2 runs per arm. One run per arm is an anecdote.
  • A test every arm fails is a test bug. Drop it instead of reading it as a result.

Effort: where high pays and where it doesn't

WorkArmScoreTimeCost per run
Code review (Opus, 9 seeded defects)medium9/9 ×230 s$0.32
high9/9 ×2, plus 1 real unseeded bug in one run52 s$0.33
Build (Sonnet, 23 hidden tests)medium19 and 20/2378 s$0.39
high20 and 20/23125 s$0.48

On the build, high bought nothing measurable: quality within noise, +25% cost, +60% time, because the spec already bounded the work. On the review, high cost about the same and one run found a real bug nobody had seeded: pausing right after mount cancelled the first animation frame and left a blank stage. Effort pays where judgment has to find what nobody listed. It does not pay where a spec already lists everything.

So builds and fix rounds run at medium, and verification roles run at high.

Opus vs Sonnet for review: where my benchmark was wrong

The model axis got the same treatment. On the same 9-defect review, the day Sonnet 5.5 launched:

ReviewerSeeded defectsUnseeded bugAvg costAvg time
Sonnet 5.5, high9/9 ×21/2$0.2339 s
Opus 5.5, medium9/9 ×20/2$0.2233 s
Opus 5.5, high9/9 ×21/2$0.4152 s

On paper, Sonnet at high effort matched Opus at high effort for ~56% of the cost. So I moved the pre-merge reviewer to Sonnet, but as a trial, not a switch: for the next 3 real merges, an Opus reviewer ran in the shadow on the same diff, with one exit rule written down in advance. Any critical or important finding that only Opus caught sends the role back to Opus.

It happened on the first merge. On a branch of my own site, Sonnet returned "mergeable, 8 minor notes". Opus found an important bug: holding a key on a hold-to-confirm button moved focus into the contact form while the key was still down, so keyboard auto-repeat typed into the Name field and submitted the empty form eight times. The second merge, both reviewers caught the same bug. The third, both were clean. One Opus-only important finding in three merges, so the reviewer went back to Opus.

The lesson is not "Sonnet is worse". A static-diff bench with one fixture cannot see bugs that only exist in a real interaction, so a routing change needs a trial on real work with an exit rule.

If you want this kind of routing set up for your own team, with the trial protocol and not only the config, that is the work I take on.

Whole-team routing and the late-handoff trap

The last test runs the whole system. An Opus orchestrator delegates one build and one review. Arm A uses the house matrix (Sonnet/medium builder, Sonnet/high reviewer, the configuration at the time of the test). Arm B is told to ignore house routing and use general-purpose agents only. It chose Sonnet for the build and Opus for the review, and both inherited high.

  • Review: 9/9 in all 4 runs.
  • Build: A 22/23 ×2 (it missed only a test beyond the spec), B 23/23 ×2.
  • Total cost: A $0.76 vs B $0.86, −12% for the matrix. Nearly all of B's extra was the Opus reviewer.

Then I turned the house hooks back on with a natural, feature-sized prompt and found the trap. The session started editing on the main model, a budget hook nudged it after about 3 edits, and it handed the rest to the Sonnet builder. Same quality, 9/9 ×2, but $1.13 vs $0.77, +47%. A late handoff double-pays: the builder re-reads everything the main session already read. The fix was a rule, not a model: decide the route before the first edit. After that change, both runs judged the feature small, did it on the main session start to finish, and scored at the top of the rubric with no handoff.

Creative work is the exception

Everything above is about work that a spec or a test suite can score. Creative work is different, and the data says so. On one art-directed hero-and-preloader brief, Opus at three effort levels, 2 runs each, all six builds passed the same technical checks (console, 375 px overflow, reduced motion). The blind pass from me:

  • xhigh: 2/2 passed, ~$8.66–$9.78 per run, ~40–51 min.
  • medium: 1/2, ~$2.87–$3.00 per run, ~12 min.
  • high: 0/2.

Per passing build, medium cost ~$5.9 (two tries) and xhigh ~$9.2 (one try). A second blind judge, a designer, also put both xhigh builds on top. Part of the reason is honest and unflattering: xhigh runs loaded the house skills more often, so "xhigh wins" is partly "xhigh read the instructions". That is the same lesson as my skills library A/B benchmark: on motion work, judging from a static screenshot misses what the live build earns or loses.

One more data point keeps teams from over-paying. On a Figma-comp-to-layout bench, every one of six Sonnet and Opus runs landed within 1 px of the comp. Opus/high cost $4.57–$5.10 per run against $1.66–$2.30 for Sonnet/medium. The visible differences came from the right font file and an image-quality setting, both of which are lines in a brief, not a model tier.

The routing matrix, as it runs today

RoleModelEffortEvidence
Main session: context, judgment, acceptanceOpusmedium (raised by hand for hard problems)workflow default, not a benchmark
Settled build, fix round, port, comp → layoutSonnetmediumbuild high = medium quality at +25% cost
Pre-merge reviewOpushigh3-merge shadow trial, 1 Opus-only important
Unresolved creative/motion judgmentOpushigh, or xhigh when namedblind pass xhigh 2/2 · medium 1/2 · high 0/2
Read-only search and bulk summariesHaiku—owner-approved, not benched, never edits code

Every row records whether it was benched or only decided. Rows marked "not benched" stay provisional, and every row is re-benched with the same recipe when a new model launches.

What this changes for a team

For a founder or a head of engineering, the takeaways are operational:

  1. Pin model and effort per role. Inheritance quietly turns one person's hard problem into everyone's expensive default.
  2. Spend effort on verification, not on settled builds. Review and QA found real bugs at high; builds did not get better.
  3. Never switch a role on a benchmark alone. Run it in the shadow on real work with an exit rule written before the trial.
  4. Route before the first edit. A late handoff cost +47% in my tests.
  5. Write every decision down with its evidence. My setup keeps 12 dated decision records, so the next model launch is a re-bench, not a debate.

FAQ

Should every Claude Code subagent run on Sonnet to save money?

No. Settled builds and fix rounds did as well on Sonnet at medium effort. Pre-merge review went back to Opus after a real merge showed an Opus-only important bug. Route by role, and keep the evidence for each row.

Does higher effort make Claude Code write better code?

On a spec'd build with hidden tests, no: high scored the same as medium (20/23) for 25% more cost. On review, high was the only setting that found an unseeded bug. Effort pays for judgment, not for typing.

How do I check which model and effort a subagent really used?

Transcripts do not show effort. The API request does (output_config.effort), so I log requests through a small local proxy and tag each dispatch with a probe string. That is how I confirmed the frontmatter pins hold even when the session sits at a different effort.


I build award-level websites (19 international awards, 2× Awwwards Independent of the Year nominee) and I run that delivery on this agent system every day: same quality bar, routed so the expensive model only does the work that needs it. You can see the sites in my work. If you want the same setup for your studio, agency or product team (role agents, model and effort routing, guardrails and the measurement behind them), let's talk.

Document the thing you don’t want to debug twice.

HONTRAN.DEV · Notes / 094Studio note 089 of 094