Subagent effort set to low: 72% fewer tokens, seeded bugs still caught

TL;DR: three things to know before you set it

1. Check your version first: claude --version. Our original 2.1.197 install doesn't contain CLAUDE_CODE_SUBAGENT_EFFORT at all, so setting it silently does nothing. 2.1.316 has it.
2. It works, but low isn't guaranteed all the way down: in 2 low-effort runs the subagent handed off to the built-in code-review skill, and in one of them that nested agent ran at medium, costing as much as a high-effort run.
3. Don't trust single-run comparisons: 6 runs of the same setup ranged from 302k to 643k total tokens, a 2.1× spread. Run each arm at least 5 times and compare medians.

Background

Boris Cherny (Claude Code) reviewed the session IDs users shared and said they found no single root cause or regression; heavy usage mostly traced back to one or two heavy sessions. On effort level:

Subagents inherit it by default and it can add up fast.

Boris Cherny, X

In a follow-up reply he announced CLAUDE_CODE_SUBAGENT_EFFORT. We wanted to know how much lowering a review subagent's effort actually saves, and what it costs in quality. All numbers and interpretation below are from our own test, not claims by anyone above.

Setup

Question

Review subagent at the inherited high effort vs set to low: how do tokens, time, cost and review quality differ?

Tool

Claude Code 2.1.316 (npm latest), headless claude -p, stream-json output

Model

Pinned --model claude-sonnet-5; the subagent is the built-in general-purpose agent, also resolved to claude-sonnet-5

Session effort

Default, which resolved to high for claude-sonnet-5 (same in both arms)

Task

Review the main...feature/review-me diff of a small Python project, ledgerlite (commit 4f95831), with 6 known bugs seeded that the existing tests don't catch

Arms

A: env var unset (subagent inherits high). B: CLAUDE_CODE_SUBAGENT_EFFORT=low. Everything else identical.

Protocol

1 warm-up run (A, not counted), then A1, B1, A2, B2 … A5, B5 interleaved. git reset --hard && git clean -fd before every run, identical prompt; the main agent had to delegate the whole review to exactly one subagent (all 11 runs made exactly 1 Agent call).

Permissions

Read-only: only Read / Grep / Glob / Agent, git diff/log/show/status and unittest allowed; Edit, Write and the network denied, as were reads of the answer key and log folders. No run modified a file.

Measurement

Tokens and cost from the CLI's usage output and total_cost_usd (list-price estimate); time is wall-clock

Scoring

Against an answer key written beforehand, by hand. Strict: right location (file + line or function) and right cause. Lenient: right cause, roughly right location

Results

The low arm used clearly fewer total tokens, less wall time and less subagent output, by a margin far larger than same-setup rerun noise. Median cost also halved, but because B1 was unusually expensive, that difference is not significant. The cheapest A run (399k) still used more tokens than 4 of the 5 B runs.

Scroll the table sideways →

p-values: exact permutation test on the median difference (all 252 splits) and exact two-sided Mann-Whitney U. With 5 vs 5, the smallest achievable permutation p is 0.048 and the smallest Mann-Whitney p is 0.008. Bold = permutation p < 0.05.

Every run

Scroll the table sideways →

"Subagent request effort" is the effort parameter recorded on each API request in the subagent transcripts, with counts (nested agents included). B4's subagent ran in the background; its tokens and cost are cumulative and wall time is the time metric. Per-run scoring notes are in results.csv.

Did the env var actually take effect?

Yes, observed directly rather than inferred:

• API request parameter: every subagent transcript records the request's "output_config":{"effort":…}. Every subagent request in A was high; in B it was low (except B1's nested skill, see below).
• Hook log: a PreToolUse hook logged each tool call's effort.level and $CLAUDE_EFFORT. The main session was high in every run; subagent tool calls were high in A and low in B.
• No override: the main agent never passed an effort when it called the subagent, so the result comes from the env var alone.

The hook adds one small python process per tool call in both arms equally. The CLI's --debug log and stream-json output carry no effort field, so they can't tell you whether it worked.

Three findings

1. Older versions don't have it, and don't complain

Our first round ran on Claude Code 2.1.197. Its binary doesn't contain the string CLAUDE_CODE_SUBAGENT_EFFORT, so the "low" arm was identical to the default, with no warning. It only took effect after upgrading to 2.1.316. Run claude --version before relying on it.

2. The same setup can vary 2× between runs

Those 6 old-version runs were effectively one condition run 6 times (model claude-sonnet-5): total tokens ranged from 302,087 to 643,134. In round two, the A arm alone ranged from 399k to 723k. A one-run-each "saved X%" comparison can't tell the setting's effect from luck.

3. Setting low doesn't make every layer low

In B1 and B3, the low-effort subagent handed the review off to the built-in code-review skill, which ran as a nested agent. In B1 that nested agent made 13 requests at medium, and B1 cost $0.392, more than any A run. B3's nested agent ran at low but produced the weakest review: 4 bugs caught, 4 findings. No A run took this path.

Limitations

• One task, one model (claude-sonnet-5, session effort high), 5 runs per arm, one machine, one evening. A bigger repo could change the ratio.
• Cost is the CLI's list-price estimate (total_cost_usd), not a bill; on a subscription it shows up as quota, not dollars.
• Tokens include cache reads, which were 72–93% of each run's total (median about 88%).
• Strict scoring is manual. A4 and B4 both placed the percent-validation bug at coupons.py:16 (the field declaration, 7 lines from the key's :23); both were scored strict miss, lenient hit.
• Quality is n=5: strict-score Mann-Whitney p = 0.040, but permutation p = 0.524, and 3 of B's strict misses were location-only. Not enough to settle quality.
• No false positives in any run.

Reproduce

1. Download the experiment repo (subagent-kit.zip), unzip, git checkout feature/review-me. The answer key is not included.
2. Put our prompt.txt in the repo root.
3. Run the commands below, alternating A and B, at least 5 runs each, resetting the repo before every run:

claude --version            # 2.1.316 (Claude Code)
git checkout feature/review-me && git reset --hard && git clean -fd

# A: default (env var unset; subagent inherits the session's high)
env -u CLAUDE_CODE_SUBAGENT_EFFORT claude -p "$(cat prompt.txt)" \
  --model claude-sonnet-5 --output-format stream-json --verbose \
  --allowedTools "Read" "Grep" "Glob" "Task" "Agent" \
    "Bash(git diff:*)" "Bash(git log:*)" "Bash(git show:*)" "Bash(git status:*)" \
    "Bash(python -m unittest:*)" "Bash(python3 -m unittest:*)" \
  --disallowedTools "Edit" "Write" "NotebookEdit" "WebFetch" "WebSearch" > A1.stream.jsonl

# B: the only difference is this env var
CLAUDE_CODE_SUBAGENT_EFFORT=low claude -p "$(cat prompt.txt)" ... (same flags as above) > B1.stream.jsonl

To verify the effort level, add --settings with our hook settings, or read output_config.effort in ~/.claude/projects/…/subagents/agent-*.jsonl. The RUNBOOK inside the kit describes an interactive version using /usage; that works too, just less precisely than headless.

Raw data

• results.csv: round-two per-run tokens, cost, time, effort parameters, scores and notes (warm-up included, marked not counted)
• stats.json: per-run values, medians, % change and p-values for each metric
• phase1-results.csv: round one (6 same-condition runs on 2.1.197, plus one run per arm on 2.1.316)
• prompt.txt, hook_settings.json, subagent-kit.zip

Next

• Inherited vs fresh context: not tested here. Matt Pocock found Codex was biasing most subagents to inherit the parent's context, and changed mattpocock/skills so every subagent starts fresh (#1341). Claude Code has no official switch for this, so we first need a reproducible way to set it up.
• Codex comparison: the same task in Codex, shown side by side only, since the two meter usage differently.
• Other tasks and models: a bigger diff and a different model, to see whether −72% holds.

Sources

• Boris Cherny — session ID review, effort inheritance, /usage (X)
• Boris Cherny — CLAUDE_CODE_SUBAGENT_EFFORT landed (X)
• Matt Pocock — Codex subagents forking the parent context (X)
• mattpocock/skills PR #1341 — fresh context for every subagent