Cutting an AI Code Reviewer's Bill by 68%: The Numbers That Actually Moved

In April 2026 Cloudflare wrote about the AI code reviewer it built in-house. An average review cost a palatable $1.19. By the end of August that figure had climbed to about $2.46, and nothing architectural had changed: the same coordinator handed work to the same specialist reviewers, and the same risk tiers decided how intensive a review would be. In mid-September they pointed agents at the token stream to find out where it was all going, then cut hard. The result: with code quality at high effort, an average review now costs $0.77, down 68% from the $2.43 average across their two most expensive weeks. Median review time fell from 4m 09s to 1m 32s as a side effect. Here are the six changes, ordered by where the money was going.

Cost per review nearly doubled in five months with zero architectural changes

1. Look at Where the Money Goes First

Before optimising, the spend split looked like this: output tokens about 23%, cache writes 43%, cache reads 34%. Two thirds of the bill was context the model had already seen or was about to see again. A single session explains why. The code quality reviewer made 16.6 LLM calls per review on average, writing 72k tokens to cache and reading back 1.08M, because every call resends the whole conversation as context. Cost grows with the number of calls and the size of the context, and over the preceding months both had quietly grown: tokens per review went from 0.91 million to 2.99 million, while cache hit rate rose from 85.7% in April to 97% by September. A high hit rate on three times as many tokens is still expensive.

# Step 1: find out where the money goes BEFORE optimizing anything.
# Cloudflare's split before they started:
#   output tokens  ~23%
#   cache writes   43%
#   cache reads    34%
# Two thirds of the bill was context the model had already seen.

# Pull per-review usage from your provider and group it.
python3 summary.py reviews.jsonl
# avg_calls_per_review 16.6
# cache_write_tokens   72k
# cache_read_tokens    1.08M
# If reads >> writes and cost is still high, your prefix is too long,
# not your prompt.

2. The Real Bug: The First Request Was Not Reading the Cache

Starting September 14, the reviewer recorded the token usage of each session's first LLM call plus a hash of the first system block exactly as it was sent. The data was ugly: the first request of a session found a cached prefix only 14% to 26% of the time. Every session opens with the same tool definitions and the same agent prompt, between 11k and 21k tokens that are identical across every repository under review, so nearly all of them should have hit. The culprit was how OpenCode v1 built the system prompt: one block containing the agent prompt followed by an environment section with the working directory and the date, with the only cache breakpoint at the end of that block. The shared half was never stored on its own, so a review in one repository could never reuse what another had written. On September 16 they added a marker at the end of the agent prompt plus a plugin hook that splits the block there.

# Step 2: check whether your FIRST request actually reads the cache.
# This was the real bug: 14%-26% hit rate on the first call of a session,
# while 11k-21k tokens of tool defs + agent prompt were identical everywhere.

import hashlib, json

seen = {}
hits = total = 0
for line in open("sessions.jsonl"):
    rec = json.loads(line)
    first = rec["calls"][0]
    block = first["system"][0]["text"]        # exactly as sent
    key = hashlib.sha256(block.encode()).hexdigest()
    total += 1
    if first["usage"]["cache_read_input_tokens"] > 0:
        hits += 1
    seen.setdefault(key, 0)
    seen[key] += 1

print(f"first-call cache hit rate: {hits/total:.1%}")
print(f"distinct first blocks: {len(seen)}  (want: 1)")
print("top block appears", max(seen.values()), "times")

3. The Price of Splitting: Frozen Tool Descriptions

Splitting is not free. To keep the shared prefix byte-identical, nothing repository-specific can sit in front of it: the list of available skills was disabled, and tool descriptions cannot change between jobs. You trade per-repository tool customisation for cross-repository cache reuse. The same day they moved the internal standards reviewer's reference index into its prompt: a table of internal standards, a skill file and a list of reference files that every session used to spend its first three turns reading with tool calls. Calls per run dropped from 15.9 to 9.6 and cost per run from $1.12 to $0.72. Together these took the cost per review from $2.47 to $2.02.

# Step 3: split the block so the shared prefix can be cached on its own.
# Cloudflare added a marker at the end of the agent prompt and a plugin
# hook that splits there. Everything repository-specific moved AFTER it.

# Before: one block, one breakpoint at the very end
#   [agent prompt][cwd + date][repo details] <-- breakpoint
#   => the shared part was never stored alone, so no repo reused another's cache

# After: two blocks, breakpoint on the shared half
#   [agent prompt] <-- breakpoint (identical across all repos)
#   [cwd + date + repo details]

# Cost of the split: skills list disabled, tool descriptions frozen.
# Win: standards reviewer calls per run 15.9 -> 9.6, cost $1.12 -> $0.72

4. Delete the Calls That Cannot Succeed

The September 21 and 23 batch looked unremarkable one change at a time and saved about $0.20 per review collectively. The clearest example: sub-reviewers could see the tool that posts the final review comment, and in 57% to 76% of sessions they tried to call it, were rejected by a server-side check, then spent another call recovering. The fix is to remove the tool from their definitions rather than filter at runtime, which also strips about 2.9k tokens of tool definitions from every sub-reviewer request. Second: in 64% of the code quality reviewer's turns it made exactly one tool call, usually reading a single file, so it now reads the shared context and all patches in its first response, up to 150 KB. Third: about two thirds of jobs are a second or later review of the same merge request, so on full-tier re-reviews AGENTS.md reviews are skipped 56% of the time, performance 23% and documentation 13%.

# Step 4: stop paying for calls that cannot succeed.
# 57%-76% of sub-review sessions tried to call a tool they were not
# allowed to use (posting the final comment), got rejected, then spent
# another call recovering.

# Delete the tool from sub-reviewer definitions, do not filter at runtime.
DEFERRED = {"post_review_comment", "assign_reviewer", "merge_ready"}

def tools_for(role: str) -> list:
    if role == "sub_reviewer":
        return [t for t in ALL_TOOLS if t["name"] not in DEFERRED]
    return ALL_TOOLS

# Also: 64% of turns made exactly one tool call (usually one file read).
# Batch reads into the first response -- up to 150 KB of context + patches.

# And cancel cancelled jobs: GitLab's Kubernetes executor sends no signal,
# so 358 of 446 cancelled jobs kept running to completion.
# Poll your own status every 15s and exit.

5. Two Mistakes Almost Everyone Makes

First, the reviewer kept spending after the review was posted: OpenCode passed the result back to the model for one more turn, which re-read the entire conversation to write a summary nobody saw. Stopping that saved about $0.07 per job. Second, cancelled jobs were not actually cancelled: GitLab's Kubernetes executor sends no signal to a job's script, and in a sample of 446 cancelled jobs, 358 ran to completion. The fix is unglamorous — have each job check its own status every 15 seconds and stop. Separately, treating a slow reviewer differently from a broken provider matters: timeouts used to be classified as provider failures and retried with the same limit, burning money twice. With longer limits and lower effort, security reviewer timeouts went from 1.8% of runs to almost none.

# Step 5: share ONE cache prefix across three reviewers.
# Code quality, security and performance now send identical first requests:
# same tools, same shared rules file, then the merge request with all patches.
# Then each reviewer's own instructions. One writes the cache, two read it.

SHARED = [system_tools_block(), shared_rules_block(), mr_with_patches_block()]

reviews = {}
for role in ("code_quality", "security", "performance"):
    messages = SHARED + [role_instructions(role)]
    reviews[role] = call_model(
        messages=messages,
        cache_breakpoints=[len(SHARED) - 1],   # breakpoint on the shared tail
    )

# Prompt cache economics (Anthropic): write = 1.25x input price,
# read = 0.1x, entry expires after 5 minutes unless read again.
# Result: $2.43 -> $0.77 avg review, median 4m09s -> 1m32s.

6. The Last Step: One Prefix Shared by Three Reviewers

The most involved change was making the code quality, security and performance reviewers share a single cache prefix. They now send identical first requests — the same tools, the same shared rules file, then the merge request with every patch included — with each reviewer's own instructions appended after. One of them writes the cache and the other two read it. The economics: a prompt cache write costs 1.25 times the normal input price, a read costs 0.1 times, and an entry expires after five minutes unless something reads it, so the design only works if the prefixes are byte-identical. Final result: $0.77 per average review instead of $2.43, and a median of 1m 32s instead of 4m 09s. Reliability improved too: reviews ending without a verdict went from 0.79% to 0.27%, and security reviewer retries from 3.5% to 0.04%. Cloudflare adds a warning worth repeating: cheaper did not automatically mean fine. Performance reviews lost real findings at medium effort, and security may have as well, so both are being retested at high effort. If you run agents at volume, check two things first: whether each session's first request actually reads the cache, and whether your reviewers still find real problems at the effort level you are paying for.