Cloudflare Cut Its AI Reviewer Bill by 68%, and Most of the Money Was in Cache Writes

It is rare to get a public cost breakdown of an AI code reviewer with the numbers to two decimal places. When Cloudflare first wrote about theirs in April, an average review cost $1.19. By the end of August that had climbed to about $2.46, and not because of an architectural change: the same coordinator handed work to the same specialist reviewers, and the same risk tiers decided how intensive a review got. In mid-September they pointed their agents at the question of where the tokens were going and cut aggressively. An average review now costs $0.77, down 68% from the $2.43 average of their two most expensive weeks, and the median review takes 1m 32s instead of 4m 09s. About 40% of the savings came from code changes, the rest from experiments with which models they use and how much effort those models spend.

Cost per review: $1.22 in March, $1.44 in June, $1.74 in July, $2.32 in August.

1. The curve first: no single bigger model did this

Holding the mix of review tiers fixed so that a day full of large reviews cannot skew the data, cost per review climbed from $1.22 in March to $1.44 in June, $1.74 in July, and $2.32 in August. Part of that is compositional — 59% of reviews sat in the most expensive tier in March versus 65% in August — and they attribute that to models getting better, which lets engineers produce bigger and more complex changes. But most of the increase came from switching models. At the start of August every reviewer moved onto Anthropic models, taking cost per job from $1.38 to $2.03. Their internal standards reviewer, Codex, ended up on Claude Opus 4.7 and went from about $0.25 per review to $0.88, which made it more expensive than the coordinator agent itself. Meanwhile the cache hit rate rose from 85.7% in April to 97% by September while tokens per review more than tripled, from 0.91 million to 2.99 million. A better hit rate on three times the tokens is still three times the money.

# Step 1: instrument the first call of every session.
# You cannot fix prefix caching until you know whether the
# first request read from cache or paid to write.

def log_first_call(session, response, system_blocks):
    usage = response.usage
    log.info(
        "first_call",
        repo=session.repo,
        cache_read_tokens=usage.cache_read_input_tokens,
        cache_write_tokens=usage.cache_creation_input_tokens,
        input_tokens=usage.input_tokens,
        # hash the first system block exactly as sent
        prefix_hash=sha256(system_blocks[0].text.encode()).hexdigest()[:16],
        breakpoints=[b.cache_control for b in system_blocks],
    )

# What Cloudflare saw for September 14 onward: the first request of a
# session found a cached prefix only 14% to 26% of the time, even though
# 11k-21k tokens of tool definitions and agent prompt are identical
# in every repository being reviewed.

2. Where the money went: 43% cache writes, 34% cache reads, 23% output

Right before the optimisation work, spend split into output tokens at about 23%, cache writes at 43%, and cache reads at 34%. Most of the money was going to context the model had already seen or was about to see again, which makes sense once you look inside one session. The code quality reviewer made 16.6 LLM calls per review on average, and across those calls it wrote 72k tokens to the cache and read back 1.08 million, because each call sends the whole conversation again as context. Session cost grows with both the number of calls and the size of the context, and over a few months they had slowly increased both. Two pricing facts drive everything that follows: writing to the prompt cache costs 1.25 times the input price, reading costs 0.1 times, and an entry expires after five minutes unless something reads it. So the only thing that matters is whether a session's first request reads from cache or pays to write.

# Step 2: put the breakpoint where the sharing actually ends.

# BEFORE (OpenCode v1): one block, breakpoint at the very end.
#   [ agent prompt + cwd + date + repo-specific banners ][ breakpoint ]
#   -> the shared part is never stored on its own, so a review in
#      repo A can never reuse what a reviewer in repo B wrote.

SYSTEM = [
    {"type": "text", "text": agent_prompt + environment_section,
     "cache_control": {"type": "ephemeral"}},
]

# AFTER: split at a marker every job shares.
# The first half ends on bytes that are identical across repositories
# and gets its own breakpoint.
SYSTEM = [
    {"type": "text", "text": agent_prompt + "\n<!--SHARED-END-->\n",
     "cache_control": {"type": "ephemeral"}},   # shared prefix
    {"type": "text", "text": repo_specific_section},   # no breakpoint
]

# To keep the first half byte-stable: repo-specific content must come
# after the marker, the skills list is disabled, and tool descriptions
# cannot change between jobs.

3. Change one: the breakpoint was in the wrong place

Starting September 14 they recorded the token usage of each reviewer's first LLM call along with a hash of the first system block exactly as sent. The data was ugly: the first request of a session found a cached prefix only 14% to 26% of the time. Every session begins with the same tool definitions and the same agent prompt — 11k to 21k tokens identical across every repository they review — so almost all of those should have been hits. The cause was how OpenCode v1 built the system prompt. It constructed one block containing the agent prompt followed by an environment section with the working directory and the date, and placed the only cache breakpoint at the end of that block. The shared part was never stored on its own, so a review in one repository could never reuse what a review in another repository had written. On September 16 they added a marker to the end of every agent prompt plus a plugin hook that splits the block in two at that marker, so the first half ends on bytes every job shares and gets its own breakpoint. Keeping it byte-stable meant removing repo-specific content from in front of the marker: the skills list got disabled and tool descriptions were frozen between jobs. The same day they moved the standards reviewer's reference index — a table of internal standards, a skill file and a list of reference files — into its prompt, ending the three tool calls every session used to spend reading those files. LLM calls per run dropped from 15.9 to 9.6 and cost per run from $1.12 to $0.72. Together the two changes took cost per review from $2.47 to $2.02.

A cache breakpoint in the wrong place means every session pays the write price for a prefix it should have read.

# Step 3: one prefix, three reviewers.
# Code quality, security and performance send identical first requests:
# same tools, same shared rules file, then the merge request with all
# patches. Each reviewer's own instructions follow after that.
# Exactly one of them writes the cache; the other two read it.

async def first_turn(reviewers, request_payload):
    writer, readers = reviewers[0], reviewers[1:]
    write_task = writer.send(first_request(request_payload))

    # A cache entry cannot be read until the response that wrote it
    # has started streaming, and all three start at the same time.
    await writer.wait_for_first_token(timeout=20)
    await asyncio.gather(
        *(r.send(first_request(request_payload)) for r in readers)
    )
    # Costs each reader a small delay; saves about 9 cents per reviewer.

4. Change two: fewer useless calls and fewer useless turns

The next batch, shipped September 21 and 23, was unimpressive item by item and took about $0.20 off every review collectively. The generalisable ones: remove tools reviewers cannot use, because sub-reviewers could see the tool that posts the final review comment, tried to call it in 57% to 76% of sessions, were rejected by a server-side check, and spent another call recovering — dropping it also removes about 2.9k tokens of tool definitions from every sub-reviewer request. Read files in parallel, because 64% of the code quality reviewer's turns made exactly one tool call, usually to read a single file. Skip reviewers on re-reviews, since about two-thirds of jobs are a second or later pass and on full-tier re-reviews AGENTS.md reviews were skipped 56% of the time, performance 23%, documentation 13%. Stop after the review is posted, because OpenCode passed the result back for one more turn that read the entire conversation to write a summary nobody saw — about $0.07 per job. Treat a slow reviewer differently from a broken provider, which together with longer time limits and lower effort took security reviewer timeouts from 1.8% of runs to almost none. And actually cancel cancelled jobs: GitLab's Kubernetes executor does not signal a job's script on cancellation, so in a sample of 446 cancelled jobs, 358 kept running to completion; jobs now check their own status every 15 seconds. The most architectural change is a shared prefix across three reviewers. Code quality, security and performance now send identical first requests — same tools, a shared rules file, then the merge request with every patch — with each reviewer's own instructions after that. Exactly one of them writes the cache and the other two read it. Because an entry cannot be read until the response that wrote it starts streaming, and all three start at the same time, the second and third wait up to 20 seconds. That costs a little latency and saves about nine cents per reviewer.

# Step 4: stop paying for calls the model cannot make,
# and stop reading files one at a time.
reviewer_tool_policy:
  code_quality:
    tools:
      - read_file
      - read_files_parallel     # 64% of turns made exactly one call
      - post_review_comment
  sub_reviewer:
    tools:
      - read_files_parallel
      - post_finding
    # do NOT expose post_review_comment: 57%-76% of sub-reviewer
    # sessions tried to call it, were rejected server-side, and spent
    # another call recovering. Removing it also drops ~2.9k tokens
    # of tool definitions from every sub-reviewer request.
  stop_after_review_posted: true   # was: one extra turn summarising
                                    # a summary nobody read (~$0.07/job)
  timeout_is_not_provider_failure: true
  recheck_cancellation_every_seconds: 15   # 358 of 446 cancelled
                                            # jobs kept running

5. Models, effort, and whether cheaper meant worse

Splitting the period at every release and configuration change produces a slightly counterintuitive result: code changes account for $0.65 of the $1.67 saving, and model plus effort changes for $1.06. The coordinator had been running three model-and-effort combinations since August; on the same model, full-tier jobs at medium effort finished 20% sooner, so the other two were dropped. A later randomised split found Opus 5.5 at medium showed no measurable change in review outcomes or critical findings versus Opus 5, so they switched; Opus 5.5 at high effort cost more without a meaningful increase in findings. Opus 5.5 is cheaper per token, but price accounts for only about $0.20 of the drop — the rest is sending and generating fewer tokens. Since the start of September, cache reads per review are down 82%, cache writes down 54%, and output down 67%. On October 3 they split every Sonnet reviewer 50/50 between Sonnet 5 and Sonnet 5.5; after two and a half days 5.5 was 62% cheaper per run for code quality, 58% for performance and 47% for security, at identical per-token prices, so all of them moved on October 5. Then the hard part: proving cheaper is not worse. The headline numbers are alarming — with code quality at high effort, 40% fewer critical and warning findings than late August, and security and performance at medium effort down 56% and 71%. So they checked from two directions. First, did authors act? Comparing the reviewed commit with the author's next push, and counting a finding as acted on if the lines it pointed at changed, the act-on rate for serious findings rose from 34% to 59%, with acted-on findings per 1,000 reviews at 132 versus 145 before — inside the margin of error — and 229 to 321 across all four reviewers. Second, they had two models from different providers blindly judge 657 findings without knowing which period they came from, with disagreements going to a third model: important findings per review went from 0.28 to 0.30, intervals overlapping, while findings rated more severe than they deserved fell from 0.46 per review to 0.14, a 70% drop. What disappeared was mostly noise and inflated severity. Medium effort did lose real findings, which is why code quality moved back to high.

The quality check that mattered: authors acted on 59% of serious findings, up from 34%.

# Step 5: attribute the saving, then check the quality.
spend_split_before = {"output": 0.23, "cache_write": 0.43, "cache_read": 0.34}

saving = {
    "code_changes": 0.65,      # breakpoints, dead tools, fewer calls
    "model_and_effort": 1.06,  # which model, how much effort
}                              # total $1.67 per review

# Per-token price was only about $0.20 of the drop. The rest came from
# sending and generating fewer tokens: cache reads -82%, cache writes
# -54%, output -67% since the start of September.

quality_gate = {
    "acted_on_rate": (0.34, 0.59),        # before -> after
    "acted_on_per_1k_reviews": (132, 145),  # within noise
    "inflated_severity_rate": (0.46, 0.14),  # blinded judge, 657 findings
}
# Cheaper findings are fine only if authors still act on them.
# If acted-on rate falls, you cut findings, not noise.