Flint Puts a 39.8B Coding Agent Behind 2.5B Active Parameters — and It Runs One GPU per Replica

Recursive published Flint to Hugging Face on October 21, 2026 under Apache 2.0, with the weights downloadable directly. The positioning is narrow and useful: not a general chat model but a model for coding agents. The parameter table makes that concrete. It carries 39.8B total parameters while activating roughly 2.5B per token — the active body alone is about 2.2B, with the documented convention including the output head — which the vendor rounds to 40B-A2.5B. Of the 49 layers, two are dense and 47 are mixture-of-experts; each MoE layer holds 256 routed experts, selects 6 per token, and adds an always-active shared expert. The practical consequence: pay inference costs at 2.5B, keep representational capacity closer to 40B, and suddenly running a coding agent on hardware you own is a real option.

Thirty-five sliding-window layers see 513 tokens back; fourteen full-attention layers carry the long range.

1. The rows in the spec sheet that actually change your day

Attention is the most informative part of the sheet. Thirty-five of the 49 layers are sliding-window layers with a 513-token window — the current token plus 512 preceding ones — and the remaining fourteen are full attention. There are 32 query heads against 4 key/value heads at head dimension 128. Sliding layers use RoPE; the full-attention layers carry no positional encoding at all and instead use learned per-head attention sinks. The intent is KV cache economics: most layers look back half a thousand tokens, and long-range work is delegated to a minority of full-attention layers. The documented context is 262,144 tokens, with an explicit note that prompt and generated output share that budget. Vocabulary is 150,144 and the stated knowledge cutoff is June 2026. For anyone writing a harness, the sharing note matters more than the 262k headline — it is not 262k of input plus output on top. The practical read is that a harness tuned for a dense 8B model will look fine on short prompts and then hit an invisible wall: the thirty-five sliding layers only carry 513 tokens of history, so anything the agent must remember about the repository has to be re-stated in the prompt or kept in the tool result payload.

# Serve one replica on a single B300 (or H200). The SGLang kit ships
# in the repo; Transformers modeling code is not included yet.

pip install -U huggingface_hub openai
hf download RecursiveSI/flint --local-dir flint      # about 80 GB, BF16
cd flint
docker build -t flint-sglang sglang
docker run -d --name flint --gpus '"device=0"' --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -v "$PWD:/model" -e MODEL=/model \
  flint-sglang bash /model/sglang/serve_b300.sh      # H200: serve_h200.sh

# Readiness, not sleep(60).
until curl -sf localhost:30000/health_generate >/dev/null; do sleep 15; done

# The server is OpenAI-compatible on /v1.
python3 sglang/example_client.py --base-url http://localhost:30000/v1

2. Serving: one GPU per replica, get it running before scaling it

The deployment guidance is blunt: one B300 or H200 per replica. The repository ships an SGLang serving kit under sglang/, the BF16 checkpoint is about 80GB, and after download you build the image and pick serve_b300.sh or serve_h200.sh, which exposes an OpenAI-compatible API on /v1. Two things to internalise. First, Transformers modeling code is not included yet, so from_pretrained is not an option today; SGLang is the supported path. Second, gate readiness on the health_generate endpoint rather than a fixed sleep. Loading 80GB of weights plus graph compilation takes a variable amount of time, and the usual sleep(60) is frequently not enough while sleep(300) wastes four minutes on every restart.

# Sampling is part of the model contract, not a preference.
# Flint documents temperature 1.0, top-p 0.95, top-k 20, and an output
# budget of at least 32768 tokens INCLUDING reasoning.
from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="unused")

SAMPLING = {"temperature": 1.0, "top_p": 0.95, "extra_body": {"top_k": 20}}

messages = [{"role": "user", "content": "Fix the failing test in tests/test_dispatch.py"}]

first = client.chat.completions.create(
    model="flint",
    messages=messages,
    max_tokens=32768,
    **SAMPLING,
)
msg = first.choices[0].message

# Keep reasoning_content in history. Dropping it breaks multi-turn tool use.
messages.append({
    "role": "assistant",
    "content": msg.content or "",
    "reasoning_content": getattr(msg, "reasoning_content", "") or "",
    **({"tool_calls": msg.tool_calls} if msg.tool_calls else {}),
})

# Temperature 0 is the reflex from other coding models. Here it flattens
# the distribution the model was tuned around, and long tool loops stall.

3. Sampling settings are part of the contract, not a taste preference

The model card states temperature 1.0, top-p 0.95 and top-k 20, and recommends an output budget of at least 32768 tokens for hard problems including reasoning. Treat those lines as a contract. Two failure modes show up immediately. The first is importing instincts from other coding models and setting temperature to 0 or 0.2; that flattens the distribution the model was tuned around, and long tool loops start spinning rather than converging. The second is budgeting 8k or 16k of output, which truncates the reasoning trace mid-plan and produces something that looks like a patch but is half a plan. With only 2.5B active parameters generation is cheap, so the expensive part is not the tokens — it is the retry after a truncated trace.

One B300 or H200 per replica, about 80GB of BF16 weights.

// Bundle the sampling contract with the checkpoint name so nobody
// silently inherits another model's defaults in a shared harness.
{
  "model": "flint",
  "context_window": 262144,
  "context_note": "prompt and generated output share this budget",
  "sampling": { "temperature": 1.0, "top_p": 0.95, "top_k": 20 },
  "max_output_tokens": 32768,
  "output_budget_note": "counts reasoning tokens, not just visible text",
  "tool_calls": { "style": "openai", "parallel": true },
  "history": { "preserve_reasoning_content": true },
  "stop_when": ["tests_pass", "patch_applied", "max_turns:24"]
}

// Why the output budget matters: at 2.5B active parameters inference is
// cheap, but a reasoning trace that gets truncated mid-plan produces a
// confident half-patch. Budget the trace, not the answer.

4. The tool loop has to preserve reasoning_content

Flint supports OpenAI-style tool calls, with one detail that is mandatory rather than optional: keep the returned reasoning_content in conversation history. Drop it and the model has to re-derive its own progress on the next turn, which shows up as repeated identical tool calls or, by turn six, edits to code it just wrote. The second detail is context arithmetic. The 262,144-token window covers prompt plus output, so if a tool returns four thousand lines of logs and you reserve 32768 tokens for output, the remaining room is thinner than it looks. What works: summarise tool output, write large artefacts to disk and return a path plus line numbers, and cap the loop explicitly — 24 turns is a defensible ceiling in practice.

# Laptop-side serving. Unsloth publishes quantised GGUFs; pick by memory,
# then verify the quant against the BF16 reference before trusting it.
pip install -U "huggingface_hub[hf_transfer]"
export HF_HUB_ENABLE_HF_TRANSFER=1

# Q4_K_XL is the usual first stop for 64GB unified memory machines.
hf download unsloth/flint-GGUF --include "*Q4_K_XL*" --local-dir flint-gguf

llama-server -m flint-gguf/*Q4_K_XL*.gguf \
  --ctx-size 65536 \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  --jinja \
  --port 8080

# Notes that cost us an afternoon:
# 1. --jinja is required for the chat template and tool calls.
# 2. Start at 64k context, not 262k. KV cache scales; unified memory does not.
# 3. Quantisation changes the distribution. Re-check tool-call validity
#    on your own repo before switching a harness over.

5. How to read the three benchmark numbers

The published figures are 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2 and 74.2% on SWE-bench Verified, all at temperature 1.0, top-p 0.95 and top-k 20, with Terminal-Bench run through the Terminus 2 harness and SWE-bench through mini-swe-agent, reported as average pass@1 over at least five attempts. Two footnotes matter. First, Flint's Terminal-Bench score counts solutions obtained through web retrieval as unsuccessful, while comparison rows in the same table are developer-reported and may not apply that adjustment, so cross-row comparisons deserve a discount. Second, the 89.7% and the 74.2% describe different task distributions rather than an unstable model, which is why model selection should run on your own repository's issues instead of a leaderboard. One more habit worth adopting: record the checkpoint hash plus the sampling parameters next to every score you publish internally. Without that pairing, a regression three weeks later cannot be attributed to a model change, a quantisation change or a harness change, and the argument turns into guesswork.

Pass@1 averaged over at least five attempts, with web-retrieval solutions counted as failures.

# Reading an agentic benchmark honestly: pass@1 over at least five
# attempts, and know what the harness counts as a failure.
import statistics

def pass_at_1(run_task, tasks, attempts=5):
    scores = []
    for task in tasks:
        outcomes = []
        for _ in range(attempts):
            result = run_task(task)
            # Flint's Terminal-Bench number excludes solutions that reached
            # the answer through web retrieval. Apply the same rule or you
            # are not comparing the same thing.
            ok = result["tests_passed"] and not result.get("used_web_retrieval")
            outcomes.append(1 if ok else 0)
        scores.append(sum(outcomes) / attempts)
    return {
        "pass@1": round(statistics.mean(scores) * 100, 1),
        "tasks": len(tasks),
        "attempts": attempts,
    }

# Published by Recursive: 54.9% Terminal-Bench 2.1 (Terminus 2 harness),
# 89.7% SWE-bench Pro v2, 74.2% SWE-bench Verified (mini-swe-agent).
# Comparison rows in the same table are developer-reported and may not
# apply the web-retrieval adjustment.