Clef-omni: Multimodal Decision Models Instead of LLM Classification

Cloudflare shipped Clef-omni, a decision model that accepts audio and video on top of text and images, a week after releasing Clef and Clef-flash. The reason this is more interesting than another model launch is the shape of the output. A decision model does not generate text. You hand it a set of named fields and it returns values for those fields, scored against a schema, in a single prefill pass. That removes the step where most classification pipelines actually break, which is parsing a paragraph into a field. Cloudflare also cut Clef-flash to $0.038 per million input tokens and made Clef roughly 1.7 to 2 times faster by moving it onto SGLang. If you have ever talked yourself out of adding a classifier because the model call was too expensive, the arithmetic changed.

A decision model is a different machine from an LLM, and the pricing shows it

What shipped, and the context window trade-off

The family is now three models. Clef at $0.24 per million input tokens keeps a 64k context window. Clef-omni launched at $0.15 per million and takes text, images, audio in wav or mp3, and video in mp4 or webm. Clef-flash dropped from $0.09 to $0.038 per million, which puts it below Jev on price, but the hosted version's context window fell from 64k to 24k. Cloudflare is upfront about why: their usage data shows only 0.24% of requests exceeded 24k input tokens, so they traded a rarely used capability for a much better entry price. The weights on Hugging Face were trained for 256k context, so if you need the long window you can self-host, and Cloudflare suggests moving to Clef instead. That is a reasonable trade, but it is a real one, and 24k is not much room once you start attaching transcripts.

# Ask for named fields instead of prose. The model returns the schema, you do not parse it.
import json

def decision_request(state, questions, images=(), audio=(), videos=()):
    payload = {"model": "clef-omni", "state": state, "questions": {}}
    for name, instructions in questions.items():
        payload["questions"][name] = {"type": "noul", "instructions": instructions}
    if images:
        payload["images"] = list(images)
    if audio:
        payload["audio"] = list(audio)
    if videos:
        payload["videos"] = list(videos)
    return payload

payload = decision_request(
    state="Review the installation: a photo of the unit, an audio recording of it running, and a video of the fan.",
    questions={
        "label_visible": "Is the model and serial number label visible in the photo?",
        "sounds_normal": "Does the unit sound like it is running smoothly, without rattling or grinding?",
        "fan_running": "Is the fan running in the video?",
    },
    images=["data:image/jpeg;base64,<unit-photo>"],
    audio=["data:audio/mpeg;base64,<unit-running>"],
    videos=["data:video/mp4;base64,<fan-clip>"],
)

print(json.dumps(payload["questions"], indent=2))
print("modalities:", [k for k in ("images", "audio", "videos") if k in payload])

# Every key in the answer maps to a field you named. There is no parsing step left to get wrong.

Why a decision model behaves differently

Clef-omni is built on a Qwen3-Omni-30B-A3B-Instruct mixture-of-experts foundation, but Cloudflare keeps the comprehension backbone and discards the text-to-speech output components. That detail explains the performance profile. Because the model never generates output tokens, it skips transcribing and captioning entirely: audio and video frames map straight into one unified sequence, and the model scores every candidate value from internal embeddings using two-stage attention routing. A built-in lexical grammar keeps option semantics intact. Training follows the same recipe as Clef, freezing the Qwen3 backbone, training low-rank adapters, and applying label-smoothed cross-entropy loss with Brier score calibration. Brier calibration is the part worth noticing. A classifier that knows when it is unsure is far more useful than one that is confident and wrong.

# A decision model is cheap and fast. The engineering question is when to escalate anyway.
DECISION_LATENCY_MS = {"text": 130, "image": 150, "audio": 400, "video": 1500}
PRICE_PER_M_INPUT = {"clef-flash": 0.038, "clef": 0.24, "clef-omni": 0.15}

def plan(modalities, confidence, confidence_floor=0.72, llm_price_per_m=2.5):
    model = "clef-omni" if {"audio", "video"} & set(modalities) else "clef-flash"
    escalate = confidence < confidence_floor
    return {
        "model": model,
        "expected_latency_ms": sum(DECISION_LATENCY_MS[m] for m in modalities),
        "escalate_to_llm": escalate,
        "why": "confidence below floor" if escalate else "decision answer is good enough",
        "decision_price_per_m_input": PRICE_PER_M_INPUT[model],
        "llm_price_per_m_input": llm_price_per_m if escalate else 0.0,
    }

print(plan(["text"], confidence=0.94))
print(plan(["image", "audio"], confidence=0.61))
print(plan(["image", "audio", "video"], confidence=0.88))

# Escalation is not a fallback for failure. It is a routing decision you set on purpose.

At $0.038 per million tokens, classification stops being a cost centre

Latency and the benchmark table

Latencies are the strongest argument for this architecture. Text-only decisions return in about 130 milliseconds at the median, images in about 150, audio clips in a few hundred milliseconds, and a full 21-second video with sound is scored in roughly 1.5 seconds, all in one API call. The benchmark table is more mixed and it deserves an honest read. Clef-omni leads the family on BANKING77 macro-F1 at 94.8, CLINC150+OOS at 97.7, Amazon ESCI at 57.8 and PhishNChips trails. On BFCL case exact it scores 98.2 against Clef's 98.47 and Clef-flash's 98.76, all well above Jev's 95.75. But Clef-omni's Home appliances case exact is 69.3 where Clef-flash hits 97.73, and on When2Call it scores 63.3 against Jev's 80.97. Adding modalities did not make it uniformly better. It made it broader.

# Constrained answers are easy to validate. That is most of the value.
ALLOWED = {
    "label_visible": {"yes", "no", "unclear"},
    "sounds_normal": {"yes", "no", "unclear"},
    "fan_running": {"yes", "no", "unclear"},
}

def validate(answer, allowed=ALLOWED):
    problems = []
    for field, permitted in allowed.items():
        value = answer.get(field)
        if value is None:
            problems.append((field, "missing"))
        elif value not in permitted:
            problems.append((field, "out of schema: " + str(value)))
    return {"ok": not problems, "problems": problems}

print(validate({"label_visible": "yes", "sounds_normal": "no", "fan_running": "yes"}))
print(validate({"label_visible": "maybe", "sounds_normal": None, "fan_running": "yes"}))

# With an LLM you would be writing regexes against sentences. Here the check is a set membership test.

What Cloudflare uses it for internally

The internal examples are the most transferable part of the post. The public cloudflare-docs GitHub repository uses Clef to detect and close spam issues. EmDash, their content management system, moderates plugin libraries for phishing. The data loss prevention team scans for personally identifiable information such as government IDs. The threat intelligence team uses it to help detect malicious domains. None of these are glamorous, and that is the point: they are all the kind of job where a team previously had three bad options, which were manual review, a regex that ages badly, or standing up an ML team and gathering a corpus. Cloudflare's own framing is that decision capability moved into the model layer where anyone can call it. For most teams the honest description of the old workflow was the third option, meaning the job simply did not get done.

# Five million classifications a month turn a price per million into a business decision.
def annual_cost(requests_per_month, tokens_per_request, price_per_m,
                share_escalated=0.0, llm_price_per_m=2.5):
    steady = requests_per_month * tokens_per_request * (1 - share_escalated) * price_per_m
    escalated = requests_per_month * tokens_per_request * share_escalated * llm_price_per_m
    monthly = (steady + escalated) / 1_000_000
    return {"monthly_usd": round(monthly, 2), "annual_usd": round(monthly * 12, 2)}

print("all decision model:", annual_cost(5_000_000, 800, 0.038))
print("10% escalated:   ", annual_cost(5_000_000, 800, 0.038, share_escalated=0.10))
print("all LLM:         ", annual_cost(5_000_000, 800, 2.5))

# The escalation rate is the whole argument. Measure it before you quote a saving to anyone.

The whole family is open-weight, so self-hosting is a real option and a real comparison

How to adopt it without a rewrite

Clef is Jev-API compatible and available through AI Gateway, so Cloudflare's suggested migration is changing the model ID. Before you do that, decide your escalation policy, because the interesting design question in this whole family is not which model to call but what happens when the answer is not good enough. The pattern that works is straightforward: let the decision model handle the routine share, evaluate the returned confidence, and escalate to an LLM only for the tail. Set that threshold from measured field-level accuracy rather than a feeling, and track it per field, because a model that is reliable on label visibility and shaky on sound quality needs a threshold per field, not one per request.

# Track confidence against outcomes. A cheap model with honest confidence is a routing signal.
from collections import defaultdict

BUCKETS = [(0.0, 0.6), (0.6, 0.8), (0.8, 0.9), (0.9, 1.01)]

def calibration(samples, buckets=BUCKETS):
    rows = defaultdict(lambda: {"n": 0, "correct": 0})
    for confidence, is_correct in samples:
        for low, high in buckets:
            if low <= confidence < high:
                rows[(low, high)]["n"] += 1
                rows[(low, high)]["correct"] += int(is_correct)
                break
    out = []
    for bucket in buckets:
        row = rows[bucket]
        if row["n"]:
            out.append({
                "confidence_bucket": bucket,
                "n": row["n"],
                "accuracy": round(row["correct"] / row["n"], 3),
            })
    return out

samples = [(0.55, False), (0.62, True), (0.71, False), (0.85, True), (0.88, True), (0.95, True), (0.97, True)]
for row in calibration(samples):
    print(row)

# If the 0.6 to 0.8 bucket only lands 50% of the time, your escalation threshold is a guess.

The number that decides whether this is worth it

Cloudflare's price cut invites a calculation most teams have never run, because until recently it was not close. Five million classifications a month at roughly 800 tokens each costs about $152 a month on Clef-flash. Send all of that to an LLM at $2.50 per million input tokens and you are near $10,000 a month. The gap is wide enough that the question stops being about cost and becomes about accuracy on your specific task, which you can only answer by labelling a few hundred real examples and running both. Do the calibration pass before the business case. A cheap classifier with a measured escalation rate is a defensible proposal. A cheap classifier with a guessed escalation rate is a future incident.