Decisions API 正式开放：当「有界答案」比聊天补全更划算

这个端点到底返回什么

请求发到 POST /v1/decisions，模型写 gpt-6-luna——正式开放时唯一支持的模型。结构很简单：一份共享 input（文本、图片或 JSON），然后一个 questions 数组。每个问题都有类型。predicate 回答「这个条件是否成立」，给一个 0 到 1 的概率。choice 从你给的选项里选一个，同时返回整个分布和一个独立的 confidence 字段。score 按有序档位打分，返回档位索引的概率加权平均值——所以严重度分数可以落在两个档位之间，而不是被硬吸到某个整数上。

还有一个 refusal 类型，你怎么处理它，就是「谨慎集成」和「线上事故」的分界线。refusal 不是一个分类。如果你的路由器把所有非空答案都当成路由结果，那 refusal 就会变成默认分支碰巧是什么。要像下面的例子那样显式处理，送进人工队列。

和 API 其他部分的边界值得说清楚，因为两个方向都有人搞错。需要生成文字、抽取字段、或者输出符合你自己 JSON Schema 的对象，那是 Responses + Structured Outputs。需要模型选工具并输出参数，那是 function calling。Decisions 是第三种情况：一个被问了上千次的有界问题，延迟和成本占主导。

# POST /v1/decisions, model gpt-6-luna (the only supported model at GA).
# The shape is deliberately boring: shared input, then questions with bounded answers.

curl https://api.openai.com/v1/decisions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6-luna",
    "input": "I was charged twice for my order and nobody replied to my email.",
    "questions": [{
      "type": "choice",
      "name": "department",
      "instructions": "Which department should handle this complaint?",
      "choices": [
        {"value": "billing", "description": "Payments, invoices, and refunds."},
        {"value": "technical", "description": "Problems using the product."},
        {"value": "shipping", "description": "Delivery and tracking."},
        {"value": "other", "description": "Requests outside these categories."}
      ]
    }]
  }'

# Python equivalent. Note what you get back: an answers array whose entries are
# typed. A choice answer carries the chosen value, the distribution, and a
# separate confidence number. Handle the refusal type explicitly or your router
# will silently treat a refusal as a category.
from openai import OpenAI

client = OpenAI()
decision = client.decisions.create(
    model="gpt-6-luna",
    input="I was charged twice for my order and nobody replied to my email.",
    questions=[{
        "type": "choice",
        "name": "department",
        "instructions": "Which department should handle this complaint?",
        "choices": [
            {"value": "billing", "description": "Payments, invoices, and refunds."},
            {"value": "technical", "description": "Problems using the product."},
            {"value": "shipping", "description": "Delivery and tracking."},
            {"value": "other", "description": "Requests outside these categories."},
        ],
    }],
)

answer = decision.answers[0]
if answer.type == "refusal":
    route = "human_queue"
elif answer.type == "choice":
    route = answer.choice if answer.confidence > 0.6 else "human_queue"
    print(f"route={route} confidence={answer.confidence:.3f}")

先算账，再谈架构

计费只算输入：$0.10 / 百万输入 token，缓存输入 $0.01 / 百万，输出不出钱。最后这条比标题数字更重要。聊天补全里，你要为逐 token 生成的答案付钱，一段啰嗦的推理可能比触发它的提示词还贵；这里根本没有「生成的答案」可付。

把价格从「便宜」变成「离谱便宜」的是缓存。API 默认在共享 input 末尾放一个隐式缓存断点，可复用的前缀至少要 1,024 token。questions 里的内容（包括问题专属图片）在这个前缀之外，所以一份长长的政策文档可以一直很便宜，而每个请求都变的部分不享受折扣。你可以用 prompt_cache_key 把复用分桶，也可以用显式模式自己标断点，但大多数团队第一步该量的是默认行为。

别信自己脑子里的模型，去看真实数字：usage.input_tokens_details.cached_tokens 和 cache_write_tokens 都会返回，而且它们已经包含在 usage.input_tokens 里了。把这两个再加进总数，是让看板莫名其妙差 20% 的经典 bug。

# Price the workload before you build the retry logic. Decisions bills input only:
# $0.10 per million input tokens, $0.01 per million cached input tokens, and no
# output-token charge. That last part changes the arithmetic of small, frequent calls.

def monthly_cost(calls_per_day, tokens_per_call, cache_hit_rate, cached_tokens=1024):
    billed_prompt = tokens_per_call - (cached_tokens * cache_hit_rate)
    billed_prompt = max(billed_prompt, 0)
    uncached = billed_prompt * (1 - cache_hit_rate)
    cached = billed_prompt * cache_hit_rate
    per_call = (uncached * 0.10 + cached * 0.01) / 1_000_000
    return round(calls_per_day * 30 * per_call, 2)

# A support router at 200k calls/day with a 2,000-token prompt costs about $1.1k
# a month with no cache, and about $90 a month if the shared prefix is cached and
# your prompt is mostly shared. Same token budget, different order of magnitude.
print("no cache   ", monthly_cost(200_000, 2_000, 0.0))
print("half cached", monthly_cost(200_000, 2_000, 0.5))
print("all cached ", monthly_cost(200_000, 2_000, 0.95))

# Caching has a floor worth remembering: the reusable prefix must be at least
# 1,024 tokens, and question content never counts toward that threshold.

决定要不要上线的那个评估

切换的商业理由是延迟和价格，不切换的工程理由是不吭声的不一致。一个便宜 90%、错 3% 的路由器不是更便宜，它是一个多了几步的支持队列。所以下面这个脚本做的是很不性感的事：把同一批有标注的样本喂给两条路径，输出一致率、拒答率和平均延迟。

按这个顺序读结果。一致率说明便宜路径是不是替代品。拒答率说明你还有多少活必须交给人，这是要跟 token 一起写进同一张表里的成本。延迟说明用户能感知的收益到底是真的，还是被你自己的网络跳数吃掉了。厂商的基准是别人流量的平均值，只有你自己的标注集描述的是你。

然后手读二十条不一致。它们会聚类。我们见过的通常只有两种形状：一种是输入本身就同时属于两个类别；另一种是两个同事对某个类别描述的理解不同。前者需要阈值，后者需要你重写选项描述——五分钟就能改完，而在聊天模型上无论怎么调提示词都暴露不出来。

# An evaluation harness that answers the only question that matters: does the
# cheap path disagree with the expensive path in a way you can live with?

import json, time
from openai import OpenAI

client = OpenAI()

def as_decisions(text, choices):
    d = client.decisions.create(
        model="gpt-6-luna",
        input=text,
        questions=[{"type": "choice", "name": "label",
                    "instructions": "Pick the correct label.",
                    "choices": [{"value": c} for c in choices]}],
    )
    a = d.answers[0]
    return None if a.type == "refusal" else a.choice

def as_responses(text, choices):
    r = client.responses.create(
        model="gpt-6-luna",
        input=f"Reply with exactly one of: {', '.join(choices)}\n\n{text}",
    )
    return r.output_text.strip().lower()

labelled = [json.loads(line) for line in open("labelled.jsonl")]
agree = refusals = 0
t_dec = t_res = 0.0

for row in labelled:
    t0 = time.perf_counter()
    d = as_decisions(row["text"], row["choices"])
    t_dec += time.perf_counter() - t0

    t0 = time.perf_counter()
    r = as_responses(row["text"], row["choices"])
    t_res += time.perf_counter() - t0

    refusals += d is None
    agree += (d == r)

n = len(labelled)
print(f"agreement {agree/n:.1%}  refusals {refusals/n:.1%}")
print(f"p50-ish mean latency decisions {t_dec/n*1000:.0f} ms vs responses {t_res/n*1000:.0f} ms")

# Disagreement is not automatically failure. Print the rows where they differ and
# read twenty of them by hand; that is where the threshold you need actually lives.

切流量之前，先用同一批标注样本把老路径和新路径都跑一遍。

进热路径，但别把护栏丢了

站得住的模式是两道闸，不是一道。第一道问路由问题，第二道问「这个请求到底允不允许被自动路由」——比如一个 predicate：「是否明显属于标准类别，且不涉及法律、安全或账号注销」。只有当 predicate 概率和 choice 置信度都过了你的线，答案才允许真的执行。

这就是 Decisions 在智能体循环里真正值钱的地方。公开的客户数字全都关于「把一个决定塞进过去慢到不敢放模型的位置」：Box 在 30 个保险凭证案例上路由选择快约 5 倍、准确率与 Responses API 持平；Datadog 在执行智能体生成的 shell 命令前先做风险打分，p95 快 40 倍；Ramp 做 AI 成本归因快 4 倍、便宜 30%；Sierra 在实时对话编排里跑到 280ms p95 以内，单次调用成本低约 90%。

把这些当存在性证明，不要当基准测试。每一个都跑在那些客户自己的数据、自己的阈值上。能通用的是模式：循环内侧放便宜的、有界的判断，当有界答案不够自信时，留一个显式的出口通向更大的模型或者人。

# Wire it into the hot path with two rules: a confidence floor, and a predicate
# that decides whether the cheap answer is even allowed to act.

def should_auto_route(decision):
    gate = client.decisions.create(
        model="gpt-6-luna",
        input=decision["text"],
        questions=[{"type": "predicate", "name": "safe_to_auto",
                    "instructions": ("Is this request clearly within a standard "
                                     "category, with no legal, safety, or account "
                                     "deletion implications?")}],
    )
    p = gate.answers[0].probability
    return p >= 0.9 and decision["confidence"] >= 0.7

# The important line is the one you do not write: there is no fallback that
# quietly defaults to a category. Low probability means a human or a bigger model,
# not a coin flip.

def route(ticket):
    d = client.decisions.create(
        model="gpt-6-luna",
        input=ticket["text"],
        questions=[{"type": "choice", "name": "queue",
                    "instructions": "Which queue owns this ticket?",
                    "choices": [{"value": "billing"}, {"value": "technical"},
                                {"value": "shipping"}, {"value": "other"}]}],
    )
    a = d.answers[0]
    if a.type == "refusal":
        return "human_queue"
    entry = {"text": ticket["text"], "choice": a.choice, "confidence": a.confidence}
    return a.choice if should_auto_route(entry) else "human_review"

一份能在评审里站住的落地清单

上线前四件事，都很便宜。第一，端点体检：CI 里跑一个 predicate 调用，这样凭证或模型名变了是构建失败，而不是路由器失败。第二，验证缓存：三个完全相同的请求、共享前缀超过 1,024 token，每次打印 cached_tokens。如果这个数一直是 0，说明你的前缀并不完全一致，你的成本模型就是虚构的。

第三，把预算护栏写成「每次调用多少 token」，而不是一个没人看的月度总数。第四，准备一条命令就能执行的回滚。有一步回滚的试点叫实验；没有回滚路径的迁移叫下注，而下注应该出现在战略文档里。

有一个风险要一直摆在明面上：今天只有 gpt-6-luna 一个模型可用，意味着你的延迟和质量绑在单一模型的路线图上。这件事要像评估任何厂商锁定一样去定价——把「路由问题」和「答案的结构」留在你自己的配置里，让换实现变成改配置而不是重写。模型端点变得比路由逻辑快得多。

#!/usr/bin/env bash
# Pre-flight checks. Attach the output to the pull request instead of saying
# "it felt faster in the playground".
set -euo pipefail

echo "== endpoint sanity =="
test -n "${OPENAI_API_KEY:-}" || { echo "no key"; exit 1; }
curl -s https://api.openai.com/v1/decisions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-6-luna","input":"ping","questions":[{"type":"predicate","name":"ok","instructions":"Is this input short?"}]}' \
  | python -c 'import json,sys; d=json.load(sys.stdin); print(d["answers"][0])'

echo "== cache actually hitting? =="
python - <<'PY'
from openai import OpenAI
c = OpenAI()
shared = "policy text " * 400          # >1,024 tokens, identical every request
for i in range(3):
    r = c.decisions.create(
        model="gpt-6-luna",
        input=shared,
        questions=[{"type": "predicate", "name": "p", "instructions": "Is this policy text?"}],
    )
    u = r.usage.input_tokens_details
    print(i, "in", r.usage.input_tokens, "cached", u.cached_tokens, "written", u.cache_write_tokens)
PY

echo "== budget guard =="
python - <<'PY'
budget = 250          # currency units per month
calls_per_month = 200_000
cap_per_call = budget / calls_per_month
print(f"cap per call: {cap_per_call:.6f} -> keep prompts under ~{cap_per_call/0.10*1_000_000:,.0f} tokens")
PY

echo "== rollback =="
test -n "${LEGACY_ROUTE_COMMAND:-}" || echo "no documented fallback -> stop here"

一条命令能回滚的试点叫实验；没有回滚路径的迁移叫下注。