Copilot 用量指标开始按「客户端」记账：OpenCode、Codex、Zed，以及你的看板一直漏掉的那部分

totals_by_client 补的是哪个洞

旧报告里只有 totals_by_ide。在编辑器里跑的会自报家门，不在编辑器里跑的完全看不见。过去一年，越来越多模型流量转移到了「以 Copilot 为供应商、但不在 IDE 里」的工具上——终端里的智能体、独立 App、Agent Client Protocol 集成——这些工具不发编辑器遥测。流量是真的，账单是真的，但明细里没有。

服务端计数把这个洞堵上了。每个 client 条目带 request_count、session_count，以及包含 prompt tokens、output tokens 和平均每请求 token 的 token_usage，外加按模型的 totals_by_model 拆分。未识别的客户端会归到 other，这让 other 那一行变成一个有意义的指标：如果它在涨，说明有你的资产清单不知道的工具已经进了生产。

有一个区别要记住，因为评审里最容易混。totals_by_client 统计的是「任何使用 Copilot 的工具产生的模型流量」，也就是请求和 token。而 agent app activity 统计的是 Claude、Codex 这类 agent app 在 GitHub 上启动的 job。一个团队可以启动很多 agent job、却几乎不产生经 Copilot 的模型流量，反过来也一样。两个都是真的，回答的是不同问题。

# Both changes land in the report JSON you already download, so start by reading the
# shape instead of guessing it. A 28-day aggregated report carries day_totals, and
# totals_by_client appears inside each entry, not only at the top level.

import json
from collections import defaultdict

report = json.load(open("report-28d.json"))

per_client = defaultdict(lambda: {"requests": 0, "sessions": 0,
                                  "prompt_tokens": 0, "output_tokens": 0})

for day in report.get("day_totals", []):
    for row in day.get("totals_by_client", []):
        c = per_client[row["client"]]
        c["requests"] += row.get("request_count", 0)
        c["sessions"] += row.get("session_count", 0)
        usage = row.get("token_usage") or {}
        c["prompt_tokens"] += usage.get("prompt_tokens", 0)
        c["output_tokens"] += usage.get("output_tokens", 0)

for client, row in sorted(per_client.items(), key=lambda kv: -kv[1]["requests"]):
    print(f"{client:14} req={row['requests']:>7} "
          f"sess={row['sessions']:>6} out_tok={row['output_tokens']:>10}")

# Unrecognised clients are grouped under "other", so an "other" line that keeps
# growing is a to-do item, not a rounding error.

有人问之前就先把新旧数字对上

这种变更之后的第一次报告，永远会引发一轮争论，因为差额看起来像用量暴涨。提前把解释准备好：按客户端统计的数字本来就比只按 IDE 统计的大，而这个差额的大小，正好告诉你组织里有多少模型用量此前是没有被记账的。

权限也会在这里咬人。这些报告开放给企业所有者、账单管理员、组织所有者，以及被授予 View Copilot Metrics 权限的自定义组织/企业角色，并且必须启用 Copilot usage metrics 策略。如果拥有预算的人看不到报告，这份报告就没在起作用。

还有一条要写进文档而不是留在脑子里：这些指标来自客户端遥测，可能少报，比如客户端版本过旧或遥测被阻断。一个悄悄少报的看板，比一个承认自己会少报的看板更糟——因为财务同事做的第一件事，就是手抽一个样本把数加起来，然后发现差。

# Reconcile before you report. Client-level accounting was added to close a gap --
# tools that run outside the editor never sent editor telemetry, so their traffic
# was absent from totals_by_ide. That means old and new numbers legitimately differ,
# and a jump is not automatically a regression or a spike in usage.

totals_by_ide = 0
totals_by_client = 0

for day in report.get("day_totals", []):
    totals_by_ide += sum(r.get("request_count", 0) for r in day.get("totals_by_ide", []))
    totals_by_client += sum(r.get("request_count", 0) for r in day.get("totals_by_client", []))

delta = totals_by_client - totals_by_ide
print("ide-only:", totals_by_ide, "client-wide:", totals_by_client, "delta:", delta)
print("delta share:", f"{delta / max(totals_by_client, 1):.1%}")

# Attach this to the first report after the change so nobody has to reconstruct
# why the line moved. If the delta is large, most of your organisation's model
# traffic was always coming from somewhere your dashboard could not see.

# Also distinguish model traffic from job activity: totals_by_client counts
# requests and tokens from any tool using Copilot, while agent app activity counts
# jobs started by agent apps such as Claude and Codex on GitHub. Different question.

真正有意思的字段是 model_selection_mode

totals_by_model 的每个条目都带 model_selection_mode，说明这个模型是被直接指定（user_selected）、被自动路由选的（auto），还是被产品选的（product_selected）。未上报的模式显示为 other，无法归一的值显示为 unknown。GitHub 特别说明，code review 和 cloud agent 报 product_selected，因为是产品在选模型。

这一个字段把两种完全不同的故事分开。user_selected 占比高，说明花销跟着你指南里写的那套选择走，改模型政策就会改账单。product_selected 占比高，说明是默认值和自动化功能在塑造账单，这时候跟工程师说「选便宜点的模型」几乎什么都不会变。

实际用法很无聊但有效：报占比，而不只报总数。如果 product_selected 占请求量的 40%，那场关于成本的对话应该发生在负责自动化功能的人那里，而不是发生在正在使用它们的开发者那里。

# model_selection_mode is the field that changes decisions. Each totals_by_model
# entry says whether the model was requested directly (user_selected), picked by
# auto, or chosen by the product (product_selected). Unreported modes appear as
# other, and values that cannot be normalised appear as unknown -- keep both
# visible instead of folding them into a total.

by_mode = defaultdict(int)
by_model = defaultdict(int)

for day in report.get("day_totals", []):
    for row in day.get("totals_by_client", []):
        for m in row.get("totals_by_model", []):
            n = m.get("request_count", 0)
            by_mode[m.get("model_selection_mode", "unknown")] += n
            by_model[m.get("model", "unknown")] += n

print("by selection mode:", dict(by_mode))
print("by model:", dict(sorted(by_model.items(), key=lambda kv: -kv[1])[:10]))

# If product_selected is a large share, your bill is being shaped by defaults and
# code review / cloud agent traffic, not by the model picker your guidelines describe.
# That is the sentence to bring to the next budget review.

要报「产品代选模型」的占比，而不只是总花销。

「有名字」带来了一点治理工作

真名比一个大口袋更有用，也更危险。名字来自客户端遥测，会被去掉首尾空白，长度限制 128 字符；看起来像文件路径的名字会被刻意保留在 other 或 custom 下面。上报名字的路径有限：Copilot CLI 的交互会话与 prompt 模式、Copilot 应用、ACP 集成（如 JetBrains）、以及 Copilot SDK，且需要 CLI 1.0.97 以上或 SDK v1.0.21 以上；IDE 内置聊天遥测不上报名字。

这带来两个义务。第一，25 项上限意味着长尾被截断，当组织里有上百个 skill 时，请把这份清单当抽样看。第二，名字是人写的文本：总会有人拿客户名、工单号、或者自己的名字来命名。报告出组织之前要审一遍名字，并且把这个动作排进流程，而不是靠某个英雄临时抓。

好处也是真的。你上线一个内部 MCP server，如果采用率一周只有五次交互，你现在几天内就知道，而不是一个季度后；推进落地的对话从证据开始，而不是从感觉开始。

# Names are now reported, which creates a governance job nobody asked for:
# names come from client telemetry, are trimmed, and are limited to 128 characters.
# Names that look like file paths stay grouped under other or custom.

import re

def review_names(report, max_show=25):
    issues = {"path_like": [], "too_long": [], "other_bucket": 0}
    for day in report.get("day_totals", []):
        for key in ("totals_by_skill", "totals_by_custom_agent", "totals_by_mcp",
                    "totals_by_slash_cmd", "totals_by_plugin"):
            rows = day.get(key, [])
            for row in rows:
                name = (row.get("name") or row.get("value") or "").strip()
                if name in ("other", "custom"):
                    issues["other_bucket"] += sum(row.get("interaction_count", 0) for r in [row])
                    continue
                if re.match(r"^[~./]", name):
                    issues["path_like"].append(name)
                if len(name) > 100:
                    issues["too_long"].append(name)
            if len(rows) > max_show:
                issues.setdefault("truncated", []).append((key, len(rows)))
    return issues

# Each list now carries up to 25 entries per record, up from five, with the rest
# rolled into a trailing "other". If your long tail is bigger than that, sample it
# deliberately rather than assuming the top 25 is the whole story.
print(review_names(report))

# Names are org-authored text. Review them before sharing a report outside the org,
# and check that nothing personal ended up in a skill or MCP server name.

一份能过审计的月度复盘

把报告压成三个数字和两条注意事项。三个数字：按客户端的总请求量、按 model_selection_mode 划分的占比、以及落在 other 里未上报的自定义活动量。两条注意事项：客户端遥测可能少报；IDE 内置聊天不上报名字。

然后把对账自动化，让它在有人发问之前就跑完。一个每月打印「只按 IDE 统计」与「按客户端统计」差额、并标注这个差额从哪天引入的脚本，能终结那种反复出现的「用量是不是暴涨了」的争论。在安全团队问「到底哪些工具在打模型」的那天，它也给你一个站得住的答案。

最后一个值得在数据还新的时候养成的习惯：命名时想一下六个月后这份报告能不能读。一个叫 pr-helper-2-final 的自定义 agent 是未来的调查工作；一个叫 infra-terraform-review 的自定义 agent 是你能在评审里的一条明细分项。

#!/usr/bin/env bash
# Monthly review, as a reproducible script instead of a slide someone rebuilt by hand.
set -euo pipefail

OUT="reports/copilot-$(date +%Y-%m)"
mkdir -p "$OUT"

echo "== pull reports =="
# Use the usage metrics API endpoints your org already calls; the 1-day and 28-day
# versions both expose totals_by_client, and the policy must be enabled.
echo "enterprise 28d -> $OUT/ent-28d.json"
echo "org 28d        -> $OUT/org-28d.json"

echo "== coverage sanity =="
python - <<'PY'
import json, glob
for path in glob.glob("reports/copilot-*/*-28d.json"):
    d = json.load(open(path))
    days = d.get("day_totals", [])
    with_client = sum(1 for x in days if x.get("totals_by_client"))
    print(f"{path}: {len(days)} days, {with_client} with client data")
PY

echo "== name hygiene =="
python - <<'PY'
print("check: <=128 chars, no path-like names, count entries beyond the 25 cap")
PY

echo "== caveat to print in the report itself =="
echo "client telemetry can undercount (outdated clients, blocked telemetry);"
echo "IDE built-in chat does not report names, only SDK-based agent experiences do."

自定义项名字是人写的文本，报告出组织前要审一遍。