Amazon 告诉工程师「那些 AI 使用指标有严重缺陷」，然后继续用它们

Amazon 在 10 月 27 日公开了修订软件工程师岗位规范背后的那封备忘录。真正有意思的不是新词，而是中间两段自相矛盾的话，而且写的人清楚自己在自相矛盾。工程领导层（含杰出工程师和高级首席工程师）先把 token 使用量、AI 工具采用率列为团队被度量的输入指标，接着写这些指标「有严重缺陷」，因为它们与真正重要的事情脱节；警告把它们当目标会带来负面结果；紧接着又说，这些数字仍然与产出良好相关，谨慎使用是合适的。这不是草率，这是对当下每个工程组织都要面对的度量难题的准确描述。它值得当成一份规格文档来读，而不是一篇公关稿。

Amazon 给目标换了个名字：文艺复兴式开发者，把 AI 工具与人的判断混在一起。

一、备忘录到底按什么顺序说了什么

顺序很重要。第一步是历史：Amazon 的岗位规范一直刻意不去点名具体技术，因为公司跨的业务太多，这套做法挺过了云计算、微服务、无服务器二十年。第二步是断裂：资深工程师们辩论了「这次是否不同」，结论是不同，因为 AI 改变这项工作的速度和根本程度超过此前那些趋势。第三步是拒绝写下来：今天写下的任何「好的 AI 工程实践」定义，明年这个时候大概率已经过时，而过时的定义会把实践冻在琥珀里。第四步是转折，引用贝佐斯 2012 年那句话：别问未来五到十年什么会变，要问什么不会变。第五步是答案——对客户、业务和团队的影响——以及随之而来的标签：文艺复兴式开发者，把 AI 工具与人的判断、领域专长混在一起。

# Cost per merged change. The output-side denominator that makes a token
# number mean something. Run monthly, keep every run.
import json, subprocess
from collections import defaultdict

def gh(path):
    out = subprocess.run(
        ["gh", "api", path, "--paginate"],
        capture_output=True, text=True, check=True,
    ).stdout
    # gh --paginate concatenates JSON arrays; join them before parsing.
    return json.loads("[" + out.replace("][", ",") + "]")

def merged_prs(repo, since):
    raw = gh(f"repos/{repo}/pulls?state=closed&per_page=100&sort=updated&direction=desc")
    return [p for p in raw if p.get("merged_at") and p["merged_at"] >= since]

def tokens_by_session(log_path):
    spend = defaultdict(int)
    with open(log_path, encoding="utf-8") as fh:
        for line in fh:
            if not line.strip():
                continue
            rec = json.loads(line)
            spend[rec.get("pr_number")] += int(rec.get("total_tokens", 0))
    return spend

REPO, SINCE = "your-org/your-repo", "2026-10-01"
prs = merged_prs(REPO, SINCE)
spend = tokens_by_session("agent-sessions.jsonl")

priced = [(p["number"], spend.get(p["number"], 0)) for p in prs]
billable = [(n, t) for n, t in priced if t > 0]
total = sum(t for _, t in billable)

print(f"merged PRs: {len(prs)}   with agent spend: {len(billable)}")
print(f"tokens per merged PR: {total / max(len(billable), 1):,.0f}")

# Publish this next to the token total, never instead of it. A token number
# with no denominator is a number that can only go up.

二、每个工程经理都该抄走的两句话

备忘录不只是承认 token 计数和代码行数只是代理指标，它点名了失效模式：把 token 当目标是和「节俭」冲突的；用代码行数度量是和「化繁为简」冲突的。这两条都是 Amazon 的领导力原则，也就是说备忘录描述的是一个会跟明文价值观打架的指标，而不只是一个不够精确的指标。然后它做了一件更少见的事：告诉工程师，负面结果也是合法结论。如果你的工作没从当前 AI 工具里受益，那是一个发现，不是一次失败——写清楚原因，跟同伴分享，带给你的经理。一个能解释「AI 在哪里还帮不上我的工作」的工程师，恰恰展示了公司要求的判断力。把这段和同一封备忘录里推动采用 AI 的压力放在一起读，你看到的就是 2026 年真实的管理姿态：用，但要带证据。

# Two metrics with opposite failure modes. Both can be gamed; together they
# are much harder to game at once.
from datetime import datetime, timedelta

def defect_escape_rate(merged_prs, incidents, window_days=14):
    if not merged_prs:
        return None
    cutoff = max(p["merged_at"] for p in merged_prs)
    linked = {i["pr"] for i in incidents if i.get("pr")}
    in_window = [
        p for p in merged_prs
        if datetime.fromisoformat(p["merged_at"].replace("Z", "+00:00"))
        >= datetime.fromisoformat(cutoff.replace("Z", "+00:00")) - timedelta(days=window_days)
    ]
    if not in_window:
        return None
    escaped = sum(1 for p in in_window if p["number"] in linked)
    return escaped / len(in_window)

# How to read the pair:
#   cost per merged PR down  +  escape rate flat    -> real efficiency
#   cost per merged PR down  +  escape rate up      -> you are buying speed
#                                                      with quality; name it
#                                                      out loud before the
#                                                      customer does
#   cost per merged PR up    +  escape rate down    -> possibly a tougher
#                                                      problem, check scope
#
# Never report one of the two alone. A single number always becomes a target;
# that is what targets are for.

三、百分位那段是全文最可操作的一句

Amazon 建议看 AI 采用率的第 10 和第 90 百分位，而不是平均值。第 10 百分位是没采用起来的团队，指令是搞清原因，而不是发个提醒。第 90 百分位是重度使用 AI 的团队，指令是核对交付量是不是真的涨了。两个方向各自捕捉到平均值掩盖的真实失败：一个卡在 5% 采用的团队，和一个采用率 95% 但吞吐持平的团队，算出来的平均值跟两个 50% 的团队一样，而三种情形里只有一种健康。备忘录还把它和「离产品近」绑在一起：离产品和客户近的工程师与经理，最能从 agentic 开发里拿到收益，Amazon 说要扭转规模变大后个体贡献者离客户越来越远的趋势。这一切都不需要一个 token 看板。

# Lines of code is the metric the memo calls out by name. Here is what it
# actually counts, which is why it is easy to inflate and useless for sizing.
import subprocess

def loc_by_language(path="."):
    out = subprocess.run(
        ["git", "ls-files", "-z"], cwd=path,
        capture_output=True, text=True, check=True,
    ).stdout.split("\0")
    files = [f for f in out if f]

    counts = {}
    tracked = 0
    for f in files:
        try:
            with open(f, encoding="utf-8") as fh:
                n = sum(1 for _ in fh)
        except (UnicodeDecodeError, OSError):
            continue                      # binaries, locks, images
        ext = f.rsplit(".", 1)[-1] if "." in f else "(none)"
        counts[ext] = counts.get(ext, 0) + n
        tracked += n

    for ext, n in sorted(counts.items(), key=lambda kv: -kv[1])[:10]:
        print(f"{ext:<12} {n:>8,}")
    print(f"{'TOTAL':<12} {tracked:>8,}")
    return tracked

# Why the number is meaningless for performance review:
#   reformatting, generated clients, lockfiles and vendored dependencies all
#   inflate it; deleting 400 lines of duplicate logic shrinks it. The change
#   most worth celebrating in your repo this quarter may show up as -400.
#
# Size a change instead by files touched, hunks, and the review surface:
#   git diff --stat <base>...<head>
#   git diff --numstat <base>...<head> | wc -l

四、输入指标为什么还是活着，以及怎么办

token 计数和采用率之所以还留在会议室里，诚实的理由是延迟。产出指标——客户有没有真得到价值、这次改动有没有降本——要几个月才到，会被业务里其他所有事情混淆，而且无法归因到某个团队。输入指标明天就能到。Amazon 的折中是：谨慎使用、当成仪表而不是计分板，同时继续寻找更好的指标。工程侧对此的回应不该是拒绝输入指标，而是让操纵它变得昂贵。如果你的团队被按 token 消耗度量，就在旁边公布「每次合并改动的成本」。如果被按改动行数度量，就在旁边公布缺陷逃逸率。任何单一数字都会变成目标；一对失效方向相反的指标，想同时往两边作弊就难得多。

# Percentile review, not average review. This is the one operational habit
# from the memo that is worth copying this week.
import json, statistics

TEAMS = json.load(open("team_ai_adoption.json", encoding="utf-8"))
# each entry: {"team": "checkout", "adoption": 0.71, "throughput_delta": 0.14}

adoption = sorted(TEAMS, key=lambda t: t["adoption"])

def pct(rows, p):
    if not rows:
        return None
    idx = max(0, min(len(rows) - 1, round((len(rows) - 1) * p)))
    return rows[idx]

p10, p90 = pct(adoption, 0.10), pct(adoption, 0.90)
median = statistics.median(t["adoption"] for t in TEAMS)

print(f"median adoption: {median:.0%}")
print(f"p10: {p10['team']} at {p10['adoption']:.0%}  -> ask why, do not send a reminder")
print(f"p90: {p90['team']} at {p90['adoption']:.0%}  -> throughput delta {p90['throughput_delta']:+.0%}")

# The high-adoption team with a near-zero throughput delta is the more
# interesting conversation. Heavy tool use that does not move delivery is
# either a workflow problem or a measurement problem. Both need a human.

五、「文艺复兴式开发者」是一句工作量声明，不是恭维

这个标签很容易被读成恭维。应该去读它前后那段：备忘录要求工程师把 AI 工具与人的判断、领域专长混起来；要求领导者自己在生产工作上用过这些工具，并且能说出一个帮上忙的任务和一个没帮上忙的任务；要求经理在自己团队里把输入和产出连起来。这比它替换掉的那份岗位描述更宽，而编制没有变。唯一的缓冲是它对判断力的强调大过吞吐量：备忘录明说影响最大的领导者，是那些在「好奇求知」上加倍投入的人；在平衡交付与标准的同时追随好奇心的团队，从 AI 采用里得到的效果最正向。好奇心不是指标——这恰恰是它值得被奖励的原因。

# "A finding, not a failing": keep a ledger of tasks where AI did not help.
# Cheap to write, and in a year it is the most useful document you have.
import json
from datetime import date

LEDGER = "ai-does-not-help.jsonl"

def record(task, category, why, tried):
    row = {
        "date": date.today().isoformat(),
        "task": task,
        "category": category,       # e.g. "legacy build", "perf tuning"
        "why_ai_did_not_help": why,
        "what_was_tried": tried,
    }
    with open(LEDGER, "a", encoding="utf-8") as fh:
        fh.write(json.dumps(row, ensure_ascii=False) + "\n")

record(
    task="Make the 2019 Gradle build reproducible on a clean machine",
    category="legacy build",
    why="The failure is environment drift across three undocumented JDK patches; "
        "the model keeps proposing dependency bumps that are already applied",
    tried="Full-file context, then targeted excerpts, then a written repro script",
)

# Review it quarterly, in the same meeting where you review adoption.
# Two outcomes: either the tooling gap closes (and you delete rows), or the
# category becomes an argument for buying different tooling. Both are wins,
# and neither happens if nobody writes the ledger down.

六、一套能挺过这个矛盾的度量栈

搭三层，并且诚实地给每层贴上标签。第一层是「明天就能回答」的仪表：模型调用次数、token、每次改动的成本、Agent 起草占比、每百行评审评论数。没有人为这些被考核，它们只用来解释异常。第二层是抗操纵的成对指标：每次合并改动的成本配缺陷逃逸率，周期时间配回滚率，评审深度配合并速度。第三层是人负责的产出：这次改动有没有把客户关心的那个数字推动。然后按季度做百分位复盘，不要按周，因为采用行为变化很慢，用周视角看慢信号产出的是恐慌而不是洞察。最后，把 AI 没帮上忙的任务写下来。那份文档一年后的价值，比你这一季搭的任何一个看板都高。