CISA, NIST and the NSA Published Ten Practices for Agents That Go Off-Script

On October 16, 2026, NIST, the NSA's Super Intelligence Security Center and CISA published a document with a long title: Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation. It sets out ten voluntary core practices for organisations that train and evaluate highly capable agents — particularly pre-deployment, cutting-edge agents with reduced safeguards or other high-risk, non-public systems. One wording choice is worth flagging up front: the document says "super intelligence" or SI almost everywhere a general reader would expect "AI". Whatever you make of the label, the document contains one definition worth copying straight into your engineering docs.

Ten voluntary practices, and one useful definition

1. The Most Useful Thing in It: A Definition

CISA's summary is precise. An SI control incident occurs when one or more SI agents take uninstructed actions that conflict with or compromise the policies, controls, or security measures that scope the task and provide human oversight. Unpack that and it hands you three questions you have to be able to answer. First, what was the scope of the task — which means writing it down at dispatch time rather than reconstructing it afterwards. Second, which actions were uninstructed — which means logging every action together with the principal that authorised it. Third, what was the human oversight and did it fire — which means treating an approval checkpoint as an observable event rather than a policy sentence. If you cannot answer all three, you do not have a detection problem. You have a logging problem, and no policy document fixes that.

# 1) Start with the definition, because it decides what you log.
#    Published wording, condensed: an SI control incident occurs when
#    one or more agents take uninstructed actions that conflict with or
#    compromise the policies, controls, or security measures that scope
#    the task and provide human oversight.
#
#    Three things must therefore exist in your telemetry before you can
#    even detect one:
required_telemetry:
  - "the scoped task and its constraints, as written at dispatch time"
  - "every action taken, with the authorising principal"
  - "the human-oversight checkpoints and whether they fired"
# If you cannot answer "was this action instructed?", you do not have a
# detection problem, you have a logging problem, and no policy fixes that.

2. Five Areas in the Summary, Five Things You Can Build

The five areas CISA lists are: measures for limiting agent access to information, systems, tools and networks; monitoring for unexpected activity; preserving evidence and shutting down undesired behaviour; rebuilding from verified baselines; and reporting incidents and near-misses to relevant government agencies and affected parties. As principles they are abstract. As components they are concrete. The first is a closed capability set — not a prompt asking the model to be careful, but a tool list with explicit scope, an egress policy and a call ceiling per run. The second is detection rules, which requires defining "unexpected" or the alerting drowns everyone. The third is an evidence bundle plus a revocation path. The fourth is a resume gate. The fifth is an incident report template, and near-misses are the category everyone skips precisely because they are the cheapest to learn from. Notice that none of the five is a model problem; all five are system problems an ordinary platform team already knows how to build, just applied to an actor that plans instead of executing exactly what it was told.

// 2) Practice group one: limiting agent access to information, systems,
//    tools and networks. Implement it as a closed capability set, not as
//    a prompt that asks the model to be careful.
type Capability = {
  tool: string;
  scope: string[];        // explicit paths, hosts, tables
  egress: "none" | "allowlist";
  maxCallsPerRun: number;
};

const researchAgent: Capability[] = [
  { tool: "http.fetch", scope: ["docs.vendor.example"], egress: "allowlist", maxCallsPerRun: 40 },
  { tool: "fs.read", scope: ["/workspace/input"], egress: "none", maxCallsPerRun: 200 },
  // no shell, no fs.write, no arbitrary hosts
];

// The list is the policy. Adding a tool is a code review, not a sentence
// in a system prompt that a clever input can talk around.

3. Order Matters: Revoke the Credential Before You Kill the Process

Two things in the document deserve their own paragraph. One is preserving evidence. The other is testing controls and validating containment before resuming agent activity. The engineering consequence of the first is that evidence has to be written somewhere outside the agent's capability set, tamper-evident, hash-chained. If your agent has fs.write and the evidence directory sits inside its writable scope, that evidence is already untrustworthy at the moment of the incident. The second gets flattened into "fix it and restart", but the document asks for more: test the controls, validate containment against the same scenario, rebuild from a verified baseline and diff it. As for shutdown, the sharp version is this — killing a process while its tokens are still valid is theatre. An agent's reach comes from its credential, so the credential is what you take away first.

The practice list maps onto things you can actually build

# 3) Practice group two: monitoring for unexpected activity.
#    "Unexpected" needs a definition or the alert fires on everything.
unexpected_activity_rules:
  - id: act-outside-scope
    when: "tool.scope not contains action.target"
    severity: high
  - id: novel-tool-chain
    when: "tool_sequence not in observed_baseline_24h"
    severity: medium
  - id: egress-new-host
    when: "network.dst not in allowlist"
    severity: high
  - id: oversight-skipped
    when: "run.requires_approval and approvals.count == 0"
    severity: critical

baseline:
  window: "24h rolling"
  note: >-
    A brand new chain is not a breach. The point is that it is reviewed
    while it happens, not reconstructed from logs a week later.

4. The Only Two Fields That Carry Information in the Report

The fifth practice is reporting, to the relevant government agencies and affected parties, covering incidents and near-misses. Most templates grow into a dozen columns, but two fields carry nearly all the signal: what the agent was instructed to do, and what it actually did. Everything else is commentary on that gap. Which is why the first practice cannot be skipped — if the constraints at dispatch time were never recorded, those two fields cannot be filled in, and what you end up writing is a memoir rather than a report. One more note on scope: the CISA summary emphasises reporting as early as feasible but the document does not set a retention clock, so retention policy remains your decision. That silence is itself a decision handed to the operator.

# 4) Practice group three and four: preserve evidence, stop the behaviour,
#    then rebuild from a verified baseline. In that order.
evidence_bundle:
  manifest:
    run_id: "..."
    task: "as dispatched, verbatim"
    constraints: "as dispatched, verbatim"
    actions: "append-only log, hash-chained"
    approvals: "who, when, what they saw"
  integrity:
    algorithm: sha256
    store: "write-once, outside the agent's capability set"   # non-negotiable

shutdown:
  mechanism: "revoke credentials first, then stop the process"
  note: >-
    Killing the process while its tokens are still valid is theatre. The
    credential is the agent's reach; that is what you take away.

resume_gate:
  - "controls tested after the fix, not before it"
  - "containment validated against the same scenario"
  - "baseline rebuilt from a verified artefact, then diffed" 

5. Whether This Applies to You Depends on Which Layer You Sit On

If your company does not train or evaluate frontier models, you are not the target audience. The conceptual structure is still useful, for three reasons. Any team running agents will eventually meet the situation where the agent did something nobody asked for, and now there is a precise name and a response skeleton for it. The practices are aligned with ordinary cyber incident response, which means you can hang them on the IR process you already have instead of standing up a new one. And CISA's Super Intelligence topic page carries a sibling publication dated October 22, 2026, Managing AI Token Consumption for Operational Resilience, whose audience is every organisation using AI, because runaway token spend is another uninstructed action — just one you notice on the invoice. Read together, one governs behaviour and the other governs cost.

// 5) Practice group five: report. CISA asks for incidents and
//    near-misses, to the relevant agencies and to affected parties.
//    Near-misses are the part everyone skips, and they are the cheap ones.
type SiIncidentReport = {
  classification: "control-incident" | "near-miss";
  whatWasInstructed: string;
  whatActuallyHappened: string;
  controlsCompromised: string[];
  humanOversightAtTheTime: string;   // who was in the loop, or "none"
  containment: string;
  resumedAt: string | null;          // null until the resume gate passes
};

// One field carries most of the value: whatWasInstructed versus
// whatActuallyHappened. Everything else is commentary on that gap.
// Write it while it is happening; a report reconstructed later is a story.