Bionic Shell Command Safety 2026: How LM Studio Built a Judge for AI Commands

A command as ordinary as git diff can become a problem for an AI coding agent once a variable gets involved. Take git diff $base: if $base contains a commit hash, the command behaves as expected. But if it resolves to --output=/some/file, Git can write the result to the filesystem. LM Studio built Auto Review to catch cases like this before Bionic runs the command, turning to another language model only when it can't determine whether it's safe. In a blog post published Thursday, LM Studio said this first layer cleared as many as 82% of Bionic's commands without another model call, although the author described the figure as anecdotal rather than a benchmark.

1. Why String Matching Is Not Enough

Searching for dangerous strings only gets you so far, because shell commands can change depending on the variables, redirects and other commands inside them. The danger doesn't live in the literal shape of the string; it lives in the value. $base can be a commit hash (safe) or start with --output= (dangerous) — the same string template, two completely different behaviors. Bionic's Shell Judge gets around that by looking at the structure of the command instead of the strings: it turns shell commands into abstract syntax trees (ASTs), following variables and nested commands to see what they could affect.

// The attack that motivates the whole system: a command as
// ordinary as git diff becomes a problem once a variable gets
// involved.
//   git diff $base
// If $base contains a commit hash, the command behaves as
// expected. But if it resolves to --output=/some/file,
// Git writes the result to the filesystem. String matching
// cannot catch this — the danger depends on the VALUE.
const cmd = "git diff " + userControlled; // ???
// $base = "HEAD~1"      -> safe diff
// $base = "--output=/tmp/evil" -> writes a file!

2. Parsing Structure: ASTs and Capability Analysis

The Shell Judge uses the mvdan/sh parser for Bash, Zsh and SH, while PowerShell uses its own AST support. The Judge then works out what LM Studio calls the command's "capabilities" — essentially what the command could read or change. It can also follow a value from one command to the next: if an agent uses git merge-base to find a common ancestor commit and feeds the result into git diff, the Judge keeps track of that result when it evaluates the second command. If there are several possible values, Bionic follows up to 1,000 of them before it stops trying to account for every possibility.

// Parsing structure, not strings: turn the shell command
// into an abstract syntax tree, then compute what the
// command could read or change — its "capabilities".
// mvdan/sh parses Bash, Zsh and SH; PowerShell uses its
// own AST support.
import {{ syntax }} from "mvdan-sh";

type Capabilities = { reads: string[]; writes: string[] };

function capabilities(ast: syntax.Node): Capabilities {
  const caps: Capabilities = {{ reads: [], writes: [] }};
  walk(ast, (node) => {{
    if (isRedirection(node)) {{
      caps.writes.push(node.target); // > file, >> file
    }}
    if (isAssignment(node)) {{
      caps.writes.push(node.name);
    }}
  }});
  return caps;
}
// git diff $base with $base=--output=/tmp/x
// -> writes: ["/tmp/x"]  <- caught before execution

3. 11,651 Tests for CLI Quirks

Then there's the fact that command-line tools have their own rules. ls -la, for example, treats -la as multiple flags bundled together, while LM Studio points out that TypeScript's tsc -vh doesn't behave the same as running it with -v and -h. So parsing the shell syntax only gets Bionic partway there. The Shell Judge also has to understand how the individual tool will interpret what comes after the command. That helps explain why LM Studio has already built 11,651 test cases, covering malformed commands and the quirks in how individual tools handle their arguments.

// Following values across commands: if an agent uses
// git merge-base to find a common ancestor and feeds the
// result into git diff, the judge keeps track of that
// value when it evaluates the second command. Up to 1,000
// possible values before it stops accounting for everything.
function trackValues(ast: syntax.Node): Map<string, string[]> {
  const values = new Map();
  walk(ast, (node) => {{
    if (isCommand(node) && node.name === "git") {{
      const args = node.args;
      if (args[0] === "merge-base") {{
        // record that $var now may hold a commit hash OR
        // anything an earlier step injected into the chain
        values.set(args[2], inferPossibleValues(node));
      }}
    }}
  }});
  return values;
}
// Each possible value is evaluated; if ANY of them is
// dangerous, the command is not cleared.

4. When the Reviewer Gets Swayed: Agreeing with the Defendant

LM Studio found that just asking the reviewer whether a command should run didn't work well, because the model sometimes approved risky actions because they seemed necessary to complete the user's request. Anything the Shell Judge can't clear goes to the Shell Reviewer, a separate AI agent that evaluates the command in the context of the conversation. The reviewer now rates each command for risk, authorization and correctness without knowing what scores are needed to pass. That "unknown pass bar" design is what avoids the failure mode of the judge starting to agree with the defendant.

5. Trust Assumptions Remain Open

The Shell Reviewer needs enough of the conversation to know whether the user authorized a command, which creates another opening for prompt injection. LM Studio excludes tool results — instructions hidden in a webpage or file aren't passed directly to the reviewer — but it still sees assistant messages. If Bionic has already been compromised, those messages could carry malicious instructions with them. The Shell Judge has its own blind spots: it assumes executables such as git haven't been compromised and doesn't account for malicious configuration that could change how a command behaves. A recent npm supply chain attack showed how legitimate-looking provenance signals can be used to hide malicious payloads.

// The reviewer: when the judge can't decide, a separate AI
// agent evaluates the command in context. Just asking "should
// this run?" didn't work — the model approved risky actions
// because they seemed necessary. So the reviewer rates risk,
// authorization, and correctness WITHOUT knowing what scores
// are needed to pass.
type Verdict = { risk: 1|2|3|4|5; authorized: boolean; correct: boolean };

async function review(cmd: string, conversation: Context): Promise<Decision> {
  const v = await reviewerAgent.rate({{ cmd, conversation }});
  // The reviewer does not know the pass threshold.
  if (v.risk >= 4) return "block";
  if (!v.authorized) return "ask";
  if (!v.correct) return "warn";
  return "run";
}
// Rating without a known bar avoids the "agrees with the
// defendant" failure mode.

6. Why This Matters More in 2026

Those limits become more important as coding agents get more freedom to act. Google's Gemini coding agent, for example, recently expanded beyond its IDE boundaries, giving agents more opportunities to run commands and make changes without a developer doing each step manually. Every additional degree of freedom makes the command layer the last line of defense. The secure posture is defense in depth: AST capability analysis catches structural problems, value tracking catches injection, a reviewer agent adjudicates ambiguity in context, tool results stay out of the reviewer's context, and 11,651 test cases cover tool quirks. No single layer is perfect. Stacked together, they are the realistic baseline for agent command safety in 2026.

// Trust assumptions remain: the judge assumes executables
// like git haven't been compromised and doesn't account for
// malicious configuration. The reviewer sees assistant
// messages — so if the agent is already compromised, those
// messages can carry malicious instructions. Defense in depth:
async function safeRun(cmd: string, ctx: Context) {
  const ast = parse(cmd);
  const caps = capabilities(ast);
  if (isDangerous(caps, await trackValues(ast))) return "block";
  if (!await judgeIsConfident(ast)) {{
    return await review(cmd, sanitize(ctx)); // tool results excluded
  }}
  return "run";
}
// 11,651 test cases cover malformed commands and the quirks
// of how individual tools handle their arguments.