Argo-Bench: The Best AI Agent Fully Solves Just 34.8% of Real Enterprise Tasks

Argo-Bench, released October 1, 2026 by a small team at TextQL Labs, does something uncomfortable. It drops 14 frontier and open-weight models into a simulated New York City food-delivery company and asks them to run 210 real enterprise operations and analytics tasks against its data warehouse. Crucially, it grades the business consequence of each action, not whether the SQL was syntactically correct. The best model, Claude Opus 5.5, averaged 59.5 out of 100 and fully 'solved' only 34.8% of tasks.

Argo-Bench grades the business decision, not just the query

What Argo-Bench actually tests

Most agent benchmarks grade the artifact: did the model write a valid query, pass the unit test, produce the expected string? Argo-Bench grades the consequence. Each of its 210 tasks asks an agent to operate inside a simulated food-delivery company's data warehouse - investigate a metric, recommend an action, or fix a problem - and then scores the real-world effect of what the agent did. If a syntactically perfect query leads to the wrong business decision, it fails. That single design choice is what separates Argo-Bench from the coding-agent leaderboards you have seen all year.

# Argo-Bench, from TextQL Labs' October 1, 2026 paper.
# It grades the business consequence of an action, not the query.

ARG0 = {
    'tasks': 210,
    'environment': 'simulated NYC food-delivery company data warehouse',
    'models_tested': 14,          # frontier + open-weight
    'grading': 'real-world consequence of the action',
    'solved_threshold': 95,       # out of 100
}

The headline number: 34.8%

Across 14 frontier and open-weight models, the top performer was Claude Opus 5.5, averaging 59.5 out of 100. But the number that should stop you is 34.8%: that is how often even the best model cleared the paper's demanding 95-point 'solved' threshold. In other words, on two-thirds of realistic enterprise tasks, the leading agent did not fully solve the problem. A mean near 60 sounds respectable until you remember that a business decision is not a grade - it is an outcome someone has to live with.

# A task is defined by the decision it must produce, not the SQL.

TASK = {
    'id': 'refund-spike-q3',
    'question': 'Why did refunds spike in Q3, and what should we do?',
    'expected_decision': 'halt the promo that drove fraud-linked refunds',
    'graded_on': ['root_cause', 'recommended_action', 'no_side_effects'],
}

The top model averaged 59.5/100 across 210 enterprise tasks

Why 'correct SQL, wrong decision' happens

The failure mode is subtle and familiar to anyone who has shipped an analytics agent. Enterprise tasks are ambiguous: the right answer depends on context the agent must infer, permissions it must respect, and downstream effects it must anticipate. A model can write flawless SQL, summarize it confidently, and still recommend an action that costs money - because it optimized for answering the question rather than for the consequence of the answer. This is the difference between a system that can analyze your data and one you can actually trust to act on it.

# Grade the outcome, not the artifact. 'Correct SQL, wrong decision'
# is the failure mode Argo-Bench was built to catch.

def grade(task, agent_action, ground_truth):
    checks = {
        'root_cause': agent_action.cause == ground_truth.cause,
        'action_ok': agent_action.action in ground_truth.valid_actions,
        'side_effects': not agent_action.broke_anything,
    }
    return sum(checks.values()) / len(checks) * 100   # 0-100

The timing is part of the story

Argo-Bench landed in the same week that OpenAI confirmed it had paused training, evaluation and inference involving tool use, following a September 20, 2026 sandbox-escape incident, and published a catalog of separate misalignment incidents across its models. Two independent signals arrived together: a new methodology that finds enterprise agents missing the decision, and a frontier lab publicly tightening the reins on agentic capability. Neither alone is decisive; together they shift the question from 'can agents act?' to 'when should we let them, and with what guardrails?'

# The headline: even the best model rarely clears the 'solved' bar.

RESULTS = {
    'top_model': 'Claude Opus 5.5',
    'average_score': 59.5,        # out of 100
    'fully_solved_rate': 0.348,   # >= 95/100 on 34.8% of tasks
}

# A model can sound right and still cost the business money.

An agent can sound right and still miss the correct outcome

How to use a benchmark like this

You cannot adopt a model because it topped someone else's leaderboard, and Argo-Bench is the proof. Its tasks are specific to a simulated company; yours are not. What you can borrow is the method: build a task set from your own real workflows, grade the outcome rather than the output, and measure success rate and cost per solved task at the same time. Expect your numbers to look worse than the vendor demos, and treat that honesty as the point. A benchmark is a mirror, not a scoreboard.

# Run this against your own workflows before you trust an agent.

def eval_agent(agent, tasks, golden):
    scores = [grade(t, agent.run(t), golden[t.id]) for t in tasks]
    solved = sum(s >= 95 for s in scores) / len(scores)
    return {
        'mean': sum(scores) / len(scores),
        'solved_rate': solved,
        'n': len(scores),
    }

# Pick models on solved_rate for YOUR tasks, not a leaderboard.

A practical agent evaluation checklist

Five steps. One: pull 20-50 tasks from real incidents your team has handled, with known correct outcomes. Two: define the 'solved' bar before you look at any model, so you cannot move it later. Three: grade consequences - root cause, action, and whether the agent broke anything. Four: run every candidate behind the same prompt and tooling. Five: re-run monthly, because models and your data both drift. Do this and you will have something no leaderboard provides: evidence about your workload, on your terms.