AMD Helion: A Rack Blueprint That Puts Agent Execution on CPUs

On October 12, 2026, AMD introduced the AMD Helion agentic AI rack: an open blueprint for agentic infrastructure, built on an Open Compute Project spec-design and powered by AMD compute. The headline configuration is a CPU-only rack of 6th Gen AMD EPYC processors that can hold up to 32,768 cores. Read that number twice, because it is an unusual bet. AMD is not selling you more GPUs. It is selling you a dedicated place to run agents, next to the GPUs that serve the models those agents call.

The bottleneck for agent execution has moved from raw compute to scheduling and power

What AMD actually announced

Two rack shapes, one blueprint. The first is a CPU-only rack powered by 6th Gen EPYC processors supporting up to 32,768 cores, designed for hyperscale customers who want dedicated capacity for agent execution with connections to separate model inference infrastructure. The second is an all-in-one rack that combines EPYC CPU nodes, Instinct MI350P GPU nodes and general-purpose compute on a single platform for enterprise agentic AI. Both are OCP spec-design based, both can be air or liquid cooled, and both carry an integrated token router that directs agents' model calls across frontier and open-source models. AMD says the specification will be contributed to OCP, and configurations are planned to be available in November 2026.

# Size the rack before you buy the rack.
# A Helion CPU-only configuration tops out at 32,768 EPYC cores.
AGENT_VCPU_PER_SESSION = 0.25   # a tool-calling agent is mostly waiting
SESSIONS_PER_CORE      = 1 / AGENT_VCPU_PER_SESSION

CORE_BUDGET = 32768
MAX_SESSIONS = CORE_BUDGET * SESSIONS_PER_CORE

print(f'cores={CORE_BUDGET} concurrent agent sessions={int(MAX_SESSIONS):,}')

# The number that matters is not peak. It is the tail.
def sessions_you_can_hold(utilization=0.65, retries_per_hour=1.8, avg_minutes=4.0):
    peak = MAX_SESSIONS * utilization
    offered = peak * retries_per_hour * (avg_minutes / 60)
    return {'sustainable': int(peak), 'headroom_after_retries': int(peak - offered)}

print(sessions_you_can_hold())

Why a CPU-only rack is the interesting one

Agent execution is mostly waiting. A tool-calling loop plans, calls an API, waits, parses, retries, and reasons about the result; the arithmetic inside that loop is trivial compared to the model call. That is why a rack full of cores with no GPUs is not an oversight, it is a statement about where the cost lives. AMD's framing is explicit: the CPU-only rack provides dedicated capacity for agent execution and connects to separate model inference infrastructure. Dan McNamara, senior vice president and general manager of Compute and Enterprise AI at AMD, put it as agentic AI not being a single workload, spanning personal agents, enterprise applications, code execution and complex workflows, each with different compute requirements. The design consequence is real: you can scale concurrency without re-buying inference.

# The blueprint is an OCP spec-design, so it lands as structured config.
# Keep your own rack inventory machine-readable and diff it in CI.
import json, yaml

BLUEPRINT = '''
rack:
  name: helion-cpu-only
  standard: ocp-spec-design
  compute:
    cpu: EPYC 6th-gen
    max_cores: 32768
  cooling: [air, liquid]
  inference: external
  token_router: integrated
'''

spec = yaml.safe_load(BLUEPRINT)
print(json.dumps(spec['rack']['compute'], indent=2))

def validate(blueprint):
    problems = []
    if blueprint['rack']['compute']['max_cores'] > 32768:
        problems.append('core count above the published CPU-only ceiling')
    if blueprint['rack']['cooling'] == 'liquid' and 'facility' not in blueprint:
        problems.append('liquid cooling declared with no facility data')
    return problems or ['ok']

print(validate(spec))

32,768 cores is not a marketing number; it is a power and scheduling problem

The token router is the part to copy

Helion bundles an integrated token router that directs agents' model calls across frontier and open-source models. Put the hardware aside for a moment. A router at the infrastructure layer means routing stops being an application-level afterthought and becomes a declared policy. That matters because unrouted agent traffic is where budgets die: a summarisation call that does not need a frontier model will happily consume one, thousands of times an hour. The right pattern is to classify the call first, then pick a tier. Plan and critique steps earn frontier reasoning. Tool-argument generation is schema-shaped and cheap to verify. Summarisation and classification are high-volume and tolerate smaller open models. The hardware simply means you can enforce that split without asking every service team to reimplement it.

# The integrated token router is the part worth copying in software.
# Route each agent call by task class instead of by which model you like.
ROUTES = {
    'plan':        {'tier': 'frontier', 'max_tokens': 2048, 'why': 'reasoning quality dominates'},
    'tool_args':   {'tier': 'mid',      'max_tokens': 1024, 'why': 'schema-shaped, cheap to check'},
    'summarize':   {'tier': 'open',     'max_tokens': 512,  'why': 'high volume, tolerant of small models'},
    'classify':    {'tier': 'open',     'max_tokens': 64,   'why': 'labels only'},
}

def route(task_class, request):
    rule = ROUTES.get(task_class)
    if rule is None:
        raise KeyError(f'unrouted task class: {task_class}')
    request = dict(request, **rule)
    return request

print(route('summarize', {'model_pool': ['frontier', 'mid', 'open']}))

# An unrouted task class is an unbounded bill. Make it fail loudly.

Two numbers decide which rack you need

Before you compare rack shapes, compute the ratio between agent sessions and model tokens. A published CPU-only configuration is 32,768 cores; if a tool-calling agent needs roughly a quarter of a vCPU while it waits, that is on the order of a hundred thousand concurrent sessions at full occupancy, and considerably fewer at a realistic 60 to 70 percent. Now compute your token demand over the same window. If token demand is the binding constraint, the all-in-one rack with MI350P GPU nodes matches your shape. If your constraint is the number of agent loops in flight, and your inference already lives in a fleet you are not replacing, the CPU-led rack is the honest answer. Power is the third input, and it is usually the one that decides first.

# Epyc cores per rack are a power question as much as a compute question.
RACK = {'cores': 32768, 'watts_per_core': 1.9, 'pue': 1.15, 'kw_per_rack_budget': 90}

def power_plan(rack):
    it_load_kw = rack['cores'] * rack['watts_per_core'] / 1000
    facility_kw = it_load_kw * rack['pue']
    racks = facility_kw / rack['kw_per_rack_budget']
    return {'it_kw': round(it_load_kw, 1),
            'facility_kw': round(facility_kw, 1),
            'racks_at_budget': round(racks, 2),
            'verdict': 'air-coolable with headroom' if racks <= 1 else 'split across racks'}

print(power_plan(RACK))

# Fill in the real TDP and PUE from your own facility before you present it.

Do the capacity arithmetic before you pick a rack shape

What is not in the announcement

There are no published performance benchmarks for agent execution on Helion, no pricing, and no committed per-partner configurations beyond ecosystem statements from Cisco, Dell, Hon Hai, HPE, Lenovo, QCT, Supermicro and Wiwynn. The specification contribution to OCP is planned rather than completed, and availability is November 2026, which in infrastructure terms means watch the spec land in the repository before you write a purchase order. The honest summary is a reference design with a clear argument, published by a vendor with an interest in the argument. Treat the core count as a design ceiling to plan against, not a delivered guarantee you can put in a capacity model today.

# Availability is announced for November 2026. Plan the cutover, do not assume it.
from datetime import date

CAPACITY_PLAN = {
    'rack_shape': 'cpu-only',
    'agent_execution': {'owner': 'platform', 'slo_p95_ms': 400},
    'model_inference': {'owner': 'ml-platform', 'location': 'existing fleet'},
    'rollback': 'route agent execution back to the current cluster',
    'ga_date': date(2026, 11, 30),
}

def preflight(plan):
    checks = {
        'execution_and_inference_are_separate': 'model_inference' in plan,
        'slo_is_attached_to_a_tier': 'slo_p95_ms' in plan['agent_execution'],
        'rollback_is_a_sentence_an_oncall_can_follow':
            len(plan['rollback']) > 20,
    }
    return {k: ('pass' if v else 'fail') for k, v in checks.items()}

print(preflight(CAPACITY_PLAN))
# A rollback plan with no latency budget is a wish, not a plan.

The lesson that survives the vendor

Separate the agent execution plane from the model inference plane, and size each one on its own curve. Execution scales with concurrency and is dominated by idle-time memory and scheduling; inference scales with tokens and is dominated by accelerator memory bandwidth. Bundling them into one purchase decision means your cheap dimension gets dragged by your expensive one. Whether or not a rack with AMD silicon ends up in your data centre, that separation is the reason this announcement is worth reading: it names the two curves most teams still model as one. Write the ratio down for your own workload before the next infrastructure review, and the vendor conversation gets considerably shorter.