Evergreen Tools
|
← Back to AI Tools

TorchNeuron: Run Your PyTorch Code on AWS Trainium

Trainium registered as a standard PyTorch device: add .to('neuron') and your model runs, no rewrite and no manual porting

Tool Interface

Official product page linked below

Features

  • ✓ Registers Trainium through PrivateUse1, PyTorch's official extension point, so torch.nn, torch.tensor, torch.autocast, torch.distributed and torch.profiler work as written
  • ✓ Covers Trainium2 and Trainium3 with eager, experimental adaptive eager, torch.compile and NKI custom kernels
  • ✓ Standard distributed APIs such as DDP and FSDP2 run unmodified
  • ✓ Open source under Apache 2.0 at github.com/aws-neuron/torch-neuronx, public beta from 20 October 2026
  • ✓ Hugging Face now treats Trainium as a native backend for its Transformers and Diffusers libraries

How to Use

  1. Install the torch-neuronx public beta on a Trainium instance and follow the official setup guide
  2. Write device-agnostic code with torch.accelerator, or simply move the model with .to('neuron')
  3. For compiled workloads call torch.compiler.set_default_backend("neuron") once, then model.compile()
  4. Profile with the usual torch.profiler and trace events back to source lines in Neuron Explorer

FAQ

How much code has to change?

In the simplest case exactly one line: add .to('neuron') to the model. Modern code written against torch.accelerator runs unmodified; the compiled path needs the neuron backend named once via torch.compiler.set_default_backend("neuron").

Which chips are supported?

Trainium2 and Trainium3. It registers the chip as a standard device through PyTorch's PrivateUse1 extension point rather than shipping a separate framework.

What is the licence and how do you get it?

Apache 2.0, published at github.com/aws-neuron/torch-neuronx and going live as a public beta on 20 October 2026 around PyTorch Conference North America.

Can I write custom kernels?

Yes, through the Neuron Kernel Interface (NKI). Eager mode is for research and debugging, adaptive eager is an experimental performance mode, and torch.compile carries production workloads.

Do distributed jobs need rewriting?

No. Standard distributed APIs including DDP and FSDP2 work as written, and profiling still goes through torch.profiler with traces viewable in Neuron Explorer.

Source: the official PyTorch blog post "Introducing TorchNeuron: Native PyTorch backend for AWS Trainium" (published 19 October 2026, open-sourced as a public beta on 20 October 2026).