Skip to content
Hot eventDeveloping

RunAnywhere releases TrnRT open-source inference engine for AWS Trainium

1 reports1 reporting sourcesUpdated 8 h ago

Start here

AI digest

RunAnywhere released TrnRT, an open-source LLM inference engine for AWS Trainium that serves an OpenAI-compatible API from a single trn2.48xlarge instance and runs decode as NKI megakernels. The first supported model is GLM-5.3-Flash, a 320B-parameter mixture-of-experts model with 18B active parameters, FP8 precision and a 1M-token context; RunAnywhere says more open models are planned. Setup requires Docker, a trn2.48xlarge to serve on, and any x86_64 host to compile on, where the compile cache must be built first and takes hours. RunAnywhere reports AgentX coding-agent runs at 32 concurrent agents costing $0.040 per million tokens at spot pricing. The project describes itself as independent and not affiliated with or endorsed by AWS.

Generated from reports · updated 8 h ago

Report timeline

Follow the reports to see the event from different sides.

November 18
  1. Hacker News · AI
    RunAnywhere releases TrnRT, an open-source LLM inference engine for AWS Trainium

    RunAnywhere released TrnRT, an open-source inference engine that serves an OpenAI-compatible API from a single trn2.48xlarge and runs decode as NKI megakernels. GLM-5.3-Flash (320B MoE, 18B active, FP8, 1M-token context) is the first supported model, and the project reports AgentX coding-agent runs at 32 concurrent agents costing $0.040 per million tokens at spot pricing.

Heat for this event

Current, comparable range
8
Peak, comparable range
10November 18 01:00
Change in 24 hours
–

The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.