RunAnywhere releases TrnRT open-source inference engine for AWS Trainium
Start here
RunAnywhere released TrnRT, an open-source LLM inference engine for AWS Trainium that serves an OpenAI-compatible API from a single trn2.48xlarge instance and runs decode as NKI megakernels. The first supported model is GLM-5.3-Flash, a 320B-parameter mixture-of-experts model with 18B active parameters, FP8 precision and a 1M-token context; RunAnywhere says more open models are planned. Setup requires Docker, a trn2.48xlarge to serve on, and any x86_64 host to compile on, where the compile cache must be built first and takes hours. RunAnywhere reports AgentX coding-agent runs at 32 concurrent agents costing $0.040 per million tokens at spot pricing. The project describes itself as independent and not affiliated with or endorsed by AWS.
Generated from reports · updated 8 h ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AIRunAnywhere releases TrnRT, an open-source LLM inference engine for AWS Trainium
RunAnywhere released TrnRT, an open-source inference engine that serves an OpenAI-compatible API from a single trn2.48xlarge and runs decode as NKI megakernels. GLM-5.3-Flash (320B MoE, 18B active, FP8, 1M-token context) is the first supported model, and the project reports AgentX coding-agent runs at 32 concurrent agents costing $0.040 per million tokens at spot pricing.
Heat for this event
- Current, comparable range
- 8
- Peak, comparable range
- 10November 18 01:00
- Change in 24 hours
- –
The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.