Skip to content
Hacker News · AI·· 8 h agoAI score58

RunAnywhere releases TrnRT, an open-source LLM inference engine for AWS Trainium

Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium

AI summary

RunAnywhere released TrnRT, an open-source inference engine that serves an OpenAI-compatible API from a single trn2.48xlarge and runs decode as NKI megakernels. GLM-5.3-Flash (320B MoE, 18B active, FP8, 1M-token context) is the first supported model, and the project reports AgentX coding-agent runs at 32 concurrent agents costing $0.040 per million tokens at spot pricing.

Source:Hacker News · AI · github.com