AIR-LLM broadcasts LLM weights over RF for edge inference
Start here
Zhihui Gao and co-authors published a paper proposing AIR-LLM, an edge inference architecture in which a central radio such as a 5G base station broadcasts LLM weights over the air, and edge users compute the GEMV of LLM inference directly in the RF domain using RF mixers, so devices never store or load weights. The authors report that on NVIDIA Sionna ray-traced channels from two real urban scenes with a profiled real RF mixer, WikiText-2 perplexity degradation on LLaMA-3.1-8B is limited to 4.0%, while energy is saved 157.7x/40.4x versus FP16 and weight-only quantization baselines. With 20 users, airtime is 104.1x/26.0x shorter. Because the central radio stays user-unaware, one broadcast can serve unlimited users in coverage, the authors say.
Generated from reports · updated 12 h ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AIAir-LLM: Broadcasting AI Weights to Edge Devices via RF Computing for On-the-Fly LLM Inference
Air-LLM is an edge LLM inference architecture in which a central radio such as a 5G base station broadcasts model weights over the air, and edge users complete GEMV directly in the RF domain using RF mixers, so devices never store or load weights. Tested on NVIDIA Sionna ray-traced channels in two real urban scenes with a real RF mixer profile, it limits WikiText-2 perplexity degradation on LLaMA-3.1-8B to 4.0% while saving 157.7x/40.4x energy versus FP16 and weight-only quantization baselines. With 20 users, airtime is 104.1x/26.0x shorter, and because the central radio stays user-unaware, one broadcast serves unlimited users in coverage.
Heat for this event
Not enough continuous observations yet to draw a trend.