Graphcore and Arm compress 11B VLM to 3.7 GB on phone
Start here
Graphcore and Arm researchers compressed the 11B Llama 3.2 Vision Instruct model from 21.3 GB of bfloat16 weights to a 3.7 GB implementation that runs on an Android phone CPU. Their S3D8 format stores three weights per byte at 2.7 bits per weight using 3D vector quantisation and a small lookup table, and quantisation-aware training with sampled prompts recovers downstream task performance that direct casting loses. The team reports S3D8 reaching 33.8 GMAC/s versus 26.5 GMAC/s for INT8 on a 4096 x 14336 matrix-vector product, and about 22% extra compression over block-scaled INT at equal task performance. They say the 11B model is still too heavy for mainstream mobile use and that the implementation would benefit from Vulkan kernels to improve prefill time; they released it to support future development.
Generated from reports · updated 23 h ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AIGraphcore and Arm fit an 11B Llama 3.2 Vision model on an Android phone at 3.7 GB
Graphcore and Arm researchers compressed the 11B Llama 3.2 Vision Instruct VLM from 21.3 GB of bfloat16 weights to a 3.7 GB implementation that runs on an Android phone CPU. Their S3D8 format stores three weights per byte at 2.7 bits/weight using 3D vector quantisation and a small lookup table, and quantisation-aware training with sampled prompts recovers downstream task performance that direct casting destroys. The team reports S3D8 reaching 33.8 GMAC/s versus 26.5 GMAC/s for INT8 on a 4096 x 14336 matrix-vector product, and about 22% extra compression over block-scaled INT at equal task performance; they note the 11B model is still too heavy for mainstream mobile use and would benefit from Vulkan kernels for prefill.
Heat for this event
- Current, comparable range
- 5
- Peak, comparable range
- 10November 15 18:00
- Change in 24 hours
- –
The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.