Stanford SNAP Group Open-Sources Memorilla, a 16-Token Latent Memory Module for Frozen LLMs

Stanford researchers have open-sourced Memorilla, a 194M-parameter trainable memory module that compresses an entire document collection into 16 memory tokens for a frozen Qwen3-8B, with code and checkpoints under Apache 2.0. On FactKG it scored 84.33% versus 80.43% for full-context prompting while using 108x fewer document tokens, and it posted the highest reported average of 49.68 across 8 benchmarks. It loses on PubMedQA (59.00% versus 69.00% for LLMLingua-2), where the abstracts total only 324 tokens, and the full training recipe takes about 19 hours on 4 H100 GPUs.