Weighted State Souping for Efficient Long-Context Retrieval

Apple researchers and Harvard collaborators propose weighted state souping, a method that learns query-dependent weights for recombining cached SSM state encodings, improving long-context performance on single- and multi-hop document QA, needle-in-a-haystack retrieval, variable tracking, and tool use. A sparse variant achieves near-constant encoding cost with respect to context length while preserving or sometimes improving performance. The paper was accepted at the PALM workshop at NeurIPS 2026.