Hacker News · AI·· 1 h agoAI score34
tufazip Takes #2 on the Large Text Compression Benchmark With a 163M-Parameter Transformer
Learning to Compress a Billion Bytes of Wikipedia
AI summary
A compressor called tufazip placed second on Matt Mahoney's Large Text Compression Benchmark, compressing the billion-byte Wikipedia file enwik9 to 105,388,353 bytes (105.39 MB) including its offline weights and decoder. Its sole prediction model is a 12-layer, 163M-parameter Transformer initialized from a 34M-parameter offline model quantized to 4.33 MB, with two offline-trained layers copied into twelve layers that keep learning during compression. The decoder repeats the same learning process, so the full model's learned weights never need to be transferred.
Source:Hacker News · AI · tufalabs.ai