How Do We Compress Giant AI Models to Run at the Edge? - Léo Arsenin - Cloudflare - dotAI 2026

Why watch

Cloudflare engineer Léo Arsenin presents Unweight, a novel lossless AI model compression technique that applies Huffman coding to weight exponents instead of using quantization. By analyzing BF16 weight distribution, they discovered 99% of weights use only 16 exponent values, enabling 2.6-bit representation. The technique decompresses directly in GPU on-chip memory and feeds tensor cores without slow main memory round-trips, achieving 22% size reduction with bit-exact outputs while saving ~3GB VRAM on Llama 3.1 8B. Ideal for engineers working on AI inference optimization and edge deployment.