No Dropped Frames: designing a VLM around a latency budget | Moondream

Why watch

A technical deep-dive into building Moondream, a real-time vision language model optimized for video streams. Covers architecture decisions around upcycling dense models to MoE for latency optimization, custom SuperBPE tokenizer design, and the engineering tradeoffs between modeling choices and inference performance. Target audience is ML engineers and researchers building production inference systems.