Kimi K3: The 2.8T AI That Wakes Up Just 2% at a Time

Why watch

This video explores Kimi K3, Moonshot AI's 2.8 trillion parameter model that uses a Mixture of Experts architecture to activate only 2% of parameters during inference. Siraj Raval breaks down how the model achieves a 1 million token context window, what open-weights availability means for developers, and the training innovations that enabled its creation. The video is for developers and AI enthusiasts interested in understanding cutting-edge LLM architecture and the implications of open-source frontier models.