Run 80B Qwen on 4.3GB RAM with Expert Streaming on Mac
Swiftlet demonstrates how to run Qwen MoE models locally by keeping only the dense core in RAM and streaming routed experts from SSD on demand. It reports ~4.3GB RAM for a 4-bit 80B model on an M5 Mac, and an on-device 35B experience on iPhone around ~2.5GB RAM.