macOS VMs on Apple Silicon can run LLM inference far faster when the guest reports conservative Metal capabilities. A process-scoped capability shim steers llama.cpp onto newer Metal kernels, yielding reported 11–16× speedups while keeping the same Virtualization.framework GPU path.
h3-metal is a native MiniMax‑H3 inference engine for Apple Silicon built on Metal. It combines BF16/int8 compute paths, unified-memory-aware buffer reuse, and a stateful interactive workflow to generate video and audio locally.
Swiftlet demonstrates how to run Qwen MoE models locally by keeping only the dense core in RAM and streaming routed experts from SSD on demand. It reports ~4.3GB RAM for a 4-bit 80B model on an M5 Mac, and an on-device 35B experience on iPhone around ~2.5GB RAM.
TurboFieldfare runs Gemma 4 26B-A4B on Apple Silicon with ~2GB RAM by keeping core weights + an FP16 KV cache resident and streaming only the routed MoE experts from SSD. This post explains the memory pieces (MoE routing, KV cache, quantization) and walks through building, installing, and tuning the runtime.