The CPU Is Back in LLM Inference—Rethink the Split
LLM inference no longer looks like “CPU submits, GPU generates.” As assistants become agentic, tool calls and orchestration move onto the critical path, making CPU-heavy serving practical. This post breaks down prefill/decode, KV cache, and the software techniques (continuous batching, prefix caching, chunked prefill) that make CPU inference viable.