KV Cache and RadixAttention: How LLM Servers Avoid Redundant Computation

A visual explainer of the two layers of redundancy in LLM serving. Covers why every new token recomputes all prior key-value pairs, how the KV cache collapses that quadratic waste to linear cost, why concurrent requests still rebuild identical system prompt caches from scratch, and how RadixAttention organizes a shared KV cache as a tree so each new request pays only for its unique suffix.