# PagedAttention: how vLLM manages KV cache and GPU memory An explanation of logical and physical KV blocks, copy-on-write sharing and the limits of the original chatbot benchmark. Source: https://www.iplexlaw.co.kr/en/blog/pagedattention-vllm-kv-cache HOME / NEWS & INSIGHTS NEWS & INSIGHTS PagedAttention: how vLLM manages KV cache and GPU memory An explanation of logical and physical KV blocks, copy-on-write sharing and the limits of the original chatbot benchmark. AI 2026.10.19 published IPLEX 7 min read Why generation needs its own memory This article examines arXiv version 1 of “Efficient Memory Management for Large Language Model Serving with PagedAttention,” submitted by Woosuk Kwon, Zhuohan Li and colleagues on 12 September 2023. Autoregressive models generate tokens one at a time. Their KV cache retains computed attention keys and values, not a document containing the user’s text. Longer sequences require more cache. Reserving a large contiguous region for each request wastes unfilled space and creates fragmentation when requests have different lengths. PagedAttention addresses allocation and sharing rather than retraining the model. Hypothetical answer candidates sharing input KV blocks Logical order is separate from physical placement The method stores a fixed number of tokens’ KV data in each block. A block table maps the request’s logical sequence to physical GPU blocks and tracks filled positions; neighboring logical blocks need not be physically adjacent. Allocate blocks for the prompt, write subsequent tokens into the last block, and allocate another only when necessary. Attention reads through this mapping, preserving token order. Requests can share a memory pool while retaining their own mappings: unrelated customers’ conversations are not merged. Two answers to one prompt: a hypothetical example Suppose a support tool generates answers A and B from exactly the same prompt and instructions. Their tables can point to the same physical input blocks. Generated portions diverge and need separate storage; matching words alone do not imply equal KV values because context and position matter. If a shared final input block still has room, writing a new token requires copy-on-write when other references remain. Filled common blocks stay shared; only the block needing separation is copied, or a new block is allocated. Blocks with no remaining references are reclaimed. The illustration is conceptual, not a product screen or measured result. What the approximately twofold result means Section 6.5 reports that vLLM sustained approximately twice the request rate of three Orca baselines in a chatbot experiment. This concerns throughput under the tested latency behavior, not doubled answer accuracy or every answer finishing in half the time. The experiment used synthesized request arrivals and did not keep KV caches between conversation rounds. Its result is not a measurement of the hypothetical support service. Input/output lengths, GPU capacity and traffic patterns affect gains. Storage management and the implementation record Block allocation does not retrieve only selected history, discard less important tokens or automatically extend the model’s context window. Smaller blocks reduce unused tail space but introduce mapping overhead. FlashAttention-2 focuses on attention computation and parallel work; PagedAttention focuses on placement and sharing of growing KV data. For development and IP review, record allocation timing, mapping structure, sharing conditions, copying and reclamation, with input/output lengths and concurrent-request measurements. Distinguish these practical recommendations from the paper’s experiments and explain changed steps rather than asserting only “less memory.” The original figure labels and forms are preserved; the text explains the source material. Related service: AI patents PagedAttention paper, version 1 Original technical analysis on Tistory Read the Korean source This article reflects the information available when it was published. Contact us to discuss your circumstances. Discuss this topic ↗ All articles ON THIS PAGE Why generation needs its own memory Logical order is separate from physical placement Two answers to one prompt: a hypothetical example What the approximately twofold result means Storage management and the implementation record TALK TO IPLEX Discuss your IP questions We consider your technology and business needs together. ↗ Contact us Newer Defining the target invention in a patent-scope proceeding ↗ Older Design creativity and combinations of familiar components ↗ Related insights Artificial Intelligence 2026.10.29 SwiLA: organizing memory around context How Switching Linear Attention combines linear regressors within a fixed recurrent state, what its recall experiments establish, and where capacity and computational limits remain. ↗ Read article Artificial Intelligence 2026.10.28 Microsoft’s speech synthesis patent: combining sounds, words and sentence structure US12094447B2 explains how phoneme or character information can be combined with word embeddings and grammatical structure to generate speech. ↗ Read article Artificial Intelligence 2026.10.27 GBLA: reading long recommendation histories efficiently A technical and patent-practice reading of gated bidirectional linear attention, its hybrid encoder and the limits of the reported speed and retrieval results. ↗ Read article - https://www.iplexlaw.co.kr/en - https://www.iplexlaw.co.kr/en/blog/category/ai - https://www.iplexlaw.co.kr/en/ai-patents - https://arxiv.org/abs/2309.06180v1 - https://iplexlaw.tistory.com/11 - https://www.iplexlaw.co.kr/ko/blog/pagedattention-vllm-kv-cache - https://www.iplexlaw.co.kr/en/contact - https://www.iplexlaw.co.kr/en/blog - https://www.iplexlaw.co.kr/en/blog/pagedattention-vllm-kv-cache#article-tistory11-en-0 - https://www.iplexlaw.co.kr/en/blog/pagedattention-vllm-kv-cache#article-tistory11-en-1 - https://www.iplexlaw.co.kr/en/blog/pagedattention-vllm-kv-cache#article-tistory11-en-2 - https://www.iplexlaw.co.kr/en/blog/pagedattention-vllm-kv-cache#article-tistory11-en-3 - https://www.iplexlaw.co.kr/en/blog/pagedattention-vllm-kv-cache#article-tistory11-en-4 - https://www.iplexlaw.co.kr/en/contact - https://www.iplexlaw.co.kr/en/blog/brunch-case-1853 - https://www.iplexlaw.co.kr/en/blog/brunch-case-1852 - https://www.iplexlaw.co.kr/en/blog/swila-switching-linear-attention - https://www.iplexlaw.co.kr/en/blog/swila-switching-linear-attention - https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts - https://www.iplexlaw.co.kr/en/blog/microsoft-multilevel-tts - https://www.iplexlaw.co.kr/en/blog/gbla-generative-retrieval - https://www.iplexlaw.co.kr/en/blog/gbla-generative-retrieval