# NVIDIA’s draft-token selection: faster LLM inference by choosing what to verify An analysis of US20260236763A1: speculative decoding, marginal benefit and processing cost, with a hypothetical delivery-support example. Source: https://www.iplexlaw.co.kr/en/blog/nvidia-speculative-decoding-draft-selection HOME / NEWS & INSIGHTS NEWS & INSIGHTS NVIDIA’s draft-token selection: faster LLM inference by choosing what to verify An analysis of US20260236763A1: speculative decoding, marginal benefit and processing cost, with a hypothetical delivery-support example. AI & Software 2026.10.05 published IPLEX 10 min read Conceptual draft selection and continuation from accepted tokens The publication and the engineering question NVIDIA’s US20260236763A1, published on 13 August 2026, is a patent application publication, not evidence of a granted patent. It describes selecting draft token sets using processing-time and accepted-token information. This article focuses on that selection mechanism; it does not claim that NVIDIA NIM implements every claimed feature. A language model ordinarily produces an answer incrementally, adding each next token to its context. Preparing several continuations can reduce sequential work if the target model accepts them. Preparing long continuations that are repeatedly discarded instead consumes time and memory. The issue is how much speculative work is worth doing. Tokens, drafts and the target model A token may represent a word, part of a word or punctuation. A drafter proposes continuations, while the target model evaluates the selected positions together. The drafter need not always be a separate small model: a distinct process or components within the same model may perform that role. The target accepts a usable prefix and continues from the accepted context; disagreement at one position does not validate the rest of the draft. Figure 5 illustrates this generation, selection and parallel verification flow. Acceptance concerns the model’s generation rule, not the factual accuracy of a delivery date or any other statement. Publication Figure 5: drafter, draft-set selection and parallel target verification Why a longer draft is not automatically better The selector considers the expected increase in accepted tokens from extending a candidate, alongside the extra processing time. Here “profit” means incremental generation benefit, not sales revenue. Previous acceptance and timing measurements help estimate both quantities; total draft length alone is insufficient. In Figure 10, the procedure selects the remaining draft set with the largest marginal profit and adds it to the inference group. It then compares the marginal-profit-to-cost ratio with a threshold to decide whether to continue selecting, provided candidates remain. The chosen group is sent to the model. This is more specific than simply ranking every candidate by ratio or always choosing the longest one; resources and memory can inform the threshold. Publication Figure 10: add a high-marginal-benefit draft, then test the benefit-to-cost threshold A hypothetical delivery-support conversation Suppose drafts extend “Your order” to “Your order will” and “Your order will ship today.” These readable fragments illustrate the process and are not actual token boundaries. If the target accepts the prefix but rejects the ending, generation proceeds using its result rather than retaining the rejected suffix. Real order status must come from the service’s data and business logic. The system records processing time and accepted length, then uses those observations for the next selection. Frequently rejected endings may justify shorter speculation; predictable language may favour longer drafts. Updating these execution records is not the same as retraining model weights. The delivery illustration is hypothetical, not a customer installation or benchmark. Hypothetical delivery-support example, not actual token boundaries, a service screen or a benchmark How NIM provides a useful product context NVIDIA NIM provides inference microservices that applications call to run models. The NIM for Large Language Models 2.0.10 documentation describes speculative decoding on its vLLM backend, including EAGLE3 drafts, built-in multi-token prediction (MTP) and n-gram prompt lookup. Their model requirements and memory needs differ. Separate two design questions: how drafts are produced, and how much draft material the target should process. The application’s selector helps explain the latter. The presence of speculative decoding in a product is not, by itself, proof that this particular published claim set is implemented. Latency, throughput and workload evidence Response latency and total server throughput measure different things. Spare GPU capacity can make parallel draft verification worthwhile at low or moderate concurrency. Under saturated load, extra computation and memory may eliminate the benefit or reduce throughput. A few faster answers do not establish an improvement for the whole service. Measure realistic request lengths, concurrency, acceptance rates and lengths, memory use and processing time. Preserve the conditions of comparison. Candidate generation, selection and feedback should be separately identifiable so that an engineering improvement can be explained and evaluated against the published disclosure. Preparing a technical discussion with IPLEX For an AI or software patent review, organise the publicly shareable system flow, the rules for generating and selecting drafts, and timing comparisons with their evaluation conditions. The technical distinction may lie in the sequence and selection of computations as well as in model size or hardware. IPLEX examines those concrete processing differences when preparing patent applications and rights strategies. This reading of the publication is not a finding of infringement, patent grant or guaranteed acceleration; the implementation and relevant claims require their own assessment. Original Tistory article Related AI patent services Related analysis: updating weights at test time Related analysis: Attention Residuals Read the Korean source This article reflects the information available when it was published. Contact us to discuss your circumstances. Discuss this topic ↗ All articles ON THIS PAGE The publication and the engineering question Tokens, drafts and the target model Why a longer draft is not automatically better A hypothetical delivery-support conversation How NIM provides a useful product context Latency, throughput and workload evidence Preparing a technical discussion with IPLEX TALK TO IPLEX Discuss your IP questions We consider your technology and business needs together. ↗ Contact us Newer Covent Garden: a famous place is not necessarily a recognised service mark ↗ Older Concert lightsticks: why a shared colour did not establish patent infringement ↗ Related insights AI 2026.10.15 PaTH Attention: encoding the path between tokens An analysis of input-dependent Householder transformations, state tracking and the measured benefits and limits of PaTH Attention. ↗ Read article AI 2026.10.14 Huawei Pangu Weather: linking forecast models across time intervals A technical reading of Huawei’s US20250258318A1 publication: selecting forecast intervals, passing predicted states and combining weather data. ↗ Read article AI 2026.10.13 LaCT: updating an AI model’s temporary memory in large chunks A technical review of Large Chunk Test-Time Training, its fast weights, GPU efficiency, causal processing and task-specific experimental limits. ↗ Read article - https://www.iplexlaw.co.kr/en - https://www.iplexlaw.co.kr/en/blog/category/ai - https://iplexlaw.tistory.com/5 - https://www.iplexlaw.co.kr/en/ai-patents - https://www.iplexlaw.co.kr/en/blog/1525199 - https://www.iplexlaw.co.kr/en/blog/attention-residuals-ai-patent-analysis - https://www.iplexlaw.co.kr/ko/blog/nvidia-speculative-decoding-draft-selection - https://www.iplexlaw.co.kr/en/contact - https://www.iplexlaw.co.kr/en/blog - https://www.iplexlaw.co.kr/en/blog/nvidia-speculative-decoding-draft-selection#case-nvidia-en-0 - https://www.iplexlaw.co.kr/en/blog/nvidia-speculative-decoding-draft-selection#case-nvidia-en-1 - https://www.iplexlaw.co.kr/en/blog/nvidia-speculative-decoding-draft-selection#case-nvidia-en-2 - https://www.iplexlaw.co.kr/en/blog/nvidia-speculative-decoding-draft-selection#case-nvidia-en-3 - https://www.iplexlaw.co.kr/en/blog/nvidia-speculative-decoding-draft-selection#case-nvidia-en-4 - https://www.iplexlaw.co.kr/en/blog/nvidia-speculative-decoding-draft-selection#case-nvidia-en-5 - https://www.iplexlaw.co.kr/en/blog/nvidia-speculative-decoding-draft-selection#case-nvidia-en-6 - https://www.iplexlaw.co.kr/en/contact - https://www.iplexlaw.co.kr/en/blog/brunch-case-1798 - https://www.iplexlaw.co.kr/en/blog/brunch-case-1797 - https://www.iplexlaw.co.kr/en/blog/path-attention-householder-transformations - https://www.iplexlaw.co.kr/en/blog/path-attention-householder-transformations - https://www.iplexlaw.co.kr/en/blog/huawei-pangu-weather-time-interval-models - https://www.iplexlaw.co.kr/en/blog/huawei-pangu-weather-time-interval-models - https://www.iplexlaw.co.kr/en/blog/lact-large-chunk-test-time-training - https://www.iplexlaw.co.kr/en/blog/lact-large-chunk-test-time-training