KVP: learning which information to keep in an AI model’s KV cache
A technical analysis of learned KV-cache eviction: future-utility ranking, offline reinforcement learning, benchmark results and the limits of the reported efficiency gains.
Read article