The research problem
Gated Bidirectional Linear Attention for Generative Retrieval, by Artem Matveev, Vladislav Tytskiy, Sergei Makeev and Sergei Liamaev of Yandex, studies efficient use of long interaction histories. This analysis follows arXiv version 2, dated 8 June 2026, of the paper accepted at SIGIR 2026. Generative retrieval produces semantic item identifiers for candidate retrieval; it is not ordinary generation of a textual answer.
Bidirectional history, autoregressive output
The encoder can compare both directions within the history already available at prediction time, while the decoder generates item-ID tokens autoregressively. Bidirectionality within that observed history does not mean access to future target labels. In the reported industrial test at history length 2,048, bidirectional self-attention achieved Recall@1000 of 0.8667 versus 0.8353 for a causal encoder, a difference of 3.14 percentage points.
What makes attention linear
Ordinary attention has a sequence-length cost proportional to L²dh. Kernel-based bidirectional linear attention rearranges the calculation as φ(Q)(φ(K)ᵀV), normalised by φ(Q)(φ(K)ᵀ1), using a positive feature map such as ELU+1. The core calculation scales as Ldh² and is linear in history length L when feature width dh is fixed.
GBLA adds three mechanisms: local causal Conv1D mixing before the Q/K/V projections, a scalar key gate normalised across sequence positions, and an output gate combining RMSNorm with SiLU. The key gate weights contributions in both the feature aggregate and its denominator. It is soft importance weighting, not binary deletion or a recurrent forgetting factor. The convolution kernel size in the reported setup is four.
Why the encoder remains hybrid
The model concatenates multi-hash embeddings and projects them into the encoder, followed by a decoder with cross-attention. Its nine-layer hybrid encoder repeats [self-attention, GBLA, GBLA], retaining three self-attention layers and replacing six. Thus the full hybrid is not a wholly linear-time model. At length 2,048, hybrid Recall@1000 was 0.8668, compared with 0.8586 for all-GBLA layers, a 0.82-point gap.
Industrial retrieval results and ablations
The private Yandex experiment used seven days of data, about 400 million training examples and next-day evaluation, with a nine-layer encoder of width 1,024 and 16 heads and a one-layer causal decoder. Effective batch size was matched, but peak learning rates differed: 7×10⁻⁴ for self-attention and 5×10⁻⁴ for the hybrid. Recall@1000 measures candidate coverage for a later ranker, not clicks or revenue.
Self-attention/hybrid Recall@1000 was 0.8667/0.8668 at 2,048, 0.8784/0.8790 at 4,096 and 0.8854/0.8842 at 8,192. At the longest of these, Recall@10 was 0.3013/0.2948. The method therefore did not improve every metric. At 2,048, removing the key gate, convolution or gated RMSNorm reduced Recall@1000 from 0.8668 to 0.8661, 0.8652 or 0.8641; plain linear attention reached 0.8620. Its Recall@10 was 0.2675 versus 0.2780 for GBLA. These point estimates do not themselves prove statistical significance.
Layer speed and full training are different measurements
On H100 GPUs, the single-layer benchmark used batch size eight and averages over ten runs against FlashAttention-v3. At length 2,048, GBLA took 0.618 ms against 0.585 ms and was slower. Speedups were 1.50× at 4,096, 2.46× at 8,192 and 8.22× at 32,768, where times were 8.217 versus 67.542 ms. The 8.2× headline applies to that layer benchmark, not an entire service.
End-to-end training at length 8,192 and batch eight fell from 5,400 to 4,200 ms, about 1.29×. At length 2,048 and batch 32, the hybrid took 3,100 versus 3,000 ms. Plain linear attention was faster still at 3,800 ms for 8,192, with its quality trade-off. Batch sizes differ across sequence lengths, so these rows are not controlled measurements of length alone.
Public benchmarks and remaining limits
The Amazon Beauty, Toys and Sports experiments followed the GRID/Tiger setup with 5-core filtering, the last interaction for testing and the penultimate for validation. Flan-T5 embeddings and RK-Means semantic IDs were used; histories were at most 128 items, and results averaged five seeds. Tiger/GBLA Recall@10 was 0.0641/0.0611, 0.0584/0.0579 and 0.0345/0.0329 respectively.
The reported Recall and NDCG values were slightly lower for GBLA in these tests. Similar magnitude is not proof of equivalence, and the paper did not benchmark efficiency on these short histories. Industrial data are private, and reported improvements must be understood within the stated hardware, architecture and evaluation conditions.
A patent-practice reading
The useful technical question is how the problem of long histories is connected to a particular arrangement of local mixing, key weighting, output gating and retained self-attention. A patent analysis should distinguish individual components from their combination, identify what the claims would actually require and connect alleged effects to supporting experiments.
The paper is not evidence that any proposed claim is novel or patentable. Prior-art comparison, claim scope and disclosure sufficiency require separate assessment. For implementation, evaluate candidate recall, downstream ranking, memory and end-to-end latency together. The original diagrams and tables are retained below with local-language explanations; the linked paper and CC BY 4.0 attribution identify their source.
The original figure labels and forms are preserved; the text explains the source material.
Paper and figures: Matveev et al., GBLA v2, 2026 (CC BY 4.0) · CC BY 4.0