Skip to main content

Module sparse

Module sparse 

Source
Expand description

Learned sparse retrieval: an impact index over learned sparse vectors (SPLADE-style vocabulary-sized vectors, or any sparse term weights) with exact dot-product top-k.

Documents are stored as postings per term, ordered by document id in blocks of at most BLOCK entries: a 12-byte block header (first id, count, block maximum impact, payload offset) and a payload of varint id gaps followed by one byte per impact. Learned weights are quantized to 8-bit impacts with one global scale, so a score is scale * sum q_t * impact_t. Queries run document-at-a-time MaxScore (Turtle and Flood, 1995): terms sorted by their maximal contribution, the lowest ones become non-essential once the k-th best score exceeds their summed bound and are only consulted for documents that can still qualify, with the block maxima skipping whole blocks that cannot. The result is the exact top-k over the quantized impacts.

Structs§

ImpactIndex
The impact index.
SparseVector
A sparse vector: term ids ascending with their weights.

Constants§

BLOCK
Postings per block.