RinhaDeBackend::References

Reference dataset for KNN/IVF. Stored densely in a single binary file produced once at Docker build time and mmapped read-only at runtime.

Two on-disk slice layouts are kept, both read by the same References instance depending on which loader was used:

1. Row-major (vectors slice, block_count == 0) — produced by the legacy load_from_io JSON loader and used by spec fixtures. Each row is DIMS = 16 Int16 lanes (14 logical + 2 zero pad). Hot path is unchanged from RNH5/RNH6.

2. AOSOA-8 dim-interleaved blocks (blocks slice, block_count > 0) — produced by IvfBuilder and consumed by the runtime IVF kernel. Each block holds 8 vectors with their dims interleaved: lanes [d0_v0..d0_v7, d1_v0..d1_v7, ..., d13_v0..d13_v7], total LOGICAL_DIMS × SLOTS_PER_BLOCK = 112 Int16 lanes = 224 B per block. The runtime scan loads 8 i16 of one dim with a single VPMOVSXWD / VPSUBD / VPMULLD / VPADDQ chain, producing 8 squared distances per dim instead of one per row. Multi-stage early exit (between dim 4 and dim 8) skips a whole block of 8 vectors when every lane already exceeds the current top-5 worst.

Common (both layouts):

• labels : Slice(UInt8). Row-major: length = padded_count, one per row. Block: length = block_count * SLOTS_PER_BLOCK = padded_count, one per slot, in block-major slot order (block 0 slots 0..7, block 1 slots 0..7, ...).
• centroids : Slice(Int16), length = k * DIMS, the IVF cell centers (quantized, 14+2 zero-pad row stride — keeps centroid scan on the VPMADDWD path).
• cell_offsets: Slice(UInt32), length = k + 1. Row layout: row index. Block layout: block index (always even, so cell starts land on a 64 B boundary). Cell c spans [cell_offsets[c], cell_offsets[c+1]) in whichever unit applies.
• cell_radius : Slice(UInt32), length = k; max non-squared L2 distance from quantized centroid c to any vector in cell c (ceil). Computed over real rows only (pad slots/rows excluded — sentinel would torpedo every triangle-inequality prune).
• max_cell_radius: max over cell_radius. Cached in the header so the runtime can use it for a global outer-break check without scanning the per-cell array.
• bbox_min / bbox_max : Slice(Int16), length = k * DIMS. Per-cell axis-aligned bounding box per dimension (pad lanes pinned to 0). Used at query time for an exact cell-pruning check tighter than triangle- inequality.

Floats are quantized as (v * 10_000).round.to_i16. The -1 sentinel for indices 5/6 (no last_transaction) maps to -10_000, naturally outside the [0, 10_000] band.

Binary file format (little-endian, x86_64) — bumped to RNH7 with AOSOA-8 dim-interleaved blocks:

bytes 0..3 : magic "RNH7" (was RNH6 row-major stride-16) bytes 4..7 : count u32 (real, unpadded — for stats) bytes 8..11 : dims u32 (= 16, kept for centroid/bbox stride) bytes 12..15 : k u32 (number of IVF cells) bytes 16..19 : max_cell_radius u32 bytes 20..23 : padded_count u32 (= block_count * SLOTS_PER_BLOCK) bytes 24..27 : block_count u32 (total blocks, including alignment pads) bytes 28..63 : reserved (zeroed) bytes 64.. : blocks (block_count * BLOCK_LANES * Int16, dim- interleaved, each cell starts at an even block index = 64 B aligned) then : labels (block_count * SLOTS_PER_BLOCK * UInt8, slot-major; pad slots carry label 0) then : centroids (k * dims * Int16, stride 16) then : cell_offsets ((k + 1) * UInt32, block indices) then : cell_radius (k * UInt32) then : bbox_min (k * dims * Int16, stride 16) then : bbox_max (k * dims * Int16, stride 16)