module

Xerp::Index::BlocksBuilder

Constants

ELIGIBLE_KINDS = Set {Tokenize::TokenKind::Ident, Tokenize::TokenKind::Compound, Tokenize::TokenKind::Word}

Token kinds that count toward block size (eligible for salience normalization)

Class methods

build(db : DB::Database, file_id : Int64, result : Adapters::AdapterResult) : Array(Int64)

Builds blocks from adapter result and stores them in the database. Returns an array of database block_ids corresponding to the adapter blocks.

Source

decode_line_map(blob : Bytes) : Array(Int64)

Decodes a block_line_map blob back to block_ids per line.

Source

update_token_counts(db : DB::Database, block_idx_by_line : Array(Int32), tokens_by_line : Array(Array(Tokenize::TokenOcc)), block_ids : Array(Int64)) : Nil

Computes and stores token counts per block. block_idx_by_line: maps line index (0-based) to block index tokens_by_line: from TokenizeResult, maps line index to token occurrences block_ids: database block_ids corresponding to block indices

Source