module

Xerp::Vectors::Cooccurrence

Builds token co-occurrence counts from the indexed corpus.

Models: MODEL_LINE: Traditional linear - sliding window over whole file in text order MODEL_SCOPE: Scope-aware - level-based isolation (leaves swept alone, siblings co-occur)

SCOPE is recommended for code - respects logical structure without crossing scope boundaries. LINE is traditional word2vec-style co-occurrence (whole document, one pass).

Constants

DEFAULT_MIN_COUNT = 3
DEFAULT_TOP_NEIGHBORS = 32
DEFAULT_WINDOW_SIZE = 5

Default training parameters

MODEL_IDS = {MODEL_LINE => 1, MODEL_SCOPE => 3}

Model name to ID mapping (matches models table)

MODEL_LINE = "cooc.line.v1"

Model identifiers (name -> id mapping)

MODEL_SCOPE = "cooc.scope.v1"
SIMILARITY_SCALE = 65535.0

Similarity quantization scale (16-bit precision)

VALID_MODELS = [MODEL_LINE, MODEL_SCOPE]

Class methods

build_counts(db : DB::Database, model : String, window_size : Int32 = DEFAULT_WINDOW_SIZE) : Int64

Builds co-occurrence counts from all indexed files for a specific model. MODEL_LINE: sliding window co-occurrence (textual proximity) MODEL_SCOPE: level-based isolation (structural siblings)

Source
compute_neighbors(db : DB::Database, model : String, min_count : Int32 = DEFAULT_MIN_COUNT, top_k : Int32 = DEFAULT_TOP_NEIGHBORS) : Int64

Computes nearest neighbors from co-occurrence counts for a specific model. Uses in-memory sparse matrix multiplication with inverted index for efficiency.

Source
dequantize_similarity(quantized : Int32) : Float64

Dequantizes 16-bit integer back to similarity (0.0-1.0)

Source
model_id(model : String) : Int32

Gets model_id for a model name

Source
quantize_similarity(similarity : Float64) : Int32

Quantizes similarity (0.0-1.0) to 16-bit integer

Source

Nested types