module

Xerp::Query::Expansion

Constants

DEFAULT_MAX_DF_PERCENT = 22.0
DEFAULT_MIN_SIMILARITY = 0.25
DEFAULT_TOP_K_PER_TOKEN = 8

Default expansion parameters

DEFAULT_W_FEEDBACK = 0.2
DEFAULT_W_IDF = 0.1
DEFAULT_W_LINE = 1.0

Default blend weights for scoring

KIND_ALLOWLIST = Set {Tokenize::TokenKind::Ident, Tokenize::TokenKind::Word, Tokenize::TokenKind::Compound}

Class methods

all_tokens_with_ids(expanded : Hash(String, Array(ExpandedToken))) : Array(ExpandedToken)

Returns all expanded tokens with their token_ids (for scoring).

Source
expand(db : DB::Database, query_tokens : Array(String), top_k : Int32 = DEFAULT_TOP_K_PER_TOKEN, min_similarity : Float64 = DEFAULT_MIN_SIMILARITY, weights : BlendWeights = BlendWeights.new, max_df_percent : Float64 = DEFAULT_MAX_DF_PERCENT) : Hash(String, Array(ExpandedToken))

Expands query tokens using semantic neighbors if available. Returns identity expansion plus nearest neighbors from trained vectors. Uses union+rerank blending when both models are trained.

Source
get_neighbors(db : DB::Database, token_id : Int64, top_k : Int32, min_similarity : Float64, weights : BlendWeights, max_df_percent : Float64 = DEFAULT_MAX_DF_PERCENT) : Array(NamedTuple(token: String, token_id: Int64, score: Float64, kind: Tokenize::TokenKind))

Gets neighbors from the line model with scoring. Reranks with: score = w1similarity + w2idf + w3*feedback_boost

Source
model_trained?(db : DB::Database, model : String) : Bool

Checks if a specific model has been trained.

Source
to_entries(expanded : Hash(String, Array(ExpandedToken))) : Hash(String, Array(ExpansionEntry))

Converts expansion result to the ExpansionEntry format for QueryResponse.

Source

Nested types