struct

Hansa::Classifier

Inherits JSON::Serializable < Struct < Value < Object

Constructors

new(pull : JSON::PullParser)
Source
new(*, __pull_for_json_serializable pull : JSON::PullParser)
Source

Instance methods

classify(content : String, candidates : Array(String) = [] of String)

Scores the content over the candidate languages (go-enry v2 "hints" semantics): with no candidates every known language competes, with candidates only those do.

Source
common_extract_and_replace(content : String, re : Regex) : Tuple(Array(String), String)
Source
default_token_probability

Probability for tokens not present in a language's token table, derived from the corpus like go-enry v2 does

Source
extract_and_replace_operator(content : String) : Tuple(Array(String), String)
Source
extract_and_replace_punctuation(content : String) : Tuple(Array(String), String)
Source
extract_and_replace_regular(content : String) : Tuple(Array(String), String)
Source
extract_and_replace_sgml(content : String) : Tuple(Array(String), String)
Source
extract_and_replace_shebang(content : String) : Tuple(Array(String), String)
Source
extract_remainders(content : String) : Tuple(Array(String), String)
Source
get_sgml_attributes(sgml_tag : String) : Array(String)
Source
known_languages

Despite the name this only reports the 100 most common languages in the corpus, to avoid super unilely false positives for obscure languages

Source
languages_log_probabilities
Source
languages_log_probabilities=(languages_log_probabilities : Hash(String, Float64))
Source
skip_comments_and_literals(content : String) : Tuple(Array(String), String)
Source
tokenize(content : String) : Array(String)
Source
tokens_log_probabilities
Source
tokens_log_probabilities=(tokens_log_probabilities : Hash(String, Hash(String, Float64)))
Source
tokens_log_probability(tokens : Array(String), language : String) : Float64
Source
tokens_total
Source
tokens_total=(tokens_total : Float64)
Source