class

Cadmium::AggressiveTokenizer

Inherits Cadmium::Normalizers::RemoveDiacritics < Cadmium::Tokenizer < Reference < Object

Constructors

new(*, lang = nil)

Instance methods

tokenize(string : String) : Array(String)

ameba:disable Metrics/CyclomaticComplexity