class

Cadmium::Tokenizer::Regex

Inherits Cadmium::Tokenizer::Base < Cadmium::Tokenizer::Diacritics < Cadmium::Tokenizer::StopWords < Reference < Object

Constructors

new(pattern : ::Regex, *, gaps = nil, discard_empty = nil)

Instance methods

tokenize(string : String) : Array(String)