Cadmium::Tokenizer::VisibleChar
Inherits Cadmium::Tokenizer::Regex < Cadmium::Tokenizer::Base < Cadmium::Tokenizer::Diacritics < Cadmium::Tokenizer::StopWords < Reference < Object
Constants
REGEX_PATTERN = /\s+|(?<=[\P{Cc}])(?=[\P{Cc}])/
Constructors
new
Source