class

Cadmium::Tokenizer::WordPunctuation

Inherits Cadmium::Tokenizer::Regex < Cadmium::Tokenizer::Base < Cadmium::Tokenizer::Diacritics < Cadmium::Tokenizer::StopWords < Reference < Object

Constants

REGEX_PATTERN = /(\w+|[а-я0-9_]+|\.|\!|\'|\"")/i

Constructors