class

Cadmium::WordPunctuationTokenizer

Inherits Cadmium::RegexTokenizer / Cadmium::Tokenizer / Reference / Object

Constants

REGEX_PATTERN = /(\w+|[а-я0-9_]+|\.|\!|\'|\"")/i

Constructors