Cadmium::VisibleCharTokenizer
Inherits Cadmium::RegexTokenizer / Cadmium::Tokenizer / Reference / Object
Constants
REGEX_PATTERN = /\s+|(?<=[\P{Cc}])(?=[\P{Cc}])/
Constructors
new
SourceInherits Cadmium::RegexTokenizer / Cadmium::Tokenizer / Reference / Object