Cadmium::SentenceTokenizer
Inherits Cadmium::Tokenizer / Reference / Object
Constants
REGEX_PATTERN = /([\"\'\‘\“\'\"\[\(\{\⟨][^\.\?\!]+[\.\?\!][\"\'\’\”\'\"\]\)\}\⟩]|[^\.\?\!]+[\.\?\!])\s?/
Instance methods
tokenize(string : String) : Array(String)
Source