class

Naive::Tokeniser

Inherits Reference < Object

Constructors

new(stop_words : Array(String) = [] of String, signs : Array(String) = ["?!#%&"])
Source

Instance methods

remove_punctuation(token : String)
Source
remove_stop_words(token : String)
Source
tokenise(text : String)
Source