class

Tokenizer

Inherits Reference < Object

Constructors

new(token_size : Int32, indicator = nil)
Source
new(*delimiter)

simple tokenizer

Source
new

abstract tokenizer

Source

Instance methods

buffer
Source
clear
Source

Nested types