class

OpenAI::StaticChunkingStrategy

Inherits JSON::Serializable < Reference < Object

Constructors

new(pull : JSON::PullParser)
Source
new(max_chunk_size_tokens : Int64 = 0, chunk_overlap_tokens : Int64 = 0)
Source
new(*, __pull_for_json_serializable pull : JSON::PullParser)
Source

Instance methods

chunk_overlap_tokens

The number of tokens that overlap between chunks. The default value is 400.

Note that the overlap must not exceed half of max_chunk_size_tokens.

Source
chunk_overlap_tokens=(chunk_overlap_tokens : Int64)

The number of tokens that overlap between chunks. The default value is 400.

Note that the overlap must not exceed half of max_chunk_size_tokens.

Source
max_chunk_size_tokens

The maximum number of tokens in each chunk. The default value is 800. The minimum value is 100 and the maximum value is 4096.

Source
max_chunk_size_tokens=(max_chunk_size_tokens : Int64)

The maximum number of tokens in each chunk. The default value is 800. The minimum value is 100 and the maximum value is 4096.

Source
validate!
Source