class

Logarithm::MemoryEfficientBatchProcessor

Inherits Reference < Object

Memory-efficient batch processor.

This class processes logs in batches while monitoring memory usage to prevent out-of-memory errors.

Example usage:

processor = MemoryEfficientBatchProcessor.new(
  batch_size: 1000,
  max_memory_mb: 512
)

logs.each do |log|
  if processor.add_entry(log)
    # Entry added to current batch
  else
    # Batch full, process current batch first
    process_batch(processor.current_batch)
    processor.reset_batch
    processor.add_entry(log)
  end
end

Constructors

new(batch_size : Int32 = 1000, max_memory_mb : Int32 = 512)
Source

Instance methods

add_entry(entry : LogEntry) : Bool

Add an entry to the current batch if memory allows.

Source
batch_ready?

Check if batch is ready for processing.

Source
batch_size
Source
current_batch
Source
current_memory_usage

Get current memory usage in bytes.

Source
max_memory_mb
Source
memory_usage_percent

Get current memory usage as percentage of max.

Source
reset_batch

Reset the current batch.

Source