class

Logarithm::ConcurrentInferenceEngine

Inherits Reference < Object

Concurrent inference engine for model predictions.

This class provides concurrent model inference using a pool of workers to maximize throughput for anomaly detection.

Constructors

new(model : AbstractModel, worker_count : Int32 = System.cpu_count.to_i32)
Source

Instance methods

predict_batch(batch : Array(Tensor)) : Array(Float64)

Submit a batch for inference.

Source
predict_batches(batches : Array(Array(Tensor))) : Array(Array(Float64))

Submit multiple batches for concurrent processing.

Source
running?

Check if engine is running.

Source
start

Start the inference engine.

Source
stop

Stop the inference engine.

Source
worker_count
Source