class

Logarithm::ConcurrentInferenceEngine

Concurrent inference engine for model predictions.

This class provides concurrent model inference using a pool of workers to maximize throughput for anomaly detection.

Example usage:

engine = ConcurrentInferenceEngine.new(model, workers: 4)
engine.start

results = engine.predict_batch(log_vectors)

Constructors

new(model : AbstractModel, workers : Int32 = System.cpu_count)

predict_batch(batch : Array(Tensor)) : Array(Float64)

Submit a batch for inference.

predict_batches(batches : Array(Array(Tensor))) : Array(Array(Float64))

Submit multiple batches for concurrent processing.

running?

Check if engine is running.

start

Start the inference engine.

stop

Stop the inference engine.

workers