module

JoobQ::Latency

How long jobs take to run, as a histogram every process adds to and any process can read.

Each execution lands in one of a fixed set of logarithmic buckets (eight per decade, 1 ms to about 12 minutes, then one for anything slower). Workers keep counts in memory and add them to the cumulative hash joobq:stats:latency ("queue|bucket" => count) every few seconds, like the other joobq:stats:* counters. A reader that samples the hash twice has the distribution of what ran in between, and merges distributions by adding buckets.

A bucket is 33 % wider than the one below it, so a percentile reported as the middle of its bucket is off by at most about 15 % between 1 ms and about 12 minutes; jobs faster than 1 ms report as 1 ms and slower ones as the last bound. Delivery to Redis is at least once: counts handed back after a failed flush may include some the server had already applied.

Constants

BUCKET_COUNT = OVERFLOW + 1
FIRST_SECONDS = 0.001
FLUSH_EVERY = 5.seconds
KEY = "joobq:stats:latency"
LAST_BUCKET = 47
Log = ::Log.for("LATENCY")
OVERFLOW = LAST_BUCKET + 1
PER_DECADE = 8

Class methods

bucket(seconds : Float64) : Int32

The bucket a duration falls in: the first whose upper bound is at least the duration.

Source
flush(redis : Redis::PooledClient = RedisStore.instance.redis) : Nil

Adds what this process has counted since the last flush to the shared hash. When Redis cannot be reached the counts are kept for the next try.

Source
middle_seconds(bucket : Int32) : Float64

The geometric middle of a bucket, in seconds: the best single value to report for it.

Source
observe(queue : String, seconds : Float64) : Nil

Counts one execution. Cheap enough for every job: a lookup and an add under a lock held for nanoseconds.

Source
publish_now

Publishes what is left, without failing the caller: shutdown must not depend on Redis being up.

Source
read(redis : Redis::PooledClient = RedisStore.instance.redis) : Hash(String, Array(Int64))

Reads the shared hash: queue => counts per bucket.

Source
reset

Forgets what this process has counted; for specs.

Source
stop

Stops publishing in the background and publishes what is left. Counting again restarts it.

Source
upper_seconds(bucket : Int32) : Float64

The upper bound of a bucket in seconds; the overflow bucket reports the last bound.

Source