JoobQ

Exit status of forge after a clean shutdown, and after one that had to abandon jobs at the deadline.

Opt-in NDJSON for Log::IOBackend. Structured job/worker/queue fields emitted by the library live in data; application correlation fields live in context. Applications must redact secrets before logging them, including exception messages.

The least a step that must run gets, whatever is left of the budget.

Time kept back at the end of the budget (at most half of it) for putting abandoned jobs back and flushing the latency counters, so a stuck job that used up the wait does not leave those steps with nothing.

How long JoobQ.shutdown waits, all steps together, before it gives up and returns.

How many jobs the shutdown had to give up on (returned to their queues) because they were still running at the deadline. Zero after a clean shutdown.

The configured store, or nil while no configuration exists. Never loads the configuration, so it is safe to call from code that runs while the configuration itself is being built.

Enqueues a serialized job onto a registered queue and returns its job id. Unlike Queue#add, which logs a payload that does not fit the queue's job type and moves on, this raises: UnknownQueue for a queue that is not registered, and the JSON error (JSON::ParseException, JSON::SerializableError) for a bad payload.

Starts the schedulers, the reaper, the queues and (when enabled) the REST API, then blocks.

With trap_signals (the default), forge never returns: SIGTERM, SIGINT and SIGHUP (Process.on_terminate traps all three) run JoobQ.shutdown with shutdown_timeout and then exit the process, with status 0, or 3 when jobs were still running at the deadline and had to be returned to their queues. A second signal more than a second after the first exits at once with status 1. Code after the forge call does not run.

With trap_signals: false (also the default when the environment variable JOOBQ_TRAP_SIGNALS is 0, false or no) the application owns the signals: it calls JoobQ.shutdown itself, and forge returns once that has finished.

Initialize configuration with auto-discovery

Initialize configuration with specific loading method

Initialize configuration with hybrid loading method (requires block)

Configuration loading methods

Reset configuration (useful for testing)

Set a custom configuration

True once shutdown has begun.

Stops this process in order, within timeout in total:

1. the REST API started by forge (no new requests)
2. every scheduler of the configuration, the delayed-job scheduler and the expired-job reaper (no new work)
3. the workers of every queue (they claim nothing more)
4. the jobs claimed or running, waited for until timeout minus SHUTDOWN_RESERVE (at most half of timeout)
5. the jobs still not finished then: each is put back at the head of its queue in one atomic step, so they run next in the order this process took them (RedisStore#requeue_many_from_processing; jobs still settling are put back a moment later, in front of them), jobs not yet started are never started, and abandoned_jobs counts them
6. the latency counters (flushed to Redis)

What a job does after it was put back: its execution may well go on (a fiber cannot be killed) and finish its own side effects inside perform (a call to another system, a row written). What JoobQ itself does for it is switched off: it is not retried, dead-lettered, counted as processed or failed, announced, or removed from the processing list, so the requeued copy is the only one JoobQ schedules. Workers also claim nothing once the shutdown has begun.

Step 5 is the shutdown's own recovery: it does not rely on the start-up recovery of the next process (Queue#start and the reaper reclaim the processing lists of crashed or stopped processes, never a live one's). A job put back this way runs again from the start, possibly in another process while this one winds down: jobs must be safe to run twice, as they already must be after a crash. A job that crashes the process before step 5 (SIGKILL) is still left to the start-up recovery.

Safe to call from any fiber, any number of times: the first call does the work and returns true, later calls (also while the first is still running) wait for it and return false. After a shutdown the queues of this process cannot be started again.

True once shutdown has finished.

False when the environment variable JOOBQ_TRAP_SIGNALS is 0, false or no.

SIGTERM, SIGINT and SIGHUP (Process.on_terminate covers all three): shut down, then exit with status 0, or 3 when jobs had to be abandoned at the deadline. A second signal more than a second after the first (an impatient operator; a duplicate sent by a process group or a wrapper arrives together with the first and is ignored) exits at once with status 1.