Introduction
Background work usually means standing up a broker like Redis or RabbitMQ and a worker process to match. This post builds a working job pipeline from three small aux4 packages instead: aux4/queue carries the messages, aux4/jobs runs each one in the background with history and failure hooks, and aux4/cron schedules the drain. Our example: a queue of image-resize tasks.
Install the packages
aux4 aux4 pkger install aux4/queue aux4/jobs aux4/cron
Step 1: start the queue and publish messages
aux4/queue is a small in-memory pub/sub server. Start it, then create a named queue:
aux4 queue start
queue server started on port 8420
aux4 queue create --name resize
{ "name": "resize", "status": "CREATED" }
Now the one thing you must understand about this queue: it's pub/sub, not a stored backlog. A message is delivered to whoever is subscribed right now — if nobody is listening, it's gone. Watch it happen. Send two messages with no consumer connected:
printf '{"file":"a.png"}\n{"file":"b.png"}\n' | aux4 queue send --name resize
{ "errors": 0, "sent": 2 }
send cheerfully reports sent: 2, but try to receive them a moment later and nothing comes back — they were never buffered. So the rule is connect the consumer first, then produce. Start a receiver (in another shell) that stops after three messages:
aux4 queue receive --name resize --limit 3
receive blocks, waiting for messages; --limit 3 makes it exit cleanly after three. With it listening, publish:
printf '{"file":"a.png"}\n{"file":"b.png"}\n{"file":"c.png"}\n' | aux4 queue send --name resize
The receiver prints them in order and exits:
{"file":"a.png"}
{"file":"b.png"}
{"file":"c.png"}
Step 2: run work as background jobs
aux4/jobs runs a command in the background and remembers it. jobs run returns immediately:
aux4 jobs run "echo processing a.png && sleep 1 && echo done"
{ "id": "1", "command": "echo processing a.png && sleep 1 && echo done", "state": "RUNNING" }
The real value is what happens on failure. --onFailure fires a hook when the command exits non-zero — an alert, a retry, a log line. Run one that fails:
aux4 jobs run "false" --onFailure "echo ALERT: resize failed >> alerts.log"
{ "id": "2", "command": "false", "state": "RUNNING", "onFailure": "echo ALERT: resize failed >> alerts.log" }
Every run is recorded, so jobs list is your history — with final state and exit code:
aux4 jobs list
[
{ "id": "1", "command": "echo processing a.png && sleep 1 && echo done", "state": "COMPLETED", "exitCode": 0 },
{ "id": "2", "command": "false", "state": "FAILED", "exitCode": 1 }
]
And the failure hook ran:
cat alerts.log
ALERT: resize failed
jobs output 1 replays a job's stdout — processing a.png / done. Failures never get lost silently; they're logged, flagged, and can trigger a hook.
Step 3: drain the queue into jobs
Now wire the two together: a consumer that receives messages and dispatches a job per message. Save drain.sh:
#!/bin/sh
# receive N messages, dispatch a job to process each
aux4 queue receive --name resize --limit "${1:-3}" | while IFS= read -r msg; do
file=$(echo "$msg" | jq -r '.file')
aux4 jobs run "echo resized $file" --source drain >/dev/null
echo "dispatched job for $file"
done
Start the drain (the consumer), then publish three messages to it:
sh drain.sh 3 &
printf '{"file":"x.png"}\n{"file":"y.png"}\n{"file":"z.png"}\n' | aux4 queue send --name resize
dispatched job for x.png
dispatched job for y.png
dispatched job for z.png
The --source drain tag lets you isolate exactly the jobs this consumer created:
aux4 jobs list --source drain
[
{ "id": "3", "command": "echo resized x.png", "state": "COMPLETED", "exitCode": 0, "source": "drain" },
{ "id": "4", "command": "echo resized y.png", "state": "COMPLETED", "exitCode": 0, "source": "drain" },
{ "id": "5", "command": "echo resized z.png", "state": "COMPLETED", "exitCode": 0, "source": "drain" }
]
Messages became tracked background jobs — each with output, state, and an exit code you can audit later.
Step 4: schedule the drain
The last piece is running the drain on a schedule so the queue empties itself. aux4/cron is a small scheduler daemon — start it, then add an entry. The command to run goes in --run:
aux4 cron start
cron scheduler started on port 8421
aux4 cron add --name drain-resize --every "1 min" --run "sh /tmp/blog-queue/drain.sh 3"
{ "name": "drain-resize", "every": "1 min", "run": "sh /tmp/blog-queue/drain.sh 3", "state": "active" }
aux4 cron list
[
{ "name": "drain-resize", "every": "1 min", "run": "sh /tmp/blog-queue/drain.sh 3", "state": "active" }
]
Cron runs the drain every minute; each firing itself lands in the jobs history, so you have a full audit trail of every scheduled run. Stop everything when you're done:
aux4 cron stop
aux4 queue stop
Conclusion
Three small packages add up to a real background-work system: aux4/queue moves messages (pub/sub — connect the consumer before you produce), aux4/jobs runs each one with history and --onFailure hooks, and aux4/cron schedules the drain. No broker, no worker framework, no database — just composable commands you can inspect and audit at every step.