This release brings chains for running jobs in strict order, batches for grouping jobs with callbacks, and concurrency that's fully integrated with Oban Web. See the v0.8 Upgrade Guide for complete upgrade steps and migration caveats.
Chains
Chains link jobs together so they run in strict sequential order. A job in a chain won't run until
the job before it is in a final state, regardless of scheduling, snoozing, and retries. Declare a
chain with the chain option:
from oban import worker
@worker(queue="webhooks", chain={"by": {"args": "account_id"}})
class WebhookWorker:
async def process(self, job):
account = await Account.find(job.args["account_id"])
await account.handle_webhook_event(job.args["event"])
Here, chains are partitioned by account id, so webhooks for each account run in order while different accounts can still run concurrently. See the chain guide for the complete set of partitioning and holding options.
Batches
Batches link jobs together and run callbacks after jobs execute. That lets you coordinate jobs running in parallel, then react once they've all finished, or as soon as one of them fails. Workers define optional batch callbacks:
from oban import worker
from oban_pro import Batch
@worker(queue="mailers")
class EmailWorker:
async def process(self, job):
await mailer.daily_update(job.args["email"])
async def batch_completed(self, job):
jobs = await Batch.for_job(job).all_jobs()
await mailer.notify_admins([job.args["email"] for job in jobs])
batch = Batch().add([EmailWorker.new({"email": email}) for email in emails])
await oban.enqueue_many(batch)
Here admins will be notified only when the batch completed, though finer grained callbacks are also available. See the batch guide for the full list of callbacks, hybrid batches, and appending to or cancelling a batch.
Smart Concurrency with Oban Web
Smart concurrency now works seamlessly with Oban Web in either direction. Queues report global limits, rate limits, and partitions the way Web expects. Queues can now combine a global limit with a rate limit, and must declare partitioning for that particular limit.
# Run 1 job at a time for each account, and at most 100 jobs per minute for the whole queue
[queues.exports]
limit = 10
global_limit = { allowed = 1, partition = { args = "account_id" } }
rate_limit = { allowed = 100, period = 60 }
Limits can be scaled at runtime from Python as well, and passing None clears one entirely:
await oban.scale_queue(queue="exports", rate_limit={"allowed": 200, "period": 60})
await oban.scale_queue(queue="exports", rate_limit=None)