API Reference#
This page contains auto-generated API documentation for Oban Pro.
Workflow#
- class oban_pro.Workflow#
A workflow composes jobs with dependency relationships.
Workflows are built using method chaining:
workflow = ( Workflow(name="etl") .add("extract", ExtractWorker.new({})) .add("transform", TransformWorker.new({}), deps=["extract"]) .add("load", LoadWorker.new({}), deps=["transform"]) ) jobs = await oban.insert_all(workflow)
- __init__(*, id=None, name=None, check_deps=True, ignore_cancelled=False, ignore_deleted=False, ignore_discarded=False, unique=False)#
Create a new workflow.
- Parameters:
id (
str|None) – Custom workflow ID. Auto-generated UUIDv7 if not provided.name (
str|None) – Optional name for categorizing workflows.check_deps (
bool) – Validate that dependencies exist when adding jobs.ignore_cancelled (
bool) – Treat cancelled deps as completed.ignore_deleted (
bool) – Treat deleted/missing deps as completed.ignore_discarded (
bool) – Treat discarded deps as completed.unique (
bool) – Reject enqueues whosenamematches another non-completed unique workflow. Requiresname.
- classmethod for_job(job)#
Create a Workflow instance from a job that belongs to a workflow.
This is useful for fetching workflow data when you only have a job instance, not the original workflow.
- Parameters:
job (
Job) – A job that belongs to a workflow (has workflow_id in meta).- Return type:
- Returns:
A Workflow instance with the same ID.
Example
>>> workflow = Workflow.for_job(some_job) >>> all_jobs = await workflow.all_jobs()
- classmethod apply_graft(job, sub_workflow)#
Attach a sub-workflow to a previously defined grafting point.
This must be called from within a grafter job’s process() method. The grafter job must have been added using add_graft().
- Parameters:
job (
Job) – The currently executing grafter job.sub_workflow (
Workflow) – The sub-workflow to attach at this graft point.
- Return type:
- Returns:
A Workflow ready to be inserted with oban.enqueue_many().
- Raises:
ValueError – If the job is not a grafter job (missing graft metadata).
Example
>>> @worker() ... class GrafterWorker: ... async def process(self, job): ... sub = Workflow().add_many(...) ... await oban.enqueue_many(Workflow.apply_graft(job, sub))
- classmethod append(jobs)#
Create a workflow from existing workflow jobs for appending new jobs.
This is used within a job’s process() method to extend a workflow by adding more jobs dynamically. The returned workflow preserves the original workflow’s ID and ancestor chain.
- Parameters:
jobs (
list[Job] |Job) – A job or list of jobs from the workflow to extend.- Return type:
- Returns:
A Workflow instance ready for adding new jobs.
Example
>>> @worker() ... class AppenderWorker: ... async def process(self, job): ... jobs = await Workflow.for_job(job).all_jobs() ... workflow = Workflow.append(jobs) ... workflow.add_many("dynamic", [...]) ... await oban.enqueue_many(workflow)
- add(name, job, *, deps=(), ignore_cancelled=None, ignore_discarded=None)#
Add a named job to the workflow with optional dependencies.
- Parameters:
name (
str) – Unique name for this job within the workflow.job (
Job) – The job to add.deps (
str|Sequence[str]) – Names of jobs this one depends on. Single string or list.ignore_cancelled (
bool|None) – Per-job override for cancelled dependency handling.ignore_discarded (
bool|None) – Per-job override for discarded dependency handling.
- Return type:
- Returns:
Self for method chaining.
- Raises:
ValueError – If the name is already used, deps reference unknown jobs, or the job belongs to a chain.
Example
workflow.add(“a”, WorkerA.new({})) workflow.add(“b”, WorkerB.new({}), deps=”a”) workflow.add(“c”, WorkerB.new({}), deps=[“a”, “b”])
- add_workflow(name, sub_workflow, *, deps=())#
Add a sub-workflow with optional dependencies on parent jobs.
Jobs in the sub-workflow that have no dependencies will inherit the parent deps, causing them to wait for those parent jobs to complete.
- Parameters:
- Return type:
- Returns:
Self for method chaining.
- Raises:
ValueError – If the name is already used or deps reference unknown jobs.
Example:
sub = ( Workflow() .add("x", WorkerX.new({})) .add("y", WorkerY.new({}), deps=["x"]) ) workflow = ( Workflow() .add("a", WorkerA.new({})) .add_workflow("sub", sub, deps=["a"]) .add("b", WorkerB.new({}), deps=["sub"]) )
- add_many(name, jobs, *, deps=())#
Add multiple jobs as a sub-workflow with optional dependencies.
This is a convenience method that creates a sub-workflow from a collection of jobs. Jobs in a list are auto-named by index (0, 1, 2, …). Jobs in a dict use their keys as names.
- Parameters:
- Return type:
- Returns:
Self for method chaining.
Example:
# Fan-out with auto-named jobs (0, 1, 2, ...) workflow = ( Workflow() .add("setup", SetupWorker.new({})) .add_many("process", [ProcessWorker.new(item) for item in items], deps=["setup"]) .add("finish", FinishWorker.new({}), deps=["process"]) ) # Fan-out with named jobs workflow = ( Workflow() .add_many("fetch", { "users": FetchWorker.new({"table": "users"}), "orders": FetchWorker.new({"table": "orders"}), }) )
- add_graft(name, job, **kwargs)#
Add a grafting placeholder that will attach a sub-workflow at runtime.
A graft job acts as a placeholder in the workflow. When the grafter job executes, it must call Workflow.apply_graft() to attach a sub-workflow. Downstream jobs that depend on the graft will wait for the entire grafted sub-workflow to complete.
- Parameters:
name (
str) – Unique name for this graft point within the workflow.job (
Job) – The grafter job that will build and attach the sub-workflow.**kwargs – Passed through to add() (deps, ignore_cancelled, etc.)
- Return type:
- Returns:
Self for method chaining.
Example
>>> @worker() ... class GrafterWorker: ... async def process(self, job): ... user_ids = await get_user_ids(job.args["account_id"]) ... sub = Workflow() ... for uid in user_ids: ... sub.add(str(uid), NotifyWorker.new({"user_id": uid})) ... await Workflow.apply_graft(job, sub) ... >>> workflow = ( ... Workflow() ... .add_graft("users", GrafterWorker.new({"account_id": 123})) ... .add("summary", SummaryWorker.new({}), deps=["users"]) ... )
- add_cascade(name, func_or_tuple, **kwargs)#
Add a cascade function that receives context and auto-records results.
Cascades are functions that execute as workflow jobs, receiving a merged context dict containing workflow context and recorded values from deps.
Return values are automatically recorded.
- Parameters:
name (
str) – Unique name for this cascade within the workflow.func_or_tuple (
FunctionType|tuple[list|dict,FunctionType]) – Either: - A function taking (context) -> result - A tuple of (items, func) where func takes (item, context) -> result**kwargs – Passed through to add() (deps, ignore_cancelled, etc.)
- Return type:
- Returns:
Self for method chaining.
Example
>>> async def extract(context): ... return {"data": fetch_data(context["source"])} ... >>> async def transform(context): ... return process(context["extract"]["data"]) ... >>> async def load(context): ... save(context["transform"]) ... >>> workflow = ( ... Workflow() ... .add_context({"source": "api"}) ... .add_cascade("extract", extract) ... .add_cascade("transform", transform, deps=["extract"]) ... .add_cascade("load", load, deps=["transform"]) ... )
Fan-out with items:
>>> async def process_user(user_id, context): ... return {"processed": user_id} ... >>> workflow = ( ... Workflow() ... .add_cascade("users", ([1, 2, 3], process_user)) ... .add_cascade("summary", summarize, deps=["users"]) ... )
- add_context(value)#
Store shared context data for the workflow.
Context is stored as an already completed job and is available through get_context() or automatically within cascades. Jobs should never depend on it.
- Parameters:
- Return type:
- Returns:
Self for method chaining.
Example:
workflow = ( Workflow() .add_context({"user_id": 123, "batch_size": 100}) .add("process", ProcessWorker.new({})) )
- async get_job(name, *, with_subs=False)#
Fetch a single job from the workflow by name.
- Parameters:
- Return type:
Job|None- Returns:
The job if found, None otherwise.
Example
>>> job = await workflow.get_job("extract")
- async all_jobs(*, names=None, deps_of=None, with_subs=False)#
Fetch all jobs belonging to the workflow.
- Parameters:
names (
list[str] |None) – Filter to only return jobs with these names.deps_of (
Job|None) – Filter to only return jobs that are dependencies of this job.with_subs (
bool) – Also include jobs from sub-workflows of this workflow. When combined with names, each name matches either the job’s name or its enclosing sub_name.
- Return type:
list[Job]- Returns:
A list of jobs ordered by id.
Example
>>> jobs = await workflow.all_jobs() >>> jobs = await workflow.all_jobs(names=["a", "b"]) >>> jobs = await workflow.all_jobs(deps_of=job) # Get deps of job >>> jobs = await workflow.all_jobs(with_subs=True) # Include sub-workflow jobs
- async get_recorded(name)#
Fetch the recorded return value from a workflow job.
- Parameters:
name (
str) – The job name to fetch the recorded value from.- Return type:
- Returns:
The decoded recorded value, or None if the job hasn’t recorded a value.
Example
>>> result = await workflow.get_recorded("extract")
- async all_recorded(*, names=None, deps_of=None)#
Fetch all recorded return values from workflow jobs.
- Parameters:
- Return type:
- Returns:
A dict mapping job names to their recorded values. Jobs without recorded values are excluded. Results from sub-workflows other than this one are nested under their sub_name key.
Example
>>> results = await workflow.all_recorded() >>> results = await workflow.all_recorded(names=["a", "b"]) >>> results = await workflow.all_recorded(deps_of=job)
- async get_context()#
Retrieve the workflow’s context data.
Example
context = await workflow.get_context()
- async status()#
Get the current status of the workflow and all sub-workflows.
- Return type:
- Returns:
A WorkflowStatus with state, counts, timing, and nested sub-workflow statuses.
Example
>>> status = await workflow.status() >>> print(status.state) # 'executing', 'completed', 'cancelled', 'discarded' >>> print(status.counts) # {'completed': 5, 'executing': 2, ...}
- async cancel_jobs(*, names=None, with_subs=True)#
Cancel jobs in the workflow.
Jobs are marked as cancelled. Only jobs that aren’t already in a final state (completed, cancelled, discarded) can be cancelled. Held jobs that depend on cancelled jobs are cancelled as well, unless the workflow ignores cancellations.
For executing jobs, the database state is updated immediately, but running tasks are not forcefully terminated. Workers should check for cancellation at safe points.
- Parameters:
- Return type:
- Returns:
The number of jobs cancelled.
Example
>>> count = await workflow.cancel_jobs() >>> count = await workflow.cancel_jobs(names=["a", "b"])
- async signal(name_or_names, payload)#
Deliver a signal to one or more named jobs in this workflow.
Resolves workflow job names to ids, then delivers the signal. See
oban_pro.signal()for the underlying semantics. Jobs paused insideoban_pro.await_signal()resume immediately when signaled.- Parameters:
- Return type:
Example
>>> await workflow.signal("approval", {"decision": "approved"}) >>> await workflow.signal(["reviewer_a", "reviewer_b"], {"vote": "yes"})
- async retry_jobs(*, names=None, with_subs=False)#
Retry jobs in the workflow.
Jobs currently available or executing are ignored. Jobs without dependencies are set to available, while jobs with dependencies are set to suspended to wait for their dependencies to complete. Retried jobs whose dependencies already completed are released immediately.
- Parameters:
- Return type:
- Returns:
The number of jobs retried.
Example
>>> count = await workflow.retry_jobs() >>> count = await workflow.retry_jobs(names=["a", "b"])
- to_graph()#
Build a Graph representation of the workflow.
Creates a directed graph with jobs as vertices and dependencies as edges. Sub-workflows are represented as nested graphs, and edges to or from a sub-workflow use its name. A grafter job stands in for its graft. This is the foundation of the visualization methods, to_dot() and to_mermaid().
- Return type:
Graph- Returns:
A Graph with vertices and edges representing the workflow structure.
Example
>>> graph = workflow.to_graph() >>> print(graph.vertices) # {'a': 'WorkerA', 'b': 'WorkerB'} >>> print(graph.edges) # [('a', 'b')]
- to_dot()#
Convert the workflow to Graphviz DOT format.
- Return type:
- Returns:
A string containing the workflow in DOT format.
Example
>>> print(workflow.to_dot()) digraph { compound=true; node [shape=box]; n1 [label="a (WorkerA)"]; n2 [label="b (WorkerB)"]; n1 -> n2; }
- class oban_pro.workflow.WorkflowStatus#
Status of a workflow and its sub-workflows, returned by Workflow.status().
- id#
The workflow ID.
- name#
The workflow name, if set.
- state#
Overall state: executing, completed, cancelled, discarded, or unknown.
- total#
Total number of jobs.
- counts#
Job counts by state.
- duration#
Time from start to completion, or None while executing.
- started_at#
When the first job was inserted.
- stopped_at#
When the last job finished.
- subs#
Sub-workflow statuses, keyed by sub-workflow name.
Batch#
- class oban_pro.Batch#
A batch links many jobs as a group and runs callbacks after they’re processed.
Batches are built with method chaining and enqueued like any other list of jobs:
batch = ( Batch(name="nightly", callback_worker=NotifyWorker) .add([EmailWorker.new({"email": email}) for email in emails]) .add([PushWorker.new({"user_id": id}) for id in user_ids]) ) jobs = await oban.enqueue_many(batch)
- __init__(*, id=None, name=None, callback_worker=None, callback_opts=None)#
Create a new batch.
- Parameters:
id (
str|None) – Custom batch ID. Auto-generated UUIDv7 if not provided.name (
str|None) – Optional label describing the batch. Unlike the id, it needn’t be unique.callback_worker (
type|str|None) – Worker that runs callbacks for the batch. Defaults to the worker of whichever batch job triggers the callback, so it’s required for batches that mix workers.callback_opts (
dict[str,Any] |None) – Job options for callback jobs. Only args, max_attempts, meta, priority, queue, and tags are allowed.
- classmethod for_job(job)#
Create a Batch instance from a job that belongs to a batch.
Example
>>> jobs = await Batch.for_job(job).all_jobs()
- Return type:
- Parameters:
job (Job)
- classmethod append(job, **overrides)#
Create a batch from an existing batch job for adding more jobs.
The batch id and options are carried over from the job, unless overridden.
Callback jobs are only inserted once. Appending to a batch whose callbacks were already triggered, e.g. batch_completed, won’t trigger them again.
Example
>>> batch = Batch.append(job).add(EmailWorker.new({"email": email})) >>> await oban.enqueue_many(batch)
- Return type:
- Parameters:
job (Job)
- classmethod from_workflow(workflow, **opts)#
Create a batch from a workflow, so workflow jobs also form a batch.
Example
>>> workflow = Workflow().add("a", A.new()).add("b", B.new(), deps=["a"]) >>> await oban.enqueue_many(Batch.from_workflow(workflow, callback_worker=Notify))
- add(jobs)#
Add one or more jobs to the batch.
Example
>>> batch = Batch().add(EmailWorker.new({"email": email})) >>> batch = Batch().add([EmailWorker.new({"email": email}) for email in emails])
- async all_jobs()#
Fetch all jobs in the batch, excluding callback jobs.
- Return type:
list[Job]- Returns:
A list of jobs ordered by id.
Example
>>> async def batch_completed(self, job): ... jobs = await Batch.for_job(job).all_jobs() ... account_ids = [job.args["account_id"] for job in jobs]
- async cancel_jobs()#
Cancel all jobs in the batch, excluding callback jobs.
Jobs are cancelled with a bulk update, and any batch callbacks that are ready afterwards, e.g. batch_cancelled, are inserted.
- Return type:
- Returns:
The number of jobs cancelled.
Example
>>> await Batch.for_job(job).cancel_jobs()
Chunk#
Results for chunk workers, which process jobs together in groups.
Chunk workers are declared with the chunk option and define process_chunk rather than process. Return one of these results, or a list of them, to apply an outcome to specific jobs in the chunk. Any jobs that aren’t listed are completed.
Example
>>> from oban import worker
>>> from oban_pro import chunk
>>>
>>> @worker(queue="messages", chunk={"size": 100, "wait": 5})
... class SendMessages:
... async def process_chunk(self, jobs):
... responses = await send_batch([job.args for job in jobs])
... bad_token = [job for job, resp in zip(jobs, responses) if resp.bad_token]
...
... return chunk.Cancel("bad token", bad_token)
- class oban_pro.chunk.Cancel#
Cancel the listed jobs in the chunk.
Example
>>> return chunk.Cancel("bad token", [job_1, job_2])
- exception oban_pro.chunk.ChunkError#
Recorded for jobs failed with an Error result.
- class oban_pro.chunk.Error#
Fail the listed jobs in the chunk so they’re retried, or discarded on their final attempt.
Example
>>> return chunk.Error("delivery failed", [job_3])
Signals#
- async oban_pro.signal(target, payload, *, oban=None)#
Deliver a signal to one or more jobs by id.
The payload is encoded as an erlang term and merged into the target jobs’ meta under the
"signal"key. If a job is parked insideawait_signal(), itsscheduled_atis bumped tonow()so it resumes immediately.Signals are fire-and-forget. Delivery to a job in a terminal state, or a job that doesn’t exist, is a no-op.
- Parameters:
- Return type:
Example
>>> await oban_pro.signal(job_id, {"decision": "approved"}) >>> await oban_pro.signal([job_a, job_b], {"decision": "approved"})
- async oban_pro.await_signal(*, wait_for=None, wait_timeout=5.0)#
Pause the current job until a matching signal arrives.
Must be called from within an executing job’s
processmethod. Returns the decoded payload when a signal is received. RaisesSignalTimeoutwhen the overallwait_fordeadline elapses.A short live wait blocks during execution so signals that arrive immediately resume in place. Past that window the job snoozes until either a signal arrives or the deadline elapses, freeing the worker process. Signals are persisted in meta, so one delivered before
await_signalis reached is consumed on the next call.- Parameters:
wait_for (
float|timedelta|None) – Total time to wait for a signal, measured from the first snooze. Accepts seconds as a float, atimedelta, orNonefor an indefinite wait. The deadline is persisted on first snooze so subsequent retry attempts honor the same wall-clock deadline.wait_timeout (
float) – Seconds to block during execution before snoozing. Pass0to skip the live wait and snooze immediately. Defaults to 5.0.
- Return type:
- Returns:
The decoded signal payload.
- Raises:
SignalTimeout – The overall wait_for deadline elapsed.
RuntimeError – Called outside of an executing job.
Example
>>> async def process(self, job): ... try: ... payload = await oban_pro.await_signal(wait_for=1800) ... return proceed(payload) ... except oban_pro.SignalTimeout: ... return Cancel("no decision")
- exception oban_pro.SignalTimeout#
Raised by
await_signal()when the overall wait_for deadline elapses.
Relay#
- exception oban_pro.RelayError#
Base exception for Relay errors.
- exception oban_pro.RelayCancelled#
Raised when job is cancelled, either by the worker or through the API.
- exception oban_pro.RelayDiscard#
Raised when job is discarded after exhausting max attempts.
- exception oban_pro.RelayTimeout#
Raised when relay times out waiting for job completion.