Changelog for Oban Pro v1.8

This release introduces saga-style workflow compensations, cursor-based backfills, inline cron scheduling, external recorded storage, persistent pruning rules with automatic archiving, and unified configuration for queues and services.

See the v1.8 Upgrade Guide for complete upgrade steps and migration caveats.

↩️ Workflow Compensations

Workflows now offer compensations, where completed jobs are automatically reversed after a workflow is cancelled or discarded. Compensations run as dedicated jobs in reverse dependency order, including for sub-workflows, grafts, and other dynamically added jobs.

Compensations can be declared explicitly, as a function capture passed as a compensate option for any workflow shape, including cascades:

Workflow.new()
|> Workflow.add_cascade(:hold, &Inventory.reserve/1, compensate: &Inventory.release/1)
|> Workflow.add_cascade(:charge, &Billing.charge/1, deps: :hold, compensate: &Billing.refund/1)

Alternatively, worker-module style workflows that declare a compensate/1 callback will be compensated automatically:

@impl Oban.Pro.Workflow
def compensate(%Job{args: args}) do
  Billing.refund(args["charge_id"])
end

By default, compensations run after a workflow is cancelled or discarded. They run as a separate but linked workflow using standard Oban jobs, with all the normal recovery semantics. As with job execution, compensations are at-least-once, but they must be idempotent.

🛻 Cursor-Based Backfills

The new Oban.Pro.Backfill module works through datasets of any size to retroactively fill in missing data, reprocess records, or apply corrections. A backfill is composed of a chain of small jobs, where each operates on a paginated slice of records. The design avoids long-running migrations or large transactions that can lock up a database.

Create and customize a backfill module with the Oban.Pro.Backfill behaviour. It can walk through any ordered table (bigint and uuid alike), in ascending or descending order, and with optional throttling to minimize load on the database or external services.

defmodule MyApp.UserBackfill do
  use Oban.Pro.Backfill, queue: :backfills, limit: 1_000

  import Ecto.Query

  @impl Oban.Pro.Backfill
  def backfill(cursor, _extra) do
    MyApp.User
    |> where([user], is_nil(user.migrated_at))
    |> Cursor.update_all(cursor, set: [migrated_at: ^DateTime.utc_now()])
  end
end

The pagination-aware Cursor helpers cover common backfill tasks such as bulk database changes, iterating over individual rows, and processing rows as a batch; like so:

@impl Oban.Pro.Backfill
def backfill(cursor, _extra) do
  with {users, next} <- Cursor.fetch(MyApp.User, cursor) do
    MyApp.CRM.sync_all(users)

    {:cont, next, length(users)}
  end
end

Once defined, you can start a backfill like any other job, or directly from within an Ecto migration:

Oban.Pro.Backfill.start(MyApp.UserBackfill)

Calling Backfill.start/1 in a migration doesn't require the application to be started, and it's context-aware: your database won't get polluted with backfill jobs while testing.

⚙️ Streamlined Configuration

Pro's engine and core services now use shorter, slightly less bombastic, flatter module names. There's only one engine, and those are the definitive Pro implementations of each service, so simplify the naming:

Oban.Pro.Engines.Smart becomes Oban.Pro.Engine.
Oban.Pro.Plugins.DynamicCron becomes Oban.Pro.Cron.
Oban.Pro.Plugins.DynamicLifeline becomes Oban.Pro.Lifeline.
Oban.Pro.Plugins.DynamicPruner becomes Oban.Pro.Pruner.
Oban.Pro.Plugins.DynamicQueues becomes Oban.Pro.Queues.

These renames match similar changes made in Oban v2.24. You can also move configuration out of the generic :plugins list and utilize dedicated configuration keys:

config :my_app, Oban,
  engine: Oban.Pro.Engine,
  cron: {Oban.Pro.Cron, crontab: [...]},
  lifeline: Oban.Pro.Lifeline,
  pruner: Oban.Pro.Pruner,
  queues: {Oban.Pro.Queues, queues: [default: 10]}

For backward compatibility, the previous modules remain as (deprecated) shims and old configuration stays entirely valid, so you can adopt the new config style incrementally.

Global Worker Defaults

Also in the vein of simplified configuration, global defaults can be configured for select job options, recorded output and storage, and the new encrypted argument keyrings.

config :oban_pro, Oban.Pro.Worker,
  max_attempts: 5,
  priority: 1,
  queue: :default,
  recorded: [storage: MyApp.S3, limit: {32, :mb}]

These are defaults only. Worker options declared with use Oban.Pro.Worker or passed to new/2 take precedence, while options such as recorded and encrypted merge with the global defaults.

Defaults are validated at compile time, and configuration changes take effect when workers are recompiled.

⏰ Inline Cron Scheduling

Pro workers can now declare cron schedules inline, along with the actual code being scheduled, using the new :cron option:

defmodule MyApp.NightlyCleanup do
  use Oban.Pro.Worker, cron: "0 3 * * *"
  ...
end

Inline cron can also be applied to decorated functions:

use Oban.Pro.Decorator

@job cron: "@daily"
def rollup do
  ...
end

It's also possible to pass advanced options like timezone, guaranteed, paused, or a custom name using the keyword version:

use Oban.Pro.Worker,
  cron: [expression: "0 3 * * *", name: "custom-name", guaranteed: true],
  ...

Annotated workers and decorated jobs are collected at compile-time via a custom compiler, then discovered by Oban.Pro.Cron on boot. This avoids relying on loaded modules, which is especially important in development or test where lazy loading alters the load order.

☁️ External Recorded Storage

Recorded job output doesn't have to live on the jobs table anymore. A new Oban.Pro.Storage behaviour lets applications offload recorded data to external storage such as S3, Tigris, Redis, etc.

Storage can be configured for an individual worker, with localized options:

use Oban.Pro.Worker,
  recorded: [storage: {MyApp.S3, bucket: "job-output"}, limit: {32, :mb}]

Or at the workflow level, as an option to Workflow.new/1:

Oban.Pro.Workflow.new(storage: {MyApp.S3, bucket: "workflow-output"})

And finally, as a global default for all recorded workers, including workflow cascades:

config :oban_pro, Oban.Pro.Worker,
  recorded: [storage: MyApp.S3, limit: {32, :mb}]

Stored output is fetched transparently by Worker and Workflow functions. Then, when jobs are pruned, the Oban.Pro.Pruner asks the storage backend to automatically delete stored data (though lifecycle rules and TTLs are recommended).

Beyond the storage extension, recorded metadata now includes the encoded payload size, and fetch_recorded/2 can retrieve output without the original module or decoding anything. All of which are improvements that allow Oban Web to inspect arbitrary recorded jobs and warn before downloading massive objects.

🧹 Persistent Pruning and Archiving

Oban.Pro.Pruner gained persisted, runtime-editable, rules to control job retention. Rules can match any combination of queue, worker, and state, and retain jobs by age or count:

pruner: {
  Oban.Pro.Pruner,
  rules: [
    [name: "events", queue: :events, max_age: {10, :minutes}],
    [name: "discarded", state: :discarded, max_age: {7, :days}],
    [name: "audit", worker: MyApp.AuditWorker, max_len: :infinity]
  ]
}

Rules have explicit ordering, so the sequence is fully deterministic - much better than relying on the limited and implicit ordering of overrides. A final default rule handles jobs that no earlier rule matched.

Because rules are persisted, they can be inserted, modified, or even paused at runtime. Like the Cron and Queues modules, there are two sync_mode options to either preserve dynamic rules, or treat application configuration as the source of truth.

Automatic Archiving

The final Pruner addition is declarative job archiving. Rules can be configured with archive: true to keep jobs in an archive table rather than deleting them:

[name: "audit", worker: MyApp.AuditWorker, max_age: {1, :week}, archive: true]

Jobs pruned the rule are copied in full to an oban_jobs_archive table prior to deletion. The copy and delete operations happen in a single transaction, so jobs can't be lost between the two tables.

Archived rows can be queried through Oban.Pro.Archive, and they load as vanilla Oban.Job structs:

[worker: MyApp.AuditWorker, state: "completed"]
|> Oban.Pro.Archive.query()
|> Oban.all_jobs()

v1.8.0 — 2026-09-16

Enhancements

[Backfill] Support snoozing like any other job
Bring backfill jobs in line with other jobs by allowing the backfill/1 callback to return {:snooze, period} to reschedule.
[Backfill] Add backfill boolean marker to job meta
All composition primitives except for backfills had a boolean marker in meta, e.g. chain: true. Backfills now include backfill: true for consistency and easier filtering in Web.
[Chunk] Record the overall chunk size in the leader's meta
Chunk leaders were only identifiable from the attempted_by array (not filterable, degrades on retry). The leader's meta now gets a chunk_count after fetching the chunk.
[Migration] Add indexes to the archived jobs table
The archive table was created without additional indexes, which made browsing in Web a seq scan. This adds partial indexes and optimized GIN indexes to make filtering archived jobs optmized.
[Migration] Tune autovacuum for the producers table
Producer rows are updated frequently, but the table size doesn't change often. The default autovacuum thresholds, which scale with table size, can leave the table bloated slow fetching.
The v1.8.0 migration now tunes autovacuum for oban_producers so vacuum and analyze run when there are a modest number of dead tuples.
[Lifeline] Split prune scanning from deletion
Pruning ran selection, archiving, storage reaping, and deletion in a single transaction. On large tables, connections held that long may be severed under pool contention.
Jobs are now selected before the transaction, then archived, reaped, and deleted in shorter batches.
[Pruner] Preserve runtime rule changes when Pruner initializes
Some rule changes made at runtime were incorrectly overwritten on startup. Rules now store a fingerprint, the way queues and crons operate. That makes it so runtime changes persist across restarts, or until rule config actually changes.
The oban_pruners table now requires a hash column. The v1.8.0 migration creates it, but the table created by earlier release candidates doesn't have it. If you ran the migration on an earlier v1.8.0 release candidate, add the column with a migration before upgrading (see the v1.8 upgrade guide):
```
alter table(:oban_pruners) do
  add_if_not_exists :hash, :text
end
```
[Queue] Allow passing dispatch_cooldown in queue options
Oban accepts dispatch_cooldown as a per-queue option, but Pro Queues rejected it because it wasn't persisted. Now the option is validated, persisted, and passed through to the queue's producer when it starts. This makes it possible to set per-queue dispatch cooldowns and control fetch load at the database level.
[Storage] Retry failed storage writes with backoff applied
Transient exceptions and errors from Storage.put/3 are retried with backoff before failing the attempt. Retries reuse the same key, and the original exception is raised when retries are exhausted.

Changes

[Chunk] Add a dedicated process_chunk/1 callback
Chunk workers previously used the same process/1 callback as normal workers. Now, a dedicated process_chunk/1 callback is the preferred way to define chunk workers.
[Engine] Guard unique inserts with advisory locks when the index isn't unique
Uniqueness is still enforced by the unique index without any locks. On partitioned tables, or databases without partial unique indexes, inserts are now guarded by advisory locks instead of relying on the index alone.
Conflicted jobs are returned as the stored row and no longer have uniq_conflict: true written to their meta.

Bug Fixes

[Chain] Stop setting legacy hold and unique indicators on released chains
Releasing a chain always flipped the legacy on_hold boolean and unnecessarily cleared uniqueness when it wasn't required.
[Cron] Ensure an empty crontab for inline discovery
Without a crontab option, even an empty one, annotated cron discovery was skipped and entries weren't persisted to the database. Now a default empty crontab list is always set to prevent the issue.
[Cron] Load the cron manifest explicitly for discovery
The cron manifest is written after the app file during releases, so booting in embedded mode never loaded and annotated crons were silently skipped. The manifest is now explicitly loaded by path instead to ensure it works in releases.
[Cron] Label discovered decorated cron with handler meta
Decorated cron jobs now carry the same decorated and decorated_name meta as regular decorated jobs. Dashboards, like Web, can identify them by the function they call rather than the decorator worker.
[Diagnostics] Prevent diagnostics crashes from stopped instance
Diagnostics discovery subscribed to every registered Oban instance, including those in :manual or :inline testing mode. Attempting to listen on an instance that was shutting down crashed the diagnostics server with a nested exit. Now discovery skips testing instances entirely and gracefully handles stops.
This was only a problem for test suites that started and stopped Oban instances dynamically, e.g. Pro and Web.
[Engine] Flush acks recorded during shutdown
Stopping a queue flushed any pending acks before recording that the queue was paused. That allowed a race conditon between flushing existing acks and recording the paused state in the database, which could leak pending acks.
[Engine] Use unique index when clearing violations
Clearning a unique violation would use the general-purpose meta index when available. When that index was removed, as we suggest, it would fall back to a sequential scan. Now the proper unique index is used to clear conflicts instead.
[Engine] Fix token bucket on fractional refills
The token bucket truncated consumed tokens rather than available tokens during demand calculation, so a partial refill (e.g. 0.05 tokens after one second) rounded up to a full token.
Demand now floors available tokens and tracks fractional consumption, so admission matches the rate exactly.
[Lifeline] Keep held chains suspended across repairs
Chains configured with on_cancelled: :hold or on_discarded: :hold could be released by chain repair, because repair only considered active prior jobs blocking. On hold chains now stay suspended until the cancelled or discarded job is completed or deleted, as documented.
[Lifeline] Limit how many completed workflows can be pruned at a time
Completed workflows were pruned with an unbounded delete that ran on every node, regardless of leadership or schedule. Workflow pruning now deletes using the default rule's limit and timeout, and pruning only runs the leader.
[Refresher] Isolate producer refresh failures per instance
Producer refreshing and cleanup was wrapped in a single rescue clause. When an instance raised every instance after it was silently skipped and stale producers were never cleaned up.
Failures are rescued and logged per instance now, so other instances are still handled.
[Workflow] Remap sub-workflow dependencies during apply_graft/3 calls
Sub-workflows added with add_many or add_workflow inside a grafter that depended on a plain step, kept referencing the old workflow id, so the grafted jobs were errantly cancelled.
[Workflow] Exclude duplicate unique jobs from workflow counts
Workflow counting was based on all inserted jobs before unique deduplication. That drift could leave some workflows stuck with a phantom suspended or available job, and unable to finish.

v1.8.0-rc.1 — 2026-08-25

Bug Fixes

[Cron] Include the Oban Pro compiler in the release package
The compiler task was omitted from the package, causing projects configured with the :oban_pro compiler to fail with a missing task error.

v1.8.0-rc.0 — 2026-08-25

Enhancements

[Backfill] Add a module for for cursor-based dataset backfills
Backfill walks a dataset with recursive jobs, advancing a keyset cursor one window per job. Each iteration is a single Oban job that processes a batch and schedules its successor, so a backfill makes steady progress without holding a long transaction or running a long-lived process.
The cursor walks any unique, ordered column—integer or UUID alike—so the same backfill works across key types. Configure the key, batch limit, and order; throttle to pace the chain; bound each window with a timeout; or dry run to confirm a backfill terminates without persisting any writes.
Cursor helpers translate a query into the cursor's advance protocol: update_all/delete_all for in-place writes, and each/fetch for per-row or batched work such as external API and inference calls.
Trigger a backfill from a migration or console with start/2, and stop one with cancel/2. A dedicated index on backfill_id keeps lookups fast.
[Chunk] Store only customized options in chunk job meta
Chunk workers previously persisted the full set of chunk options to every job's meta, including defaults like size, sleep, timeout, and leading. Now only options explicitly set on the worker are stored, and defaults are merged in at processing time. This shrinks the meta written for each chunk job while preserving identical runtime behavior.
[Cron] Schedule jobs inline on workers and decorated functions
Add a :cron option to Oban.Pro.Worker and Oban.Pro.Decorator. The bare form accepts a cron expression, while the keyword form supports :expression, :timezone, :name, :guaranteed, and :paused. An :oban option scopes annotations to a particular Oban instance.
A Mix compiler records annotated workers and decorators in a per-app manifest that Oban.Pro.Cron loads at startup. Discovery no longer depends on which modules happen to be loaded, fixing missing entries in environments with lazy code loading.
Using cron: without adding :oban_pro to the project's compilers emits a compile-time warning.
[Diagnostics] Include executing pid in diagnostics reply
The diagnostics reply now carries executing job's pid alongside the existing process info.
[Engine] Add per_node option to scale global partitioned limits
Global limits are normally a fixed ceiling across all nodes. The new per_node: true modifier treats allowed as a per-node value that scales with the number of nodes running a queue.
It's intended for partitioned queues, where each partition stays bounded while the cluster's throughput grows with capacity. Without a partition it's equivalent to a local_limit of the same value.
[Lifeline] Add selective matching to retry_exhausted
The retry_exhausted option now accepts a simple match spec in addition to a boolean, so exhausted jobs can be retried for specific workers or queues. Pass workers and/or queues and any exhausted job that doesn't match is still discarded:
```
plugins: [
  {DynamicLifeline, retry_exhausted: [workers: [MyWorker], queues: [:safe]]}
]
```
Passing true or false behaves as before, retrying or discarding every exhausted job.
[Migration] Optimize chain and chunk indexes for large tables
Replace existing chain and chunk indexes with definitions tailored to their lookup patterns. Chain lookups avoid scanning completed jobs, while chunk fetching uses index order for priority and scheduling, keeping both operations fast as job tables grow.
[Pro] Promote dynamic plugins to top-level services
Oban v2.24 introduced dedicated cron, lifeline, pruner, and queue configuration keys, so the corresponding Pro modules no longer belong under :plugins and no longer need the Dynamic prefix to distinguish them:
- Oban.Pro.Plugins.DynamicCron → Oban.Pro.Cron
- Oban.Pro.Plugins.DynamicLifeline → Oban.Pro.Lifeline
- Oban.Pro.Plugins.DynamicPruner → Oban.Pro.Pruner
- Oban.Pro.Plugins.DynamicQueues → Oban.Pro.Queues
The old modules remain as shims so existing configuration and function calls keep working, with @deprecated annotations pointing at the new names. Nothing needs to change immediately, but the shims will be removed in a future major version.
[Migration] Drop indexes renamed by the v1.7 migration
The v1.7 migration renamed the indexes it replaced with an _old suffix to retain the originals. Those aren't needed now, and v1.8 drops the remaining indexes to reclaim space and database cycles.
[Pruner] Use persisted and runtime-updatable rules
Replace implicitly ordered queue, state, and worker overrides with database-backed pruning rules. Rules support compound matches, explicit first-match precedence, per-rule retention limits, pausing, and runtime management without restarting Oban.
Existing mode and override configuration is translated automatically for backward compatibility.
[Pruner] Add declarative job archiving
Rules can now set archive: true to preserve matched jobs instead of deleting them. Archived jobs are copied to a dedicated oban_jobs_archive table before they're removed from oban_jobs, retaining the job's full record (args, meta, errors, timestamps) for long-term storage and later inspection.
The copy runs in the same transaction as the delete, so a job is never lost between the two tables, and rows are moved with a single set-based insert without round-tripping through the application.
The oban_jobs_archive table mirrors oban_jobs and is created by the standard v1.8.0 migration. Read archived jobs back through the new Oban.Pro.Archive queryable, which loads rows as ordinary Oban.Job structs:
```
Repo.all(Oban.Pro.Archive)

[worker: MyApp.AuditWorker, state: "completed"]
|> Oban.Pro.Archive.query()
|> Oban.all_jobs()
```
Prune telemetry now includes an archived_count measurement alongside pruned_count.
[Worker] Add global overrides for pro worker defaults
Oban bakes in system-wide defaults for options like max_attempts (20), which previously could only be changed by setting the option on every worker. Pro workers can now override the defaults for max_attempts, priority, and queue across an entire application:
```
config :oban_pro, worker_defaults: [max_attempts: 5, priority: 1]
```
Defaults apply to any Pro worker that doesn't set the option itself, so values from use or passed to new/2 always take precedence. The config is read at compile time and validated against the allowed subset, so unknown or invalid options raise with a clear error.
[Worker] Add fetch_recorded/2 to enhance fetching from external storage
Define Worker.fetch_recorded/2 that is capable of fetching externally stored recordings without the worker module loaded, as well as optionally bypoassing decode. Recorded data also includes a size attribute as an external hint, so Web can alert users about the overal size before downloading and displaying large recordings.
[Worker] Support encrypted key rotation and authentication
Encrypted workers accept a :keyring module in place of a static :key. Each job records which key encrypted it, so keys can rotate without stranding jobs inserted beforehand. Define a keyring module, and configure it for a worker:
```
use Oban.Pro.Worker, encrypted: [keyring: MyApp.Keyring]
```
New jobs use aes_256_gcm rather than aes_256_ctr, authenticating args so tampering fails the job instead of decrypting to garbage. Existing jobs are still read with aes_256_ctr.
[Worker] Add on_retried/2 worker hook for retried jobs
Retrying a job marks it as available for future execution but doesn't run the worker immediately, so there was no execution hook that fired at the moment an operator retried a job.
Add on_retried/2 to complete the set of external state hooks. It's called with the :manual reason after a job's state is reset. As with the other external hooks, the callback runs against the fully loaded job (decryption and structuring applied) and exceptions are caught and logged without affecting the retry.
[Worker] Use a units syntax for recorded size limits
Add units syntax, {32, :mb}, with support for kb, mb, and gb units to simplify defining storage size limits.
[Worker] Add pluggable storage for recorded job output
Recorded output can now be written to and read from external object stores (S3, R2, Tigris, etc.) through the new Oban.Pro.Storage behaviour, rather than only living inline on a job.
Setting global worker defaults has moved to :oban_pro, Oban.Pro.Worker for consistency with other compile-time options, and setting a global recorded option is now also possible.
```
config :oban_pro, Oban.Pro.Worker, recorded: [storage: MyApp.S3, limit: 32_000_000]
```
[Workflow] Add saga-style workflow compensations
Steps declare compensation through an automatically detected compensate/1 worker callback or an explicit function capture. Declarations are serialized into job meta when the workflow is built, keeping behavior stable across deploys.
When a workflow terminates according to its compensate_on policy, Pro materializes a linked compensation workflow with completed steps arranged in reverse dependency order. Cascade compensations receive reconstructed context and recorded ancestry.
Workflow rescue repairs drifted counters and re-evaluates unresolved candidates so a crash can't strand pending compensation.
[Workflow] Expose compensations in workflow status
Rework status/1 to read from the workflow table rather than raw jobs, and add a new compensation field with linked workflow details.
Reading from the table is significantly faster, counts include the suspended state, durations are only reported once a workflow finishes, and unnamed workflows fall back to a single worker name.

Changes

[Engine] Rename the Smart engine to Oban.Pro.Engine
The engine is now Oban.Pro.Engine to match the flatter module names introduced for Pro's services. Update the :engine option in your config:
```
  config :my_app, Oban,
-   engine: Oban.Pro.Engines.Smart
+   engine: Oban.Pro.Engine
```
Oban.Pro.Engines.Smart still works and delegates all callbacks to the new module, so there's no rush to change it and no deprecation warnings.
[Worker] Drop the obsolete recorded to option
The option was never promoted into recorded meta or used at fetch time for bulk operations. This silently drops the option to prevent failing validation for any projects that may have set it.

Bug Fixes

[Testing] Await hook execution to prevent leaking side-effects
Hooks triggered by cancellation, exhaustion, and workflow operations were executed in fire-and-forget tasks. During testing this allowed hooks to finish after a test had already concluded, producing racy results when draining jobs.
When draining, the task is awaited so all side-effects complete before the drain returns. Outside of draining it remains fire-and-forget and isolated from the caller.
[Worker] Warn about risky unique states in Pro workers
Pro workers validated the shape of :unique options but skipped the advisory check that Oban.Worker performs, so state lists that can't detect duplicates or omit in-flight states compiled silently.
[Worker] Await async hook side-effects during testing
Bulk worker hooks (on_retried, on_cancelled, on_discarded) and other deferred side-effects ran in a detached task that borrowed the test's sandbox connection. When a test process exited while that task was still querying, the connection was reclaimed mid-query, leaking a Postgrex "owner exited" error even though no assertions failed.
Utils.async_maybe_await now awaits whenever testing is enabled, not only during draining, so these side-effects complete before a test concludes.
[Worker] Unify shared result type in the stage module
The c:after_process/3 callback stated it only returned :ok, which was incorrect. Now there's a properly defined t:result/0 that is shared between *process callbacks.
[Workflow] Return nil for missing recordings in workflows
External recorded output outlives its job and may expire or be reaped from the store while the job row remains. In that case all_recorded/3 and get_recorded/2 now return nil for the absent entry, matching their documented contract, rather than leaking an {:error, :missing} tuple. A genuinely unreachable store still returns {:error, reason} so an outage isn't silently mistaken for absent output.
Cascades continue to treat missing dependency output as an error and retry, since a just-completed step's output should still be present.

Next Page → Overview