The Pruner extends the default Oban.Plugins.Pruner with fine-grained control over how jobs are retained. Where the standard pruner runs on a fixed schedule and treats every job the same, the Pruner lets you set a custom cron schedule and define rules that prune specific queues, workers, or states differently.

Using the Plugin

To use Pruner, configure it through the top-level :pruner option in config.exs:

config :my_app, Oban,
  pruner: Oban.Pro.Pruner
  ...

Without any additional configuration the pruner keeps a single implicit rule—the "default"—that retains a conservative 1,000 completed, cancelled, or discarded jobs. All other retention is expressed by adding rules.

Pruning Rules

Job pruning is controlled through database backed rules. A rule describes which jobs to prune and the retention strategy. Each rule has:

• A name which acts as a unique identifier
• A retention limit, either max_len: count or max_age: age
• An optional match on queue, worker, or state

Rules are provided as a list through the rules option:

pruner: {
  Pruner,
  rules: [
    [name: "events", queue: :events, max_age: {10, :minutes}],
    [name: "discarded", state: :discarded, max_age: {7, :days}]
  ]
}

Modes

A rule's retention limit determines how jobs are retained, set with either max_len (retain by length) or max_age (retain by age). Exactly one is required.

Setting max_len: count keeps the most recent count matching jobs. For example, to limit retention for the "events" queue to the most recent 10_000 jobs:

[name: "events", queue: :events, max_len: 10_000]

Setting max_age: age keeps matching jobs younger than age. Provide the age in seconds, or as a {value, unit} period where unit is :second, :minute, :hour, :day, :week, or :month. For example, retain the jobs in a less active "media" queue for two days:

[name: "media", queue: :media, max_age: {2, :days}]

With either limit, use :infinity to retain jobs indefinitely, rather than setting an arbitrarily high value like max_age: {999, :years}:

[name: "audit", worker: MyApp.AuditWorker, max_len: :infinity]

Legacy Overrides

Prior to rules, retention was configured with a global :mode and a set of override options. These options are still supported and automatically translated into rules.

The base :mode translates into the "default" rule, and each override becomes a rule matching the given queue, state, or worker:

pruner: {
  Pruner,
  mode: {:max_age, {7, :days}},
  queue_overrides: [events: {:max_age, {10, :minutes}}],
  state_overrides: [discarded: {:max_age, {2, :days}}],
  worker_overrides: ["MyApp.SecretWorker": {:max_age, {1, :second}}]
}

Overrides take precedence in the order: queue, then state, then worker, then the default :mode (the same first-match ordering described in Precedence).

Indexing Worker Matches

Rules that match on worker can't use Oban's standard indexes. If you prune a high volume of jobs with worker matches, add a compound index so pruning doesn't fall back to sequential scans:

create_if_not_exists index(:oban_jobs, [:worker, :state, :id], concurrently: true)

Managing Rules at Runtime

Rules are persisted in the database, so you can add, update, or remove them while the system is running rather than only via a config change.

The following functions are available to manage rules:

• all/1 — list every rule, ordered by position
• get/2 — fetch a single rule by name
• insert/2 — add a rule, or replace one with a matching name
• update/3 — change an existing rule's options
• delete/2 — remove a rule (the "default" can't be deleted, only updated)

For example, to start pruning a new dynamic queue without a deploy:

Pruner.insert(name: "media", queue: :media, max_age: {1, :week})

Changes take effect on the next pruning cycle.

Pausing

To stop a particular rule from pruning without deleting it, you can pause it. A paused rule is skipped entirely, it has no effect on later rules, so matched jobs fall through to the rules behind it:

Pruner.update("media", paused: true)

Sync Mode

On start, all configured rules are reconciled against the rules already in the database. The :sync_mode option controls what happens to persisted rules that aren't in the configuration.

There are two possible modes:

• :manual (the default) — configured rules are inserted or updated, and any other persisted rules are left in place. This preserves rules added at runtime, e.g. from a dashboard.

• :automatic — persisted rules absent from the configuration are deleted, making the configuration the source of truth. The "default" rule is always preserved.

Set the mode with :sync_mode:

pruner: {Pruner, sync_mode: :automatic, rules: [...]}

What Configuration Overwrites

Reconciling doesn't overwrite everything. Options declared in config - e.g. :archive, :limit, :timeout, match, and retention - are reapplied in either mode. Runtime changes to those options are replaced by the configured values.

Pausing is never overwritten, regardless of mode. A rule paused with update/3 stays paused across restarts even when it's also listed in config, because pausing is an operational decision. Resume it the same way you paused it:

Pruner.update("media", paused: false)

Ordering depends on the mode:

• :manual — a rule's :position applies only when the rule is first inserted. After that, ordering belongs to whatever changed it last, so rules rearranged at runtime keep their order across restarts. A newly configured rule is appended after the rules already persisted, rather than claiming the position implied by its place in the list.

• :automatic — positions are rewritten from the configuration on every start, matching the order rules are listed in. Any runtime reordering is lost on the next restart.

In both modes the "default" rule is moved to the end, and it can't be repositioned.

Keeping Up With Inserts

By default each rule deletes at most 10,000 jobs per pruning pass, within a 60 second query timeout. The limit exists to prevent connection timeouts and excessive table locks. A busy system can easily insert more than 10,000 jobs per minute during standard operation. If you find that jobs are accumulating despite active pruning you can raise the limit.

Set at the plugin level, :limit and :timeout act as defaults for any rule that doesn't set its own. Here we raise the delete limit to 25,000 and allow 90 seconds per query:

pruner: {
  Pruner,
  limit: 25_000,
  timeout: to_timeout(second: 90)
}

Deleting is typically very fast, and the 10k default is rather conservative. Feel free to increase the limit to a number that your system can handle.

Setting a Schedule

By default, pruning happens at the top of every minute based on the CRON schedule * * * * *. You're free to set any CRON schedule you prefer for greater control over when to prune. For example, to prune once an hour instead:

pruner: {Pruner, schedule: "0 * * * *"}

Schedules are evaluated in :timezone, which defaults to "Etc/UTC". To prune once a day at midnight in your local timezone:

pruner: {
  Pruner,
  limit: 100_000,
  schedule: "0 0 * * *",
  timezone: "America/Chicago"
}

Pruning less frequently can reduce load on your system, particularly if you're using multiple rules. However, be sure to set a higher limit to compensate for more accumulated jobs.

Archiving Jobs

Some jobs are a historic record worth keeping past the point they'd normally be pruned. Rather than deleting matching jobs, a rule can archive them instead. Archived jobs are copied to a dedicated table (oban_jobs_archive) before they're removed from oban_jobs, keeping the job's full record (args, meta, errors, timestamps, etc.) for long-term storage and later inspection.

Set archive: true on any rule whose jobs you'd like to retain:

pruner: {
  Pruner,
  rules: [
    [name: "audit", worker: AuditWorker, max_age: {1, :week}, archive: true]
  ]
}

After one week all finished AuditWorker jobs are moved to the archive table rather than simply being deleted.

Archiving runs in the same transaction as deleting, so a job is never lost between the two tables. Rows are copied whole with a single set-based insert, without round-tripping through your application.

The archive is append-only and managing retention is entirely up to you, e.g. with table partitioning or a scheduled cleanup job. Read archived jobs back through Oban.Pro.Archive, which loads them as ordinary Oban.Job structs:

Repo.all(Oban.Pro.Archive)

Executing a Callback Before Delete

When archiving isn't enough (e.g. streaming to an external warehouse, writing to object storage, or applying a custom transformation), you can provide a before_delete callback.

To accomplish this, specify a callback to execute before proceeding with the deletion:

defmodule DeleteHandler do
  def call(job_ids) do
    # Use the ids at this point, from within a transaction
  end
end

pruner: {
  Pruner,
  before_delete: {DeleteHandler, :call, []},
  ...

The callback receives a list of the ids for the jobs that are about to be deleted. The callback runs within the same transaction that's used for deletion, and you should keep it quick or move heavy processing to an async process. Note that because it runs in the same transaction as deletion, the jobs won't be available after the callback exits.

To pass in extra arguments as "configuration" you can provide args to the callback MFA:

defmodule DeleteHandler do
  import Ecto.Query

  def call(job_ids, storage_name) do
    jobs = MyApp.Repo.all(where(Oban.Job, [j], j.id in ^job_ids))

    Storage.call(storage_name, jobs)
  end
end

before_delete: {DeleteHandler, :call, [ColdStorage]}

Reaping External Storage

When pruning jobs with externally recorded output, the Pruner asks each Oban.Pro.Storage backend to delete the corresponding payloads. Cleanup happens after pruning and errors don't fail the prune, so use lifecycle rules or TTLs as a fallback for removing orphaned output.

Workflow Preservation

By default, jobs that are part of an active workflow are retained to prevent partial deletion of workflow data while the workflow is still running. A workflow is considered active if it contains any jobs in an incomplete state.

This check includes linked sub-workflows and parent workflows, ensuring that all related jobs across workflow hierarchies are preserved until the entire workflow completes.

Workflows with compensations are also preserved while compensation is pending or running, and after a failed compensation so the original jobs remain available for retries. The jobs are released once compensation completes, or when a failed compensation workflow is pruned under the normal retention rules—after which the compensation can no longer be retried.

To disable workflow preservation and prune all eligible jobs regardless of workflow status:

pruner: {
  Pruner,
  preserve_workflows: false,
  ...

Disabling workflow preservation can improve pruning performance for systems with many workflows, but may result in incomplete workflow job histories if workflows are pruned while still active. Note that preserve_workflows: false is incompatible with compensations, as original jobs may be pruned before their compensation steps execute.

Implementation Notes

Some additional notes about pruning in general and nuances of the Pruner plugin:

• Pruning is best-effort and performed out-of-band. This means that all limits are soft, so jobs beyond a specified age may not be pruned immediately after jobs complete.

• Pruning is only applied to jobs that are completed, cancelled or discarded (has reached the maximum number of retries or has been manually killed). It'll never delete a new, scheduled, or retryable job.

• Jobs that are part of an active workflow are retained regardless of their state. A workflow is considered active if it contains any jobs in suspended, scheduled, retryable, available, or executing states. This behavior is controlled by the preserve_workflows option, which defaults to true.

• Only a single node will prune at any given time, which prevents potential deadlocks between transactions.

Instrumenting with Telemetry

The Pruner plugin adds the following metadata to the [:oban, :plugin, :stop] event:

• :archived_count - the number of deleted jobs that were copied to the archive
• :pruned_count - the total number of jobs that were deleted