Store recorded job output outside the Oban jobs table.

By default, recorded output is stored with the job in Postgres. Use a Storage backend when output is too large for the database, belongs in an existing object store, or needs an independent retention policy. Backends may use services such as S3, GCS, or Redis, and recorded output is fetched transparently through the worker or workflow APIs.

Configuring Storage

Configure storage on a worker when only that worker's output belongs in the external store:

use Oban.Pro.Worker, recorded: [storage: {MyApp.S3, bucket: "some-output"}]

Configure a compile-time default to use the backend for all recorded workers:

config :oban_pro, Oban.Pro.Worker, recorded: [storage: MyApp.S3]

Or configure a workflow to use one backend for every recorded job in that workflow, including cascades. Workflow storage takes precedence over an individual worker's setting:

Oban.Pro.Workflow.new(storage: {MyApp.S3, bucket: "workflow-output"})

Disabling for Tests

Disable external storage in tests to keep recorded output in Postgres regardless of worker or workflow configuration:

# test.exs
config :oban_pro, Oban.Pro.Storage, disabled: true

This setting is independent of Oban's testing mode. For integration tests that exercise a backend, leave storage enabled and mock or provide the backend's client.

Storage Modules

Use one of these recipes as-is or adapt it for your application:

• S3 (and S3-compatible stores like Cloudflare R2 and Tigris)
• GCS (Google Cloud Storage, with service-account auth)
• Redis (fast, in-memory output with native key expiry)

Writing Storage Modules

A backend implements four callbacks:

• init/1 turns the persisted options into runtime configuration. Use it to resolve credentials, connection names, or clients.
• put/3 stores one payload.
• fetch_all/2 returns all available payloads for a list of keys.
• delete_all/2 removes payloads when their jobs are pruned.

Treat keys as generated identifiers and payloads as opaque binaries. fetch_all/2 may omit missing keys, while delete_all/2 must treat missing keys as successfully deleted.

This example stores output on the local filesystem:

defmodule MyApp.FileStorage do
  @behaviour Oban.Pro.Storage

  @impl Oban.Pro.Storage
  def init(opts), do: Keyword.put_new(opts, :dir, "/tmp/oban")

  @impl Oban.Pro.Storage
  def put(key, payload, conf), do: File.write(path(conf, key), payload)

  @impl Oban.Pro.Storage
  def fetch_all(keys, conf) do
    payloads =
      for key <- keys, {:ok, payload} <- [File.read(path(conf, key))], into: %{} do
        {key, payload}
      end

    {:ok, payloads}
  end

  @impl Oban.Pro.Storage
  def delete_all(keys, conf), do: Enum.each(keys, &File.rm(path(conf, &1)))

  defp path(conf, key), do: Path.join(conf[:dir], key)
end

Configure the module on a recorded worker:

use Oban.Pro.Worker, recorded: [storage: {MyApp.FileStorage, dir: "/tmp"}]

Pruning External Objects

Oban.Pro.Pruner calls delete_all/2 after pruning jobs with externally recorded output. Cleanup errors don't fail or delay pruning, so configure lifecycle rules or TTLs in the backend as a fallback.

To retain output after its job is pruned, make delete_all/2 a no-op and manage retention entirely through the store.