Recording to Redis

Store recorded output in Redis. Redis is a good fit for transient results because it supports opaque binary values and expiration on every key.

This backend uses Redix. Add the dependency and start a named connection in your supervision tree:

def deps do
  [
    {:redix, "~> 1.0"}
  ]
end

{Redix, name: MyApp.Redix, host: System.get_env("REDIS_HOST", "localhost")}

The Module

The module accepts a supervised connection name and a TTL applied to every recorded payload. It fetches a batch with MGET and omits keys that are missing or expired.

defmodule MyApp.Redis do
  @behaviour Oban.Pro.Storage

  @impl Oban.Pro.Storage
  def init(opts) do
    conn = Keyword.get(opts, :conn, MyApp.Redix)
    ttl = Keyword.get(opts, :ttl, to_timeout(day: 1))

    %{conn: conn, ttl: ttl}
  end

  @impl Oban.Pro.Storage
  def put(key, payload, %{conn: conn, ttl: ttl}) do
    case Redix.command(conn, ["SET", key, payload, "PX", ttl]) do
      {:ok, "OK"} -> :ok
      {:ok, other} -> {:error, other}
      {:error, reason} -> {:error, reason}
    end
  end

  @impl Oban.Pro.Storage
  def fetch_all(keys, %{conn: conn}) do
    case Redix.command(conn, ["MGET" | keys]) do
      {:ok, values} ->
        payloads =
          for {key, value} <- Enum.zip(keys, values), not is_nil(value), into: %{} do
            {key, value}
          end

        {:ok, payloads}

      {:error, reason} ->
        {:error, reason}
    end
  end

  @impl Oban.Pro.Storage
  def delete_all(keys, %{conn: conn}) do
    case Redix.command(conn, ["DEL" | keys]) do
      {:ok, _count} -> :ok
      {:error, reason} -> {:error, reason}
    end
  end
end

Every write has an expiry, so Redis cleans up output even if its job isn't pruned. To retain output for the full TTL after a job is pruned, make delete_all/2 a no-op.

Using It

Configure the backend on a recorded worker, overriding the connection or TTL as needed:

use Oban.Pro.Worker, recorded: [storage: {MyApp.Redis, ttl: to_timeout(hour: 1)}]

To send every worker's output to Redis without annotating each one, set it as the compile-time default:

config :oban_pro, Oban.Pro.Worker, recorded: [storage: MyApp.Redis]