The DynamicPrioritizer plugin automatically adjusts job priorities to ensure all jobs are eventually processed.

Using mixed priorities in a queue causes certain jobs to execute before others. For example, a queue that processes jobs from various customers may prioritize customers that are in a higher tier or plan. All high priority (0) jobs are guaranteed to run before any with lower priority (1..9), which is wonderful for the higher tier customers but can lead to resource starvation. When there is a constant flow of high priority jobs the lower priority jobs will never get the chance to run.

The DynamicPrioritizer solves this by periodically boosting the priority of jobs that have been waiting longer than a configured threshold. Each cycle, qualifying jobs have their priority decremented by one (e.g. from 3 to 2), gradually converging toward the highest priority.

Using the Plugin

To use the DynamicPrioritizer plugin, add the module to your list of Oban plugins in config.exs:

config :my_app, Oban,
  plugins: [Oban.Pro.Plugins.DynamicPrioritizer]
  ...

Without any additional options the plugin will automatically increase the priority of any jobs that are available for 5 minutes or more, checking once per minute and reprioritizing up to 10,000 jobs per group each cycle. The after threshold, interval, and limit are all configurable.

By default jobs may climb all the way to the highest priority (0). Set max_priority to stop short. For example, max_priority: 2 keeps jobs from being boosted above priority 2, reserving 0 and 1 for genuinely high priority work.

Queue and Worker Overrides

The :after threshold applies globally to all queues and workers. Overrides let you fine tune reprioritization for specific queues or workers.

Configure the analysis queue to nudge jobs after only 1 minute:

plugins: [{
  Oban.Pro.Plugins.DynamicPrioritizer,
  queue_overrides: [analysis: {1, :minute}]
}]

Disable reprioritization globally while enabling it for a single queue:

plugins: [{
  Oban.Pro.Plugins.DynamicPrioritizer,
  after: :infinity,
  queue_overrides: [analysis: {1, :minute}]
}]

Override on a per-worker basis:

plugins: [{
  Oban.Pro.Plugins.DynamicPrioritizer,
  worker_overrides: [
    "MyApp.HighSLAWorker": {30, :seconds},
    "MyApp.LowSLAWorker": {10, :minutes}
  ]
}]

Queue and worker overrides can be combined. Note that overrides are applied independently—a job matching both a queue override and a worker override will be boosted by each:

plugins: [{
  Oban.Pro.Plugins.DynamicPrioritizer,
  interval: {2, :minutes},
  after: {5, :minutes},
  queue_overrides: [media: {10, :minutes}],
  worker_overrides: ["MyApp.HighSLAWorker": {30, :seconds}]
}]

Instrumenting with Telemetry

The DynamicPrioritizer plugin adds the following metadata to the [:oban, :plugin, :stop] event:

• :reprioritized_count — the number of jobs reprioritized