Oban.Pro.Plugins.DynamicPrioritizer (Oban Pro v0.13.1)
The DynamicPrioritizer plugin automatically adjusts job's priorities to ensure all jobs are eventually processed.
Using mixed priorities in a queue causes certain jobs to execute before others. For example, a
queue that processes jobs from various customers may prioritize customers that are in a higher
tier or plan. All high priority (0
) jobs are guaranteed to run before any with lower priority
(1..3
), which is wonderful for the higher tier customers but can lead to resource starvation.
When there is a constant flow of high priority jobs the lower priority jobs will never get the
chance to run.
using-the-plugin
Using the Plugin
To use the DynamicPrioritizer
plugin add the module to your list of Oban plugins in
config.exs
:
config :my_app, Oban,
plugins: [Oban.Pro.Plugins.DynamicPrioritizer]
...
Without any additional options the plugin will automatically increase the priority of any jobs
that are available
for 5 minutes or more. To reprioritize after less time waiting you can
configure the :after
time:
plugins: [{Oban.Pro.Plugins.DynamicPrioritizer, after: :timer.minutes(2)}]
Now lower job priorities are bumped after 2 minutes of waiting, and every minute after that. To
lower the reprioritization frequency you can change the :interval
along with the :after
time:
plugins: [{
Oban.Pro.Plugins.DynamicPrioritizer,
after: :timer.minutes(10),
interval: :timer.minutes(5)
}]
Here we've specified that job priority will increase every 5 minutes after the first 10 minutes of waiting.
providing-overrides
Providing Overrides
The after
option applies to jobs for any workers across all queues. The DynamicPrioritizer
plugin allows you to specify per-queue and per-worker overrides that fine tune reprioritization.
Let's configure reprioritization for the analysis
queue so that it nudges jobs after only 1
minute:
plugins: [{
Oban.Pro.Plugins.DynamicPrioritizer,
queue_overrides: [analysis: :timer.minutes(1)]
}]
We can also effectively disable reprioritization for all other queues by setting the period to
:infinity
:
plugins: [{
Oban.Pro.Plugins.DynamicPrioritizer,
after: :infinity,
queue_overrides: [analysis: :timer.minutes(1)]
}]
If per-queue overrides aren't enough we can override on a per-worker basis instead:
plugins: [{
Oban.Pro.Plugins.DynamicPrioritizer,
interval: :timer.seconds(15),
worker_overrides: [
"MyApp.HighSLAWorker": :timer.seconds(30),
"MyApp.LowSLAWorker": :timer.minutes(10)
]
}]
Naturally you can mix and match overrides to finely control reprioritization:
plugins: [{
Oban.Pro.Plugins.DynamicPrioritizer,
interval: :timer.minutes(2),
after: :timer.minutes(5),
queue_overrides: [media: :timer.minutes(10)],
worker_overrides: ["MyApp.HighSLAWorker": :timer.seconds(30)]
}]
instrumenting-with-telemetry
Instrumenting with Telemetry
The DynamicPrioritizer
plugin adds the following metadata to the [:oban, :plugin, :stop]
event:
:reprioritized_count
— the number of jobs reprioritized
Link to this section Summary
Functions
Returns a specification to start this module under a supervisor.
Link to this section Functions
child_spec(init_arg)
Returns a specification to start this module under a supervisor.
See Supervisor
.