Scaling on Fly

Scale a Fly app's machine count with a small Oban.Pro.Cloud module built on Req and the Machines API. The app and token are resolved once in init/1, and scale/2 reconciles the running machine count against the desired count by cloning or destroying machines.

def deps do
  [
    {:req, "~> 0.5"}
  ]
end

The Module

The Oban.Pro.Cloud.init/1 callback pulls the app name and API token from the system environment that Fly injects into every machine. Because the Machines API works one machine at a time, Oban.Pro.Cloud.scale/2 lists the app's machines and then clones an existing one or destroys extras until the count matches, returning {:ok, conf} so the scaler can track the response.

defmodule MyApp.Fly do
  @behaviour Oban.Pro.Cloud

  @base "https://api.machines.dev/v1"

  @impl Oban.Pro.Cloud
  def init(opts) do
    opts
    |> Keyword.put_new(:app_id, System.fetch_env!("FLY_APP_NAME"))
    |> Keyword.put_new(:auth_token, System.fetch_env!("FLY_API_TOKEN"))
    |> Map.new()
  end

  @impl Oban.Pro.Cloud
  def scale(desired, conf) do
    req =
      Req.new(base_url: "#{@base}/apps/#{conf.app_id}/machines", auth: {:bearer, conf.auth_token})

    with {:ok, %{status: 200, body: machines}} <- Req.get(req) do
      reconcile(req, machines, desired, conf)
    else
      error -> normalize(error)
    end
  end

  defp reconcile(req, machines, desired, conf) do
    case desired - length(machines) do
      delta when delta > 0 ->
        source = %{region: hd(machines)["region"], config: hd(machines)["config"]}

        run(1..delta, conf, fn _ -> Req.post(req, json: source) end)

      delta when delta < 0 ->
        machines
        |> Enum.take(delta)
        |> run(conf, fn machine -> Req.delete(req, url: "/#{machine["id"]}", params: [force: true]) end)

      _zero ->
        {:ok, conf}
    end
  end

  defp run(enum, conf, fun) do
    Enum.reduce_while(enum, {:ok, conf}, fn item, _acc ->
      case fun.(item) do
        {:ok, %{status: status}} when status in 200..299 -> {:cont, {:ok, conf}}
        error -> {:halt, normalize(error)}
      end
    end)
  end

  defp normalize({:ok, response}), do: {:error, response}
  defp normalize({:error, reason}), do: {:error, reason}
end

New machines clone the region and config of one that's already running, so the image and resources match the rest of the pool. Scaling down passes force: true to stop and destroy machines in a single call. A deploy token from fly tokens create deploy is enough; set it as a secret so it lands in FLY_API_TOKEN alongside the FLY_APP_NAME that Fly already provides.

Using It

Point a scaler at the module — the app name and token come from the environment, so no options are required:

{DynamicScaler, scalers: [range: 1..5, cloud: MyApp.Fly]}