Autoscaler

Scale your woodpecker agents automatically to the moon and back based on the current load.

Usage

If you are using docker-compose you can add the following to your docker-compose.yml file:

# docker-compose.yml
version: '3'

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:next
    [...]

  woodpecker-autoscaler:
    image: woodpeckerci/autoscaler:next
    restart: always
    depends_on:
      - woodpecker-server
    environment:
      - WOODPECKER_SERVER=https://your-woodpecker-server.tld # the url of your woodpecker server / could also be a public url
      - WOODPECKER_TOKEN=${WOODPECKER_TOKEN} # the Personal Access Token you can get from the UI https://your-woodpecker-server.tld/user/cli-and-api
      - WOODPECKER_MIN_AGENTS=0
      - WOODPECKER_MAX_AGENTS=3
      - WOODPECKER_WORKFLOWS_PER_AGENT=2 # the number of workflows each agent can run at the same time
      - WOODPECKER_GRPC_ADDR=https://grpc.your-woodpecker-server.tld # the grpc address of your woodpecker server, publicly accessible from the agents
      - WOODPECKER_GRPC_SECURE=true
      - WOODPECKER_AGENT_ENV= # optional environment variables to pass to the agents
      - WOODPECKER_PROVIDER=hetznercloud # set the provider, you can find all the available ones down below
      - WOODPECKER_HETZNERCLOUD_API_TOKEN=${WOODPECKER_HETZNERCLOUD_API_TOKEN} # your api token for the Hetzner cloud

The agents will use WOODPECKER_GRPC_ADDR and an agent token automatically created on the server by the autoscaler to connect to the server. Therefore the WOODPECKER_GRPC_ADDR has to be publicly accessible from the newly created agents. Check for example how you could use caddy to expose the grpc connection.

DigitalOcean

Set WOODPECKER_PROVIDER=digitalocean and configure at least:

• WOODPECKER_DIGITALOCEAN_API_TOKEN (or WOODPECKER_DIGITALOCEAN_API_TOKEN_FILE)

DigitalOcean support is currently experimental: it has not been tested by the project maintainers, as none of them have real provider access.

Useful optional settings:

• WOODPECKER_DIGITALOCEAN_REGION (default: nyc1)
• WOODPECKER_DIGITALOCEAN_SIZE (default: s-1vcpu-1gb)
• WOODPECKER_DIGITALOCEAN_IMAGE (default: ubuntu-24-04-x64, slug or name)
• WOODPECKER_DIGITALOCEAN_SSH_KEYS (names or fingerprints; if unset, a key named random-autoscaler-key is created and reused, its private key is discarded)
• WOODPECKER_DIGITALOCEAN_TAGS
• WOODPECKER_DIGITALOCEAN_PUBLIC_IPV4_ENABLE (default: true; set to false to create private droplets without a public network interface, which requires WOODPECKER_DIGITALOCEAN_NAT_GATEWAY)
• WOODPECKER_DIGITALOCEAN_NAT_GATEWAY (name or ID of an existing VPC NAT gateway; the agents are placed in the VPC it serves as default gateway so they can reach the server and pull images)
• WOODPECKER_DIGITALOCEAN_PUBLIC_IPV6_ENABLE (default: true, requires public IPv4)

OpenStack

Set WOODPECKER_PROVIDER=openstack. The prefix for all the following environment variables is WOODPECKER_OPENSTACK_.

You have to supply the AUTH_URL pointing to your Keystone. If necessary, you can also specifiy the DOMAIN_NAME, REGION and PROJECT_NAME.

Both USERNAME/PASSWORD authentication and application credentials via APPLICATION_CREDENTIAL_ID and APPLICATION_CREDENTIAL_SECRET are supported. Credentials can also be read from files, to do so append _FILE to the appropriate variable name and set it to the file path.

You can select the flavor and image for the agent instances via FLAVOR/IMAGE_NAME or UUID reference (FLAVOR/IMAGE_REF). The agent platform is taken from the image's hw_architecture (or architecture) property. Without it the platform is unknown, and workflows that set a platform label do not trigger scaling. If you set VOLUME_SIZE, block storage volumes are used.

You can add your OpenStack SSH keypair via KEYPAIR.

Server type fallbacks

Hetzner Cloud (WOODPECKER_HETZNERCLOUD_SERVER_TYPE), AWS (WOODPECKER_AWS_INSTANCE_TYPE), Scaleway (WOODPECKER_SCALEWAY_SERVER_TYPES) and Oracle Cloud (WOODPECKER_ORACLE_SHAPES) take an ordered list of server types. If a type is out of capacity, the next type of the same architecture is tried; any other error fails the deploy.

Limits:

• There is no fallback across architectures: x86 and arm types are separate pools, each with its own fallback order.
• Workflows without a platform label are deployed on the architecture of the first listed type.
• Workflows that ask for a platform none of the types provides do not trigger scaling.

Oracle Cloud

Set WOODPECKER_PROVIDER=oracle and configure at least:

• WOODPECKER_ORACLE_COMPARTMENT_OCID
• WOODPECKER_ORACLE_SUBNET_OCID
• credentials, either an API signing key (WOODPECKER_ORACLE_TENANCY_OCID, WOODPECKER_ORACLE_USER_OCID, WOODPECKER_ORACLE_REGION, WOODPECKER_ORACLE_FINGERPRINT, WOODPECKER_ORACLE_PRIVATE_KEY) or WOODPECKER_ORACLE_USE_INSTANCE_PRINCIPAL=true when the autoscaler itself runs on an Oracle Cloud instance

Oracle Cloud support is currently experimental: it has not been tested by the project maintainers, as none of them have real provider access.

Useful optional settings:

• WOODPECKER_ORACLE_AVAILABILITY_DOMAINS (default: every availability domain of the compartment; the agents are launched in the first one that has capacity, in the given order)
• WOODPECKER_ORACLE_SHAPES (default: VM.Standard.E4.Flex; ordered list, shapes with an Ampere or NVIDIA Grace processor such as VM.Standard.A1.Flex are arm64 and every other shape is amd64; each shape is looked up at startup, so a name that does not exist in the compartment fails immediately)
• WOODPECKER_ORACLE_OCPUS (default: 1, only sent for flexible shapes)
• WOODPECKER_ORACLE_MEMORY_IN_GBS (default: 8, only sent for flexible shapes)
• WOODPECKER_ORACLE_IMAGE_OCID (default: the most recent image matching the operating system settings below, looked up per shape; an explicit image must boot on every configured shape)
• WOODPECKER_ORACLE_IMAGE_OPERATING_SYSTEM (default: Canonical Ubuntu)
• WOODPECKER_ORACLE_IMAGE_OPERATING_SYSTEM_VERSION (default: 24.04)
• WOODPECKER_ORACLE_BOOT_VOLUME_SIZE (default: 50, Oracle Cloud rejects smaller boot volumes)
• WOODPECKER_ORACLE_ASSIGN_PUBLIC_IP (default: true; set to false for a private subnet that reaches the server through a NAT gateway)
• WOODPECKER_ORACLE_SSH_KEY
• WOODPECKER_ORACLE_TAGS (comma separated key=value freeform tags; Oracle Cloud allows 10 per instance and the autoscaler already uses one of them to mark its pool)

Capacity is the common failure mode on Oracle Cloud, in particular for the free VM.Standard.A1.Flex shape. If a launch fails with an out-of-capacity error the next availability domain is tried, then the next shape of the same architecture, so listing more than one is worthwhile.

The subnet is checked at startup: a subnet bound to one availability domain limits the agents to that domain, and a private subnet requires WOODPECKER_ORACLE_ASSIGN_PUBLIC_IP=false.

The agents block the instance metadata service (169.254.169.254, and fd00:c1::a9fe:a9fe over IPv6) before the workflow steps start, so a step cannot read the agent token back out of user_data. Oracle Cloud also serves DNS (port 53) and NTP (UDP port 123) on that address, so instead of a blackhole route DNS and NTP are let through and every other packet to that address is dropped, which blocks the metadata API on port 80.

Only virtual machine shapes are billed per second with a one-minute minimum; bare metal shapes carry a one-hour minimum, which the per-second teardown policy below does not account for.

Teardown policy

How idle agents are torn down depends on how the selected provider bills:

• Per-second billing (e.g. AWS, Scaleway): an idle agent is drained and removed once it has been idle for WOODPECKER_AGENT_IDLE_TIMEOUT. Holding an idle agent open buys nothing.

• Hourly-rounded-up billing (e.g. Linode, Hetzner Cloud, Vultr): a partial hour costs the same as a full one, so an idle agent is kept schedulable for the rest of the hour that has already been paid for and is only torn down just before its next hour boundary (anchored at its creation time). A busy agent simply rolls into the next paid hour; you never pay for an idle hour.

  The teardown window is WOODPECKER_AGENT_BILLING_TEARDOWN_MARGIN (default 2m) plus WOODPECKER_RECONCILIATION_INTERVAL, so a reconciliation can never tick straight past the boundary. With the defaults (2m margin, 1m interval) an idle agent becomes eligible for teardown in the last 3 minutes of each paid hour.

The billing model is selected automatically by the provider, so no extra configuration is required to benefit from this.

Roadmap

• Add support for multiple providers
  • Hetzner Cloud
  • Amazon AWS
  • Google Cloud
  • Azure
  • Digital Ocean [experimental] (untested by the maintainers against real provider access, see above)
  • Linode
  • OpenStack [experimental]
  • Oracle Cloud [experimental] (untested by the maintainers against real provider access, see above)
  • Vultr
  • Scaleway
• Cleanup agents
  • Remove agents which exist on the provider but are not in the server list (they wont be able to connect to the server anyway as their is no agent token for them)
  • Remove agents from server list which do not exist on the provider
  • Remove agents which have not connected for a long time
• Release as container image
• Add docs
• Support agent deployment with specific attributes (e.g. platforms, architectures, etc.)