Skip to content

[improve][ci] Reduce disk usage and extend Broker Group 1 timeout in Pulsar CI - #27196

Merged
lhotari merged 1 commit into
apache:masterfrom
lhotari:lh-improve-ci-disk-space
Oct 29, 2026
Merged

lhotari merged 1 commit into
apache:masterfrom
lhotari:lh-improve-ci-disk-space

Conversation

@lhotari

@lhotari lhotari commented Oct 29, 2026

Copy link
Copy Markdown
Member

Motivation

Some GitHub runners have less disk space and fewer CPUs than others. As with #26558, one goal is to make Pulsar CI work on smaller runners.

  • On runners with fewer CPUs, the unit test group BROKER_GROUP_1 can go over the default 60-minute job timeout.
  • The BACKWARDS_COMPAT integration test group pulls several older apachepulsar/pulsar images in addition to the java-test-image, so it can run out of disk space on runners with less free disk.
  • Each job keeps the downloaded gradle-build-outputs.tar and Docker image archives on disk after extracting or loading them. This wastes disk space for the rest of the job.

Modifications

  • Set the BROKER_GROUP_1 unit test timeout to 70 minutes, 10 minutes more than the default of 60.
  • Run the clean-disk action for the BACKWARDS_COMPAT integration test group. It uses the default mode with clean-threshold-gb: '30', not full mode. The action does nothing if 30 GB or more is free. Otherwise it removes /usr/local/lib/android and then /opt/ghc until the threshold is met, and falls back to cleaning apt state.
  • Remove gradle-build-outputs.tar in the same step that extracts it, in every job. Remove the downloaded Docker image archive in the same step that loads it, in the integration test and system test jobs. Each step stops at the first failing command, so the archive is only removed after a successful extract or load.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

…Pulsar CI

Some GitHub runners have less disk space and fewer CPUs.

- Increase the BROKER_GROUP_1 unit test timeout from the default 60 to 70 minutes
- Clean disk for the BACKWARDS_COMPAT integration test group, using the default
  mode with a 30GB threshold
- Delete the build outputs tar and the downloaded Docker image archives in the
  same step where they are extracted or loaded

Assisted-by: Claude Code (claude-opus-5-5)
@lhotari
lhotari merged commit 8b827b7 into apache:master Oct 29, 2026
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants