Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Keeps pinned dependencies current. Takes effect only on the default branch.
# https://docs.github.com/code-security/dependabot/dependabot-version-updates
version: 2
updates:
# GitHub Actions: bumps the SHA pins and their `# vX.Y.Z` comments in workflows.
# Grouped into one PR to avoid five separate bumps.
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
groups:
actions:
patterns: ["*"]

# Python runtime/dev deps in python/pyproject.toml.
- package-ecosystem: pip
directory: /python
schedule:
interval: weekly
groups:
python:
patterns: ["*"]

# Pinned dev/build/publish tooling in python/requirements/dev.txt (ruff, pytest,
# build, twine). Dependabot discovers the *.txt files in this directory.
- package-ecosystem: pip
directory: /python/requirements
schedule:
interval: weekly
groups:
tooling:
patterns: ["*"]
131 changes: 131 additions & 0 deletions .github/workflows/python-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
name: Publish Python package to PyPI

# Trusted publisher on PyPI must reference this exact filename and the `pypi`
# environment, or publishing is rejected.

on:
release:
types: [published]

permissions:
contents: read

# cancel-in-progress: false — cancelling mid-publish can corrupt a release.
concurrency:
group: python-release-${{ github.ref }}
cancel-in-progress: false

jobs:
# Source of truth for the test matrix: generated from [tool.python-support].
# python-v* tags only — test/build/publish inherit this via `needs`.
versions:
name: python versions
if: startsWith(github.ref_name, 'python-v')

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a plan for generating these tags? Would this be done from a command in python_dev.py file?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now the python-v* tag is created manually via the GitHub Release UI. A python_dev.py release/tag command that bumps the pyproject version and cuts the tag would be a good follow-up addition

runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
data: ${{ steps.set.outputs.data }}
steps:
- name: Check out the repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"

- name: Generate supported versions from pyproject.toml
id: set
run: echo "data=$(python scripts/python_dev.py list-supported-versions)" >> "$GITHUB_OUTPUT"

# Full support matrix gates the release; build/publish depend on it. Sanity check with
# Ubuntu-only tests since package is pure-Python.
test:
name: test (py${{ matrix.python-version }})
needs: versions
runs-on: ubuntu-latest

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The release test gate runs only on ubuntu-latest, whereas regular CI (python.yml) additionally tests Windows and macOS at the ends of the range — precisely where interpreter/platform differences surface. As written, a Windows- or macOS-only regression at the range boundary passes the release gate and ships. If that's a deliberate cost/time tradeoff, a one-line comment saying so would help; otherwise mirror CI's include matrix (min/max on windows-latest + macos-14) here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, added a comment in next commit.

timeout-minutes: 15
Comment thread
mwaisaida-13 marked this conversation as resolved.
strategy:
fail-fast: false # report every version, not just the first failure
matrix:
python-version: ${{ fromJson(needs.versions.outputs.data).all }}
steps:
- name: Check out the repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Comment thread
mwaisaida-13 marked this conversation as resolved.

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}

- name: Install and run the test suite
timeout-minutes: 10
run: python scripts/python_dev.py test

build:
name: build and verify
needs: test
if: startsWith(github.ref_name, 'python-v')
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out the repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"

# Tag must match pyproject.toml version, and the publish guard `Private :: Do Not Upload`
# must be gone.
- name: Verify the tag matches the packaged version and the package is publishable
run: python scripts/python_dev.py verify "$GITHUB_REF_NAME"

- name: Build wheel and sdist, check metadata
timeout-minutes: 5
run: python scripts/python_dev.py package

- name: Save built distributions for the publish job
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: python-dist
path: python/dist/
if-no-files-found: error # an empty artifact silently breaks the publish job

publish:
name: publish
needs: build
if: startsWith(github.ref_name, 'python-v')
runs-on: ubuntu-latest
timeout-minutes: 10
# Must match the environment name on the trusted publisher; carries the
# required-reviewer gate so a run alone cannot publish.
environment: pypi
permissions:
# Job permissions REPLACE the top-level block, so restate contents: read.
contents: read
id-token: write # OIDC token for PyPI auth + PEP 740 attestation signing
steps:
# Checkout so the smoke-test script is on the runner.
- name: Check out the repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"

- name: Retrieve built distributions from the build job
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: python-dist
path: dist/

- name: Publish to PyPI with attestations
uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
with:
attestations: true

- name: Smoke-test the published package
run: python scripts/python_dev.py smoke "${GITHUB_REF_NAME#python-v}"
76 changes: 44 additions & 32 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,67 +7,79 @@ on:
branches: [main]
paths:
- "python/**"
- "scripts/**"
- ".github/workflows/python.yml"
pull_request:
paths:
- "python/**"
- "scripts/**"
- ".github/workflows/python.yml"

defaults:
run:
working-directory: python
Comment on lines -16 to -18

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you verify that this was intentionally removed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it's intentional. The runner now lives at scripts/python_dev.py and is invoked from the repo root, so a working-directory: python default would break the path to the script.


jobs:
lint:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Check out the repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: actions/setup-python@v5
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"

- name: Install package with dev dependencies
run: python -m pip install -e ".[dev]"
- name: Lint and format check
run: python scripts/python_dev.py lint

# Lint and format are interpreter-independent, so they run once rather
# than across the whole test matrix.
- name: Ruff lint
run: python -m ruff check .
# Load Python versions generated from [tool.python-support].
versions:
name: python versions
runs-on: ubuntu-latest
outputs:
data: ${{ steps.set.outputs.data }}
steps:
- name: Check out the repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Ruff format
run: python -m ruff format --check .
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"

- name: Generate supported versions from pyproject.toml
id: set
run: echo "data=$(python scripts/python_dev.py list-supported-versions)" >> "$GITHUB_OUTPUT"

test:
name: py${{ matrix.python-version }} on ${{ matrix.os }}
needs: versions
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# The floor and the ceiling on each OS, plus Apple Silicon. Every version
# in requires-python is covered on Linux; the other platforms test the
# ends of the range, which is where interpreter differences show up.
os: [ubuntu-latest, windows-latest, macos-14]
python-version: ["3.10", "3.14"]
# Every supported version runs on Linux;
# the other platforms test the ends of the range, which is where
# interpreter differences show up.
os: [ubuntu-latest]
python-version: ${{ fromJson(needs.versions.outputs.data).all }}
include:
- os: ubuntu-latest
python-version: "3.11"
- os: ubuntu-latest
python-version: "3.12"
- os: ubuntu-latest
python-version: "3.13"
- os: windows-latest
python-version: "${{ fromJson(needs.versions.outputs.data).min }}"
- os: windows-latest
python-version: "${{ fromJson(needs.versions.outputs.data).max }}"
- os: macos-14
python-version: "${{ fromJson(needs.versions.outputs.data).min }}"
- os: macos-14
python-version: "${{ fromJson(needs.versions.outputs.data).max }}"

steps:
- uses: actions/checkout@v4
- name: Check out the repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}

- name: Install package with dev dependencies
run: python -m pip install -e ".[dev]"

- name: Run tests
run: python -m pytest -q
- name: Install and run tests
run: python scripts/python_dev.py test
9 changes: 4 additions & 5 deletions python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,10 @@ classifiers = [
"Programming Language :: Python :: 3.14",
]

[project.optional-dependencies]
dev = [
"pytest>=7.0",
"ruff>=0.6",
]
# Supported Python range — the source for the CI/release test matrices.
[tool.python-support]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[tool.python-support] is now the source of truth for the test/release matrix, but the supported range is still declared in two other hand-maintained places: requires-python = ">=3.10" (line 17) and the Programming Language :: Python :: 3.10…3.14 trove classifiers (lines 29-33). Bumping max here to 3.15 won't add the classifier or move the floor, so the matrix and the package metadata will silently disagree. Since the whole point of this table is one source of truth, consider deriving the classifiers (and validating the requires-python floor against min) from [tool.python-support] — e.g. generate/verify them in dev.py — or at least add a check that the three stay in sync.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, will add a check to verify there's no drift

min = "3.10"
max = "3.14"

[tool.ruff]
# The floor in requires-python. Keeps ruff from suggesting syntax that would not
Expand Down
17 changes: 17 additions & 0 deletions python/requirements/dev.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Dev/CI tooling, pinned so CI is reproducible and a new release of any of these
# can't silently break an unrelated PR. Dependabot bumps them (pip entry in

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a mechanism to install the required dev tooling:

  • Python 3.12 + PIP
  • Dev tooling pip dependencies

Then we need dependencies for each of the supported languages (that should work for each of the supported versions for each language):

  • Build -> dependencies required to build, test and package the lib
  • Runtime -> runtime dependencies of the lib

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would also be a follow-up for multiple language support. Currently the different scripts runs run(py, "-m", "pip", "install", "-q", "-e", ".", "-r", "requirements/dev.txt") to install the dependencies. will add an additional setup script for local development

# dependabot.yml). Installed by scripts/python_dev.py setup.
# Requires Python 3.11+

# install
pip==26.2.1

# build
build==1.5.0

# dev
ruff==0.16.3
pytest==9.1.1

# publish
twine==7.0.0
Loading