Skip to content

chore(deps): migrate dependency management to pyproject.toml + uv (v2) - #247

Merged
matrixise merged 8 commits into
masterfrom
feature/migrate-to-uv-171-v2
Sep 25, 2026
Merged

matrixise merged 8 commits into
masterfrom
feature/migrate-to-uv-171-v2

Conversation

@matrixise

@matrixise matrixise commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Reworked take on #172, from scratch, rebased on current master (that PR is now CONFLICTING/stale and had a real CI regression, see below). Replaces the requirements/*.in + pip-compile-generated requirements/*.txt workflow with a single pyproject.toml + committed uv.lock, matching the project's move to a uv-only workflow.

  • pyproject.toml: [project.dependencies] (main) + a dev dependency-group, tool.uv.package = false (this is a deployable Django app, not a distributed library). Drops the now-unused pythonie/setup.py
  • uv.lock: committed, resolved lock file (115 packages)
  • All current dependency versions carried over unchanged, including yesterday's security patches (django, wagtail, pillow, sqlparse, soupsieve, djangorestframework, msgpack, pip). django capped at <6.1 and ruff pinned to 0.15.16 to avoid incidental version bumps changing behavior as a side effect of this migration
  • No more root requirements.txt: Heroku's Python buildpack now installs from uv.lock natively (see "Heroku deployment" below)
  • Dockerfile: uv sync --frozen --all-groups, .venv on PATH so existing docker compose run ... python ... invocations are unchanged
  • Taskfile.yaml / toast.yml: dependencies:* tasks now call uv lock / uv lock --upgrade[-package] / uv tree / uv run pip-audit instead of uv pip compile
  • .github/workflows/test.yml: astral-sh/setup-uv + uv sync --all-groups, every step prefixed with uv run. Keeps every existing quality gate (ruff format, ruff check, pip-audit, Django tests). This is the part Migrate dependency management to pyproject.toml with uv #172 regressed, silently dropping format/lint/security checks and downgrading actions/checkout/setup-python to @v2
  • .tool-versions (asdf) → mise.toml (native format for the team's tool version manager going forward)
  • Docs (README.md, CONTRIBUTING.md, DEVELOPMENT.md, CLAUDE.md) updated to the uv sync / uv run workflow

Heroku deployment

The heroku/python buildpack picks its package manager from the files at the repo root, and aborts the build (Error: Multiple Python package manager files were found) if it finds more than one of requirements.txt, poetry.lock, uv.lock. An earlier revision of this PR kept a uv export-generated requirements.txt next to uv.lock, which would have broken the deploy. It has been removed, along with the dependencies:export task.

With only uv.lock present, the buildpack (uv support since v286, 2025-05-13):

  • reads the Python version from .python-version (3.13, already on master; runtime.txt is deprecated and not needed)
  • runs uv sync --locked --no-default-groups into the slug's system Python, so the dev group is not installed and uv.lock must be in sync with pyproject.toml
  • runs collectstatic as before

At runtime python and gunicorn live in /app/.heroku/python/bin and uv is not needed, so the Procfile (release: python pythonie/manage.py migrate ..., web: gunicorn ...) is unchanged.

The pythonie app uses heroku/python from the buildpack registry (not pinned to a tag), so it gets the current buildpack.

Why not just update #172?

#172 is 9 months stale, conflicts with master (mostly from yesterday's dependency-security PRs touching requirements/*.txt), pins Django 5.2.x/Wagtail 7.2.x (both outdated now), and its CI workflow drops ruff format --check, ruff check and pip_audit entirely while downgrading actions/checkout/setup-python to @v2. Rather than untangling that history, this PR rebuilds the same end state cleanly on top of current master. Happy to close #172 once this is reviewed, or keep both open for comparison, your call.

Test plan

  • uv sync --all-groups installs cleanly (115 packages resolved)
  • uv lock --check: lock file in sync with pyproject.toml
  • uv run python pythonie/manage.py test pythonie --settings=pythonie.settings.tests (SQLite, 7/7 passed)
  • uv run ruff format --check pythonie / uv run ruff check pythonie: both clean, matching master
  • uv run pip-audit: no known vulnerabilities
  • docker build . succeeds; python, ruff, django/wagtail imports resolve via the .venv on PATH inside the image
  • Real heroku-buildpack-python (v352) bin/compile run against this branch in the heroku/heroku:24-build image, with production config vars: Python 3.13.15 from .python-version, uv sync --locked --no-default-groups (68 packages), collectstatic OK (257 files)
  • Runtime simulation with the slug at /app and a clean env: manage.py check --settings=pythonie.settings.production reports no issues, the migrate command loads, gunicorn pythonie.wsgi boots and serves requests (HTTP 301 SSL redirect, as expected)
  • heroku buildpacks -a pythonie: heroku/python from the registry, not pinned
  • First real deploy: watch the build log and heroku releases, keep task heroku:rollback at hand (the first uv build is a full rebuild since the pip cache is discarded)

Replaces requirements/main.in, dev.in, production.in and their
pip-compile-generated *.txt with a single pyproject.toml
([project.dependencies] + a "dev" dependency-group) and uv.lock as the
committed, resolved lock file. The project is marked non-packaged
(tool.uv.package = false) since this is a deployable Django app, not a
distributed library, so pythonie/setup.py is removed as unused.

Dependency versions are carried over unchanged from the current
requirements/*.txt (including the yesterday's security patches);
django is capped at <6.1 to avoid an incidental minor-version bump
from the unconstrained resolution, and ruff is pinned to 0.15.16
since 0.16 changes the default lint rule set.

requirements.txt at the repo root is now generated via `uv export`
(see the new `task dependencies:export`) and kept only for Heroku's
buildpack, which expects that exact filename.

The empty requirements/production.in (a bare `-c main.txt` constraint
with no packages of its own) is dropped entirely rather than carried
over as an empty dependency-group.
…gement

Replaces the pip-compile invocations in `task dependencies:*` / `toast
deps:*` with their uv equivalents (uv lock, uv lock --upgrade[-package],
uv tree, uv run pip-audit) and adds `dependencies:export` /
`deps:export` to (re)generate the Heroku-facing requirements.txt from
uv.lock.
Replaces the requirements/*.txt-based pip/uv install with
`uv sync --frozen --all-groups` against the committed pyproject.toml
and uv.lock, and puts the resulting .venv on PATH so existing
`docker compose run ... python ...` invocations keep working
unchanged.
Swaps the requirements/*.txt pip install for astral-sh/setup-uv (with
lock-file-keyed caching) and `uv sync --all-groups`, and prefixes
every step with `uv run`. Keeps the existing checkout version and all
quality gates (ruff format, ruff check, pip-audit, Django tests)
exactly as they are on master.
Switches the pinned tool versions (Python, Task, uv) from asdf's
.tool-versions format to mise's native mise.toml, which is the
project's preferred tool version manager going forward. mise still
reads .tool-versions for compatibility, but this makes mise.toml the
canonical source.
Updates README, CONTRIBUTING, DEVELOPMENT and CLAUDE.md to reflect
`uv sync` + `uv run` instead of a manually created venv and
`pip install -r requirements/dev.txt`, mise.toml instead of
.tool-versions, and pyproject.toml/uv.lock instead of the
requirements/*.in and *.txt files. Also refreshes the stale
"Wagtail 7.2" mentions to 7.3, matching the version actually deployed.
v10 alone doesn't resolve; the action only publishes full semver tags
for its current major.
…natively

heroku/heroku-buildpack-python detects the package manager by looking
for exactly one of requirements.txt, poetry.lock or uv.lock at the
repo root, and now hard-fails the build ("Multiple Python package
manager files were found") if it sees more than one — this stopped
being just a deprecation warning. Committing both uv.lock and a
uv-export-generated requirements.txt (as the previous commit did)
would have broken every Heroku deploy.

The buildpack has native uv support: it runs
`uv sync --locked --no-default-groups` against the slug's system
Python, so no requirements.txt is needed at all. Verified locally with
`UV_PROJECT_ENVIRONMENT=<tmp> uv sync --locked --no-default-groups`
(dev group correctly excluded) and
`manage.py check --settings=pythonie.settings.production` against
that environment.

Drops the now-pointless `dependencies:export` / `deps:export` tasks
and the requirements.txt mentions in CLAUDE.md/DEVELOPMENT.md, and
documents the "never commit a requirements.txt here" constraint
directly so it isn't reintroduced later. `.python-version` (already
present, Python 3.13) continues to pin the buildpack's Python version.

Caught by an independent review pass (h/t python-validator).
@matrixise

matrixise commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author

An independent review found a real blocker: heroku-buildpack-python fails the build when it finds more than one package-manager file at the repo root (requirements.txt and uv.lock). This is no longer just a deprecation warning. I had committed a uv export-generated requirements.txt for Heroku, which would have broken the deploy.

Fix pushed (7d67c67): removed requirements.txt and the dependencies:export task. The buildpack handles uv.lock natively (uv sync --locked --no-default-groups into the slug's Python). .python-version (already present, 3.13) still drives the buildpack's Python version.

Verified:

  • locally with uv sync --locked --no-default-groups + manage.py check --settings=pythonie.settings.production
  • by running the actual buildpack (v352) against this branch in the heroku/heroku:24-build image: build, collectstatic, then manage.py check, migrate loading and gunicorn boot with the slug at /app
  • heroku buildpacks -a pythonie shows the unpinned registry heroku/python buildpack, which has supported uv since v286

@matrixise
matrixise merged commit 5ad0d96 into master Sep 25, 2026
2 checks passed
matrixise added a commit that referenced this pull request Sep 25, 2026
The previous config was written for the pip-compile workflow and pointed
at requirements/{main,dev,production}.txt, which no longer exist since
the migration to pyproject.toml + uv.lock (#247). It was a no-op.

- Drop the pip-compile / pip_requirements / pip_setup blocks; the pep621
  manager (enabled by config:recommended) picks up pyproject.toml and
  drives `uv lock` for uv.lock.
- Main vs dev split now uses depTypes instead of file paths:
  [project.dependencies] -> "project.dependencies" (never automerged),
  [dependency-groups] -> "dependency-groups" (patch automerge). pep621
  keeps the PEP 735 group name only in managerData, so it cannot be
  matched; "dev" is currently the only group. Dev patches get their own
  group so automerge is not blocked by production deps on the same branch.
- Security: the vulnerability alert settings were inside a packageRule
  restricted to patch updates (and to non-0.x versions), which excluded
  fixes needing a minor bump. Move them to the top-level
  vulnerabilityAlerts object with no update-type restriction. prPriority
  is dropped: it is not allowed there, and vulnerability PRs already
  bypass schedule and PR limits.
- Restrict the "All non-major dependencies" group to pep621 so Dockerfile
  and GitHub Actions bumps are not mixed into the Monday Python group.
- Replace the removed matchPackagePatterns / excludePackagePatterns /
  matchFiles options with matchPackageNames regexes and depTypes.
- Disable requires-python updates: Python is pinned to 3.13.x.
- Run lockFileMaintenance weekly instead of monthly: most dependencies in
  pyproject.toml have no version specifier, so Renovate skips them
  (skipReason "unspecified-version") and the uv.lock refresh is the only
  way they get updated.

Validated with `renovate-config-validator --strict --no-global` and a
local `renovate --platform=local --dry-run=extract` run.
matrixise added a commit that referenced this pull request Sep 25, 2026
The previous config was written for the pip-compile workflow and pointed
at requirements/{main,dev,production}.txt, which no longer exist since
the migration to pyproject.toml + uv.lock (#247). It was a no-op.

- Drop the pip-compile / pip_requirements / pip_setup blocks; the pep621
  manager (enabled by config:recommended) picks up pyproject.toml and
  drives `uv lock` for uv.lock.
- Main vs dev split now uses depTypes instead of file paths:
  [project.dependencies] -> "project.dependencies" (never automerged),
  [dependency-groups] -> "dependency-groups" (patch automerge). pep621
  keeps the PEP 735 group name only in managerData, so it cannot be
  matched; "dev" is currently the only group. Dev patches get their own
  group so automerge is not blocked by production deps on the same branch.
- Security: the vulnerability alert settings were inside a packageRule
  restricted to patch updates (and to non-0.x versions), which excluded
  fixes needing a minor bump. Move them to the top-level
  vulnerabilityAlerts object with no update-type restriction. prPriority
  is dropped: it is not allowed there, and vulnerability PRs already
  bypass schedule and PR limits.
- Restrict the "All non-major dependencies" group to pep621 so Dockerfile
  and GitHub Actions bumps are not mixed into the Monday Python group.
- Replace the removed matchPackagePatterns / excludePackagePatterns /
  matchFiles options with matchPackageNames regexes and depTypes.
- Disable requires-python updates: Python is pinned to 3.13.x.
- Run lockFileMaintenance weekly instead of monthly: most dependencies in
  pyproject.toml have no version specifier, so Renovate skips them
  (skipReason "unspecified-version") and the uv.lock refresh is the only
  way they get updated.

Validated with `renovate-config-validator --strict --no-global` and a
local `renovate --platform=local --dry-run=extract` run.
@matrixise
matrixise deleted the feature/migrate-to-uv-171-v2 branch September 25, 2026 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant