Skip to content

chore: allow huggingface-hub 2.x - #1117

Open
davanstrien wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
davanstrien:davanstrien/chore/huggingface-hub-2x
Open

davanstrien wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
davanstrien:davanstrien/chore/huggingface-hub-2x

Conversation

@davanstrien

Copy link
Copy Markdown
Contributor

📋 Summary

Widens the huggingface-hub cap to <3 and upgrades the lock to 2.4.0, so Data Designer can be installed alongside packages that need hub 2.x. Data Designer's own Hub calls needed no changes. The only source change is one logger name.

🔗 Related Issue

Fixes #1110

🔄 Changes

  • Widen the cap to huggingface-hub>=1.0.1,<3 in data-designer and data-designer-engine
  • Upgrade the lock to hub 2.4.0. This also moves click to 8.4.2, hf-xet to 1.8.0, sqlfluff to 4.4.0 (4.2.1 caps click at <8.4) and datasets to 5.1.0 (5.0.1 caps hub at <2), and adds httpx2, httpcore2, httpx2-jsfetch and truststore
  • Add httpx2 to _DEFAULT_NOISY_LOGGERS. Hub 2.x sends requests through httpx2, so without this every Hub request logs at INFO, including signed CDN URLs
  • Fix test_app_rejects_unknown_options_before_script_separator for click 8.4

A correction to the issue: the test doesn't fail because of line wrapping. Click 8.4 changed the wording from No such option: --x to No such option '--x'., so normalising whitespace wouldn't fix it. The test now checks No such option and --num-recrods separately, which passes on click 8.3.3 and 8.4.2.

🧪 Testing

  • make test passes (run as pytest per package in a locked environment on macOS, Python 3.12: config 830 passed, engine 2862 passed, interface 1309 passed and 11 skipped; the httpx2 line came after this run, and the 37 logging tests were re-run after it)
  • Unit tests added/updated
  • E2E tests added/updated (if applicable)

I ran two live smoke tests on hub 2.4.0 against temporary private dataset repos, deleted afterwards. The scripts aren't part of this PR.

Small run:

  • push_to_hub_from_folder to a new repo, then again to the same repo
  • Dataset card load
  • An invalid token gives the expected authentication error

Larger run (242 MB: 3 parquet batches of 40,000 rows, 20 images, 1 processor output):

  • push_to_hub_from_folder uploaded every file in 18 s, and a downloaded batch matched by SHA-256
  • HuggingFaceSeedReader returned the expected columns and 120,000 rows
  • A full DuckDB scan through HfFileSystem took 6 s, and the row count and checksums matched
  • No HTTP Request lines appeared in the log after configure_logging()

Not tested: a live run on hub 1.x (44 mocked Hugging Face integration tests pass on 1.0.1), and custom CA certificates (SSL_CERT_FILE) under the new HTTP stack.

Notes for review

  • hf-xet 1.8.0 was released on 2026-10-30, the day this lock was generated. Hub only needs >=1.6, so it's easy to hold back if you prefer.
  • typer>=0.12 looks too low, separately from this PR: typer 0.15.2 --help raises TypeError on click 8.3.3 as well as 8.4.2, and 0.16.0 works on 8.4.2. I left it for a follow-up.
  • Running the data-designer test suite rewrites the repo's uv.lock using the local user's uv config. This also looks separate from this PR, and I haven't traced it to a specific test (the plugin install tests are my first guess). It's worth knowing if you have a custom index or exclude-newer in your uv config.

✅ Checklist

  • Follows commit message conventions
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

🤖 Generated with Claude Code

@davanstrien
davanstrien requested a review from a team as a code owner October 30, 2026 12:02
@greptile-apps

greptile-apps Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the previous finding is fixed and no new actionable issues remain.

Summary

Allows huggingface-hub 2.x alongside the existing 1.x range.

  • Data Designer packages can use Hugging Face Hub versions below 3.
  • Default logging keeps Hub 2.x request messages quiet.

Reviews (3) · Last reviewed commit: "test: annotate noisy logger test paramet..." · Reviewed by Greptile

@andreatnvidia

Copy link
Copy Markdown
Contributor

Thanks for this! Supporting Hub 2.x while keeping 1.x compatibility is useful. We’ll confirm which release to target shortly.



_DEFAULT_NOISY_LOGGERS = ["httpx", "matplotlib", "mcp"]
_DEFAULT_NOISY_LOGGERS = ["httpx", "httpx2", "matplotlib", "mcp"]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we extend test_configure_logging_silences_default_noisy_loggers to cover httpx2 too? Parametrizing it over httpx and httpx2, and resetting each logger’s level before configuring logging, would protect against regressions and test-order effects. Non-blocking suggestion.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d363fa7, with type annotations added in 0c0cc2e.

@andreatnvidia

Copy link
Copy Markdown
Contributor

Could we add a patched httpx2 minimum to the existing workspace/e2e security constraints and regenerate constraints.txt? The locked 2.16.0 is fine, but Hub’s dependency range also permits older releases with known advisories. This would follow the existing approach for transitive security floors. Non-blocking suggestion.

davanstrien and others added 2 commits November 2, 2026 09:22
Widen the huggingface-hub cap to <3 and upgrade the lock to 2.4.0.
Quieten the httpx2 logger, which hub 2.x uses for requests.
Update the CLI test for the new click 8.4 error message.

Fixes NVIDIA-NeMo#1110

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Daniel van Strien <davanstrien@gmail.com>
Add httpx2>=2.12.0 to the workspace and e2e security constraints and
regenerate constraints.txt. Hub 2.x allows httpx2>=2.0.0, and 2.12.0 is
the lowest version that patches all published advisories.

Parametrize the noisy logger test over httpx, httpx2 and matplotlib, and
reset each logger's level first to avoid test-order effects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Daniel van Strien <davanstrien@gmail.com>
@davanstrien
davanstrien force-pushed the davanstrien/chore/huggingface-hub-2x branch from 384ba4c to d363fa7 Compare November 2, 2026 09:24
@davanstrien

Copy link
Copy Markdown
Contributor Author

Thanks, both suggestions done in d363fa7, and I rebased on main.

  • Added httpx2>=2.12.0 to the workspace and e2e constraints and regenerated constraints.txt. 2.12.0 is the lowest version that patches all five published advisories. The lock stays on 2.16.0.
  • Parametrized the noisy logger test over httpx, httpx2 and matplotlib, with each level reset first. The httpx2 case fails if the logger is removed from the defaults.

make check-vulnerabilities is clean, and the logging, CLI and security constraint tests pass. I didn't re-run the full tests locally after the rebase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Daniel van Strien <davanstrien@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow huggingface-hub 2.x (widen cap from <2 to <3)

2 participants