CI: provision Python via simple_python(); pin pandas<3 - #7
Open
jefferis wants to merge 2 commits into
Open
Conversation
Provision CI Python through nat.python's own simple_python() engine (pip pandas into the managed r-reticulate env) instead of reticulate::py_install(), which pulled a conda-forge pandas 3.0.5 that would not import against numpy 2.5.2 -- silently skipping every pandas2df() test and dropping coverage. The test helper now distinguishes an absent module (skip) from one that is installed but fails to import (error, surfacing the Python error) so a broken environment can no longer masquerade as a benign skip.
pandas 3.0 defaults string columns to Arrow-backed dtype when pyarrow is present; pandas2df() does not yet convert ArrowStringArray back to R, so string columns returned as raw python objects and the fast-path tests failed. Pin the baseline until pandas2df() gains Arrow-string support.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #7 +/- ##
===========================================
+ Coverage 47.62% 66.18% +18.55%
===========================================
Files 6 6
Lines 485 485
===========================================
+ Hits 231 321 +90
+ Misses 254 164 -90 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Provisions Python in CI through nat.python's own
simple_python()— the exact end-user path — instead of a bespokereticulate::py_install()call, and pins the baseline topandas < 3for now.Why
On PR #6, coverage came out at 47.6% (vs an estimated 66%): a conda-forge pandas 3.0.5 build would not import against the installed numpy, and the test helper collapsed "installed but import fails" into "absent", so five convert tests silently skipped rather than failing. This branch fixes both the provisioning and the diagnostic gap.
Changes
.github/workflows/{R-CMD-check,test-coverage}.yaml— install nat.python viapak::local_install(), then provision withsimple_python("minimal")(coverage leg addspyarrow), and pinRETICULATE_PYTHONfor the run.tests/testthat/helper-nat.python.R—py_module_status()distinguishesok/broken/absent;skip_if_no_module()now errors when a module is installed but fails to import, and only skips when genuinely absent — so a broken install can never masquerade as a skip again.R/env.R—simple_python()pins the baseline topandas < 3for now (ecosystem caution while pandas 3 settles).pandas 3 support in
pandas2df()(and lifting this pin) is handled separately in the stackedpandas3-supportbranch.