Repository navigation
Conversation
Signed-off-by: Pengyi Peng <pengpengyi92@users.noreply.github.com>
pengpengyi92
requested review from
feussy,
hintse,
ptomecek and
timkpaine
as code owners
November 12, 2026 06:43
timkpaine
approved these changes
Nov 12, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #295 +/- ##
=======================================
Coverage 93.84% 93.85%
=======================================
Files 205 205
Lines 22920 22944 +24
Branches 1509 1511 +2
=======================================
+ Hits 21509 21533 +24
Misses 1120 1120
Partials 291 291 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
2 of 11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
add_field_metadata()rebuilds the Arrow schema with new field metadata, butpa.schema(new_schema)silently discards the original schema-level metadata.This happens even when the supplied mapping is empty or only names an absent
column.
In particular,
Table.from_pandas()stores index reconstruction informationunder the schema's
pandaskey. Annotating a column currently resets a namedRangeIndex, or turns named Index/MultiIndex fields into ordinary data columns
on
to_pandas().Retain
table.schema.metadatawhen constructing the replacement schema. Theexisting selected-field replacement semantics, untouched fields, data types,
nullability, and values remain unchanged. No dependency or API changes.
The tests cover raw/binary table metadata, empty/unmatched mappings, named
RangeIndex/Index/MultiIndex round trips, and the existing selected-field
replacement behavior with no schema metadata.
Type of Change
Validation
Windows, Python 3.12.13, PyArrow 26.0.1, pandas 3.1.1, Polars 2.1.0.
With the new regression tests and the original implementation, all six
metadata/index subtests fail; the other seven test cases pass. With the fix,
the utility module reports seven passed and six subtests passed.
The broader local selection (all utils plus the Arrow and file-publisher
tests) is not fully green on this Windows host: 319 passed, 24 failed.
The clean upstream commit
05ce2450aaa53d865bca9cb68903900b039fa032has316 passed and the exact same 24 failing test IDs under the same interpreter
and dependencies. Failures include Windows/Hydra path assumptions, CLI tests,
and temporary-directory access errors; this PR does not change those areas.
Checklist
make lint/make testCI matrix (not claimed locally)Out of scope: metadata handling in type-conversion helpers, metadata merging,
recursive conversion, storage adapters, and unrelated Windows test fixes.