Skip to content

fix: pair embedding vocabulary with heterogeneous dataset axes - #35

Draft
jkobject wants to merge 2 commits into
mainfrom
fix/ntv3-heterogeneous-axis-mapping
Draft

fix: pair embedding vocabulary with heterogeneous dataset axes#35
jkobject wants to merge 2 commits into
mainfrom
fix/ntv3-heterogeneous-axis-mapping

Conversation

@jkobject

Copy link
Copy Markdown
Owner

Problem

scPRINT historically exposed data.gene_embeddings and paired it in cli.py to model.precpt_gene_emb. The parameter was moved model-side in 2025, but multi-species artifacts have heterogeneous native VAR axes; merely filtering model embeddings after DataModule construction does not preserve a shared positional mapping.

Changes

  • restore gene_embeddings as an explicit DataModule source for the model vocabulary
  • read only the parquet index, intersect it with the registry, and preserve embedding order
  • precompute one native-axis → model-axis mapping per unique artifact axis
  • make Collator apply the storage-specific mapping before gene sampling
  • accept canonical _store_idx while retaining _storage_idx compatibility
  • make genes_dict preserve the embedding-subset order used by the model
  • fail closed on duplicate global genes, empty axes, low-overlap axes, and expression/axis length mismatch

Validation

4 targeted tests passed
ruff passed
py_compile passed
git diff --check passed

The broader historical tests/test_base.py invocation exceeded the bounded local runtime and is not claimed green here. The scPRINT-2 PR will pin the immutable reviewed SHA for the distributed smoke.

Downstream

Required by cantinilab/scPRINT-2#17.

@jkobject

Copy link
Copy Markdown
Owner Author

Downstream pairing

Consumed by cantinilab/scPRINT-2#17. That PR restores data.gene_embeddings -> model.precpt_gene_emb and pins this PR head 0a2fb2080dd2bf471c1532c04ee0789ce06441c2.

Targeted validation at this head: 5 mapping/sampler tests, Ruff, py_compile and diff-check. The historical broad test_base.py command exceeded the bounded local runtime and is not claimed green.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 76.47059% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.18%. Comparing base (831fabd) to head (0a2fb20).

Files with missing lines Patch % Lines
scdataloader/datamodule.py 75.60% 10 Missing ⚠️
scdataloader/collator.py 77.77% 6 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #35      +/-   ##
==========================================
+ Coverage   48.84%   53.18%   +4.33%     
==========================================
  Files          10       10              
  Lines        1984     2044      +60     
==========================================
+ Hits          969     1087     +118     
+ Misses       1015      957      -58     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants