Compare data-keyed result maps by shape, not by key - #107
Merged
Conversation
test_live_result_matches_recorded_shape treated every key in a recording as schema, so a backend content change could fail CI as a missing key -- the opposite of this module's stated contract, that content changes must not fail while a key or type disappearing from a payload must. Four recorded maps are keyed by data: Examples and Images by template short_form, Domains and Licenses by index. Which keys they carry follows whichever images the indexer picked. Class documents cap anatomy_channel_image at ten entries however many images a class really has -- 56,384 for adult cholinergic neuron, 12 for medulla -- so a class whose only image on some template falls outside that ten loses the whole template key. That is what happened to medulla (FBbt_00003748) on 2026-09-03: it lost VFB_00030786, adult brain template Ito2014, while VFB_00030810 still had its in_register_with edge in the PDB, its own SOLR document, and every image file serving 200. Compare those four the way lists are already compared: the map must still be non-empty and one value must still have the recorded shape, but the key set is free. Meta is deliberately not in the set -- its keys are the schema -- and neither is headers, where a lost column is a real regression. Verified against the live backend: the full example suite passes, including the term_info_FBbt_00003748 case that has failed since 1 Sep.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compare data-keyed result maps by shape, not by key
test_live_result_matches_recorded_shapetreated every key in a recording as schema, so a backend content change could fail CI as a missing key — the opposite of this module's stated contract, that content changes must not fail while a key or type disappearing from a payload must.Four recorded maps are keyed by data:
ExamplesandImagesby template short_form,DomainsandLicensesby index. Which keys they carry follows whichever images the indexer picked, and the class documents capanatomy_channel_imageat ten entries however many images a class really has:So a class whose only image on some template falls outside that ten loses the whole template key. That is what happened to medulla on 2026-09-03: it lost
VFB_00030786(adult brain template Ito2014), whileVFB_00030810still had itsin_register_withedge in the PDB, its own SOLR document, and every image file serving 200. Nothing was missing from the backend.These four are now compared the way lists already are — non-emptiness plus the shape of one value, with a free key set.
Metais deliberately excluded because its keys are the schema, as isheaders, where a lost column is a real regression.How to test
pytest src/test/test_example_queries.pyagainst the live backend — 15 passed here, including theterm_info_FBbt_00003748case that has failed since 1 Sep. The two new unit tests need no backend.Follow-up, not in this PR
The other image that missed medulla's cap,
VFB_00107fob("ME_R on JRC2018Unisex"), has no SOLR document of its own and itsvolume.obj404s whilevolume.nrrdandvolume_man.objserve. That one looks like a genuine defect.