You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
resolve_metric_params treats all three as required — it raises if any is missing from both the model overrides and the profile (metric_catalog.py:180-190). BASE_PROFILE duly supplies all three:
Two of the three do nothing. The kernel passes them to np.histogram_bin_edges:
edges=np.histogram_bin_edges(preds, bins=bins, range=(low_bin, high_bin)) # line 162
and NumPy ignores range= entirely when bins is a sequence. Verified 2026-08-02:
b= [0, 0.5, 2.5, 5.5, 10.5, 25.5, 50.5, 100.5, 250.5, 500.5, 1000.5]
np.array_equal(np.histogram_bin_edges(p, bins=b, range=(0, 10000)),
np.histogram_bin_edges(p, bins=b, range=(0, 1)))
# True ← range has no effect
So the genome contract advertises three tuning knobs, two of which are inert. A researcher who widens high_bin to fix the out-of-range crash from #32 would see no change at all — a false sense of control over the metric, and directly contrary to the epic doctrine's "no magic numbers, no defaults, no silent no-ops".
Note documentation/CICs/MetricCatalog.md states the invariant in one direction only — "All hyperparameters required by a metric function must be declared in the spec's genome tuple" — with no converse requirement that every declared param actually be used. That gap is what let this through.
⚠ Decision required before implementing
Two viable options. Decide and record the choice in this issue before writing code.
Option A — Drop low_bin/high_bin (recommended). genome=("bins",); remove both keys from BASE_PROFILE and HYDRANET_UCDP_PROFILE if present. Explicit bin edges are the only form actually supported, and after #32 the edges are also what the error message reports. Simplest, and makes the contract honest. Cost: a breaking change for any external profile supplying those keys — resolve_metric_params rejects unknown params (metric_catalog.py:169-174), so a stale profile will now raise. That is correct fail-loud behaviour, but it must be noted in #38 and governed by #28's deprecation policy.
Option B — Honour them.
Keep the genome and make the kernel use range only when bins is an integer count, raising if bins is a sequence andlow_bin/high_bin are supplied (contradictory configuration). Preserves the tuning knob but adds a branch and a second supported bin-specification mode that nothing currently uses.
Recommendation: A. No caller uses integer bins; keeping a parameter alive to preserve an unused code path adds surface without value.
The work (assuming Option A)
metric_catalog.py:88-89 — genome=("bins",).
profiles/base.py:34-38 — remove low_bin and high_bin from the Ignorance entry.
Check profiles/hydranet_ucdp.py — it spreads **BASE_PROFILE, so it inherits the change; confirm no local override.
native_metric_calculators.py:142-173 — drop the low_bin/high_bin parameters from the signature and the np.histogram_bin_edges call. Keep the keyword-only * marker so ADR-042's no-defaults rule still holds.
The decision (A or B) is recorded in this issue with its rationale before code is written
Option A: Ignorance genome is ("bins",); low_bin/high_bin gone from all profiles and from the kernel signature
resolve_metric_params("Ignorance", {}, BASE_PROFILE) returns only the params the function actually consumes
A profile still supplying low_bin/high_binfails loudly (existing unknown-param rejection) — assert this with a Red test rather than leaving it implicit
Ignorance values are numerically unchanged for in-range input — test_ignorance_known_bin_distribution passes unmodified
TestCatalogStructuralIntegrity passes — in particular test_base_profile_covers_all_implemented_genomes
No logging added (Level 0 exempt)
Full suite green, ruff clean
Validation
conda run --name views_pipeline pytest tests/test_metric_catalog.py -v
conda run --name views_pipeline pytest tests/test_metric_calculators.py -v -k "ignorance or Ignorance"
conda run --name views_pipeline pytest tests/ -q
conda run --name views_pipeline python - <<'PY'from views_evaluation.evaluation.metric_catalog import METRIC_CATALOG, resolve_metric_paramsfrom views_evaluation.profiles.base import BASE_PROFILEprint("genome:", METRIC_CATALOG["Ignorance"].genome)print("resolved:", resolve_metric_params("Ignorance", {}, BASE_PROFILE))try: resolve_metric_params("Ignorance", {"high_bin": 10000}, BASE_PROFILE) print("SILENT PASS - BUG: stale param accepted")except ValueError as e: print("stale param rejected OK ->", str(e)[:80])PY
Epic: #26 · Wave 2 · Closes: register C-28(b) · Blocked by: #32 · Needs a decision before implementation
Background
Ignorancedeclares a three-parameter genome (metric_catalog.py:88-89):resolve_metric_paramstreats all three as required — it raises if any is missing from both the model overrides and the profile (metric_catalog.py:180-190).BASE_PROFILEduly supplies all three:Two of the three do nothing. The kernel passes them to
np.histogram_bin_edges:and NumPy ignores
range=entirely whenbinsis a sequence. Verified 2026-08-02:So the genome contract advertises three tuning knobs, two of which are inert. A researcher who widens
high_binto fix the out-of-range crash from #32 would see no change at all — a false sense of control over the metric, and directly contrary to the epic doctrine's "no magic numbers, no defaults, no silent no-ops".Note
documentation/CICs/MetricCatalog.mdstates the invariant in one direction only — "All hyperparameters required by a metric function must be declared in the spec's genome tuple" — with no converse requirement that every declared param actually be used. That gap is what let this through.⚠ Decision required before implementing
Two viable options. Decide and record the choice in this issue before writing code.
Option A — Drop
low_bin/high_bin(recommended).genome=("bins",); remove both keys fromBASE_PROFILEandHYDRANET_UCDP_PROFILEif present. Explicit bin edges are the only form actually supported, and after #32 the edges are also what the error message reports. Simplest, and makes the contract honest.Cost: a breaking change for any external profile supplying those keys —
resolve_metric_paramsrejects unknown params (metric_catalog.py:169-174), so a stale profile will now raise. That is correct fail-loud behaviour, but it must be noted in #38 and governed by #28's deprecation policy.Option B — Honour them.
Keep the genome and make the kernel use
rangeonly whenbinsis an integer count, raising ifbinsis a sequence andlow_bin/high_binare supplied (contradictory configuration). Preserves the tuning knob but adds a branch and a second supported bin-specification mode that nothing currently uses.Recommendation: A. No caller uses integer
bins; keeping a parameter alive to preserve an unused code path adds surface without value.The work (assuming Option A)
metric_catalog.py:88-89—genome=("bins",).profiles/base.py:34-38— removelow_binandhigh_binfrom theIgnoranceentry.profiles/hydranet_ucdp.py— it spreads**BASE_PROFILE, so it inherits the change; confirm no local override.native_metric_calculators.py:142-173— drop thelow_bin/high_binparameters from the signature and thenp.histogram_bin_edgescall. Keep the keyword-only*marker so ADR-042's no-defaults rule still holds.MetricCatalog.md(S13: Update Intent Contracts for changed behaviour (ADR-021) #39): every declared genome param must be consumed by its function. A test for this is in scope for S14: Restore a documentation-contract test guard and wire validate_docs.sh into CI #40 if cheap.Acceptance criteria
Ignorancegenome is("bins",);low_bin/high_bingone from all profiles and from the kernel signatureresolve_metric_params("Ignorance", {}, BASE_PROFILE)returns only the params the function actually consumeslow_bin/high_binfails loudly (existing unknown-param rejection) — assert this with a Red test rather than leaving it implicitIgnorancevalues are numerically unchanged for in-range input —test_ignorance_known_bin_distributionpasses unmodifiedTestCatalogStructuralIntegritypasses — in particulartest_base_profile_covers_all_implemented_genomesruffcleanValidation
Dependencies
Files
views_evaluation/evaluation/metric_catalog.py:88-89views_evaluation/evaluation/native_metric_calculators.py:142-173views_evaluation/profiles/base.py:34-38views_evaluation/profiles/hydranet_ucdp.py(verify only)tests/test_metric_catalog.py,tests/test_metric_calculators.py