Never let a class-connectivity backend failure read as count 0 - #111
Merged
Conversation
_aggregate_class_connectivity returned [] on five infrastructure
failures (Neo4j membership query raising or returning False, per-instance
connectivity unobtainable for every instance, partner classes unresolved,
partner membership query failing). The callers turned that into
{'count': 0}, @with_solr_cache stored it (count >= 0 is "valid"), and
the v3-cached nginx edge pinned the 200 for CACHE_STALE_TIME. One
transient outage became a month of "There is no data to display" for
gamma Kenyon cell downstream connectivity (2026-09-04).
Raise ConnectivityBackendError on those paths instead. The decorator
does not catch it and ha_api answers 5xx, which neither cache stores.
A real zero (no instances, or no positive-weight edges) is unchanged.
A result computed with some instances' connectivity missing is now
tagged 'partial': the Solr result cache refuses to store it and ha_api
sends X-Accel-Expires: 600 so the edge keeps the underestimate only
briefly. Any empty run_query result is sent with X-Accel-Expires: 0
(plus Cache-Control: no-store) so the edge never stores an empty body;
a true zero is served from the Solr cache in milliseconds, so the cost
is negligible. X-Accel-Expires is honoured by owl_cache's nginx as-is
(only Cache-Control/Expires/Set-Cookie are in proxy_ignore_headers), so
no edge change or redeploy is needed.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Gamma Kenyon cell (
FBbt_00100247) showed "There is no data to display" for DownstreamClassConnectivity on 2026-09-04 even though the backend holds 3,886 rows. Thecount: 0was not only pinned at the v3-cached nginx edge — a cache-busted request to the origin returned the same zero, so the origin had produced it and the Solr result cache had kept it._aggregate_class_connectivityhad fivereturn []paths that are infrastructure failures, not "no partners": the Neo4j membership query raising, or returningFalse(whichdict_cursorsilently turns into[]); per-instance connectivity unobtainable for every instance; partner classes unresolved although the instances have edges; and the partner membership query failing. The callers turned each into{'count': 0, 'rows': []},@with_solr_cachestored it (count >= 0is its definition of valid), and the edge kept the 200 forCACHE_STALE_TIME(one month). Three caches, one lie.Changes
vfb_queries: those paths now raiseConnectivityBackendError.@with_solr_cachedoes not catch it andha_apianswers 5xx, which neither cache stores. A true zero — a class with no connectivity instances, or none with a positive-weight edge to an in-scope partner class — is unchanged and still cacheable.vfb_queries/solr_result_cache: a result computed with some instances' per-instance connectivity missing (counts are underestimates) is taggedpartial. The Solr result cache serves it but refuses to store it, so the next call recomputes rather than inheriting wrong counts.ha_api:/run_querynow setsX-Accel-Expiresper response. Empty results (count == 0) getX-Accel-Expires: 0+Cache-Control: no-store, so the edge never stores an empty body; a real zero costs one origin hit per visitor, answered from the Solr cache in milliseconds. Partial results getX-Accel-Expires: 600(VFBQUERY_PARTIAL_EDGE_TTL). Everything else keeps the edge default. owl_cache's nginx honoursX-Accel-Expiresas-is (onlyCache-Control/Expires/Set-Cookieare inproxy_ignore_headers), so no owl_cache change or redeploy is needed. Emptiness is judged on the full stored result, not the page, so a page past the end of a non-empty result is not treated as "no data".How to test
src/test/test_class_connectivity_no_silent_zero.py— 15 backend-free tests that monkeypatch every Neo4j/Solr seam of_aggregate_class_connectivityand pin: the two true-zero cases still return[]; each failure path raises; partial coverage is reported and flagged; the Solr cache treats a flagged result as not cacheable; and the header logic (0for empty,600for partial, none otherwise, page-past-end not empty).test_ha_api_dispatch,test_ha_api_validationandtest_solr_cache_failoverpass unchanged (36 tests).Live:
get_downstream_class_connectivity('FBbt_00100247')→ 3,886 rows,get_upstream_class_connectivity→ 4,231, neither flagged partial, both from the Solr cache in under a second.After release, a cold request that hits an outage should return 5xx (edge
MISS, nothing stored) instead of a cached zero:Follow-ups
except … return []) exists in other query functions; this PR fixes the class-connectivity family only. Worth a sweep.get_term_info'scount: -1two-phase placeholder is a separate edge-poisoning route and is not touched here.