Skip to content

fix: resolve subscriber trace scope through forwards and same-process units - #15

Merged
cboulay merged 1 commit into
devfrom
fix/profiling-trace-subscriber-scope
Aug 24, 2026
Merged

cboulay merged 1 commit into
devfrom
fix/profiling-trace-subscriber-scope

Conversation

@cboulay

@cboulay cboulay commented Aug 24, 2026

Copy link
Copy Markdown
Member

Starting a profiling trace on a publisher plotted a single line. It was the publish delta — the two subscriber metrics never arrived at all, because the dashboard asked the processes for a scope that could not match anything.

Same-process subscribers were never enabled

The publisher's own route unit was scoped with subscriber_topics set to the publisher's own topic. A subscriber is registered under its own input topic, never under the topic of the publisher feeding it, so that filter disabled every subscriber in the publisher's process. The resolved downstream topics only ever reached other processes' route units, and the publisher's own was explicitly discarded from that fan-out (route_units_for_subscribers.discard(route_unit)).

In a graph whose publisher/subscriber edges are intra-process — an ordinary single-process pipeline, or any collection whose units share a process — that left lease_time_ns and user_span_ns disabled everywhere. The publisher's route unit now carries the same resolved scope the remote ones get.

The scope walked only one hop

A publisher nested in collections reaches its subscribers through one forwarding topic per boundary, and only the topic at the end of that chain is one a subscriber holds:

HUB1/SDA/OUTPUT_SIGNAL -> HUB1/UCDF/INPUT_SIGNAL -> HUB1/UCDF/BUTTER0/INPUT_SIGNAL

One hop finds UCDF/INPUT_SIGNAL — a pure forward with no unit behind it — and stops, so HUB1/SDA reported "No subscriber profiling data is available for this topic" and could not be traced at all. _topic_scope_for_seed and topicScopeForPublisher now walk the chain to exhaustion. Forwards terminate at subscriber input topics, so the closure stays inside the publisher's downstream, and a visited set keeps a cyclic graph from spinning.

Diagnosability

The trace-control response now reports the resolved topic_scope. Without it, an empty route_units (every subscriber lives in the publisher's own process) is indistinguishable from a scope that resolved to nothing — which is what made the original failure so hard to see from outside.

Verification

Against a live 3-process pipeline with two levels of collection nesting, replaying the exact request the UI sends:

trace target before after
HUB1/UCDF/BUTTER0/OUTPUT_SIGNAL publish delta only + 1105 lease_time_ns, 1105 user_span_ns on LRR/INPUT_SIGNAL
HUB1/SDA/OUTPUT_SIGNAL "No subscriber profiling data" scope resolves through both forwards; 501 samples of each metric on UCDF/BUTTER0/INPUT_SIGNAL

ezmsg core needed no change — its publisher/subscriber filtering was correct once handed a scope that matches.

Tests

  • 58 backend, 44 frontend unit (4 new covering the forward chain, fan-out, cycles and relay aliases), 32 Playwright e2e — all passing; ruff clean.
  • Two backend tests asserted the old subscriber_topics values and were updated; one new test covers the forward chain together with the in-process case.
  • Frontend rebuilt and the vendored _web bundle refreshed per docs/development.md.

🤖 Generated with Claude Code

… units

Starting a profiling trace on a publisher plotted a single line. It was the
publish delta -- the two subscriber metrics never arrived at all, because the
dashboard asked the processes for a scope that could not match anything.

The publisher's own route unit was scoped with `subscriber_topics` set to the
publisher's own topic. A subscriber is registered under its own input topic,
never under the topic of the publisher feeding it, so that filter disabled
every subscriber in the publisher's process. The downstream topics only ever
reached other processes' route units, and the publisher's own route unit was
explicitly discarded from that fan-out. In a graph whose publisher/subscriber
edges are intra-process -- an ordinary single-process pipeline, or any
collection whose units share a process -- that left lease time and user span
disabled everywhere. The publisher's route unit now carries the same resolved
downstream scope the remote ones get.

The scope itself only walked one route hop. A publisher nested in collections
reaches its subscribers through one forwarding topic per boundary, and only the
topic at the end of that chain is one a subscriber holds. One hop finds the
first alias and stops, so a publisher two or more boundaries away from its
subscriber reported "No subscriber profiling data is available for this topic"
and could not be traced at all. Both the service and the panel now walk the
forward chain to exhaustion; forwards terminate at subscriber input topics, so
the closure stays inside the publisher's downstream, and a visited set keeps a
cyclic graph from spinning.

The trace-control response now reports the resolved `topic_scope`. Without it
an empty `route_units` -- every subscriber lives in the publisher's own process
-- is indistinguishable from a scope that resolved to nothing, which is what
made the original failure so hard to see from outside.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cboulay
cboulay merged commit 4a9a355 into dev Aug 24, 2026
11 checks passed
@cboulay
cboulay deleted the fix/profiling-trace-subscriber-scope branch August 24, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant