Skip to content

feat: live data viewer for graph publishers - #16

Merged
cboulay merged 5 commits into
devfrom
feat/live-data-viewer
Sep 1, 2026
Merged

cboulay merged 5 commits into
devfrom
feat/live-data-viewer

Conversation

@cboulay

@cboulay cboulay commented Aug 25, 2026

Copy link
Copy Markdown
Member

Adds a live view of the data flowing through any publisher, alongside the
existing topology, settings and profiling views.

Click View Data on an expanded publisher row and a dock opens showing what
that publisher is sending.

How it works

The backend already holds a GraphContext, so it subscribes to the topic
directly — no unit injected, no shared-memory hop, no helper process. The tap
adds no edge to the topology and does not appear in profiling (both verified
against a live graph).

Two properties of ezmsg's Subscriber carry the design:

  • leaky=True releases publisher backpressure for every notification it
    drops, so a wedged browser tab cannot stall the graph it is watching. This is
    the first thing to check in any "tap a live pipeline" scheme and ezmsg already
    provides it.
  • recv_zero_copy() lends the message without pickling it. Everything
    downstream either copies into the ring or copies explicitly.

Leakiness costs contiguity, so each tap owns a ring written at message rate and
read at frame rate, and reports gaps instead of drawing across them.

Views

The view follows the data, and a stream supporting more than one offers a
dropdown — switching is a different reading of the same frames, so it needs no
re-subscription.

stream opens as also offers
(time, ch) sweep channel map, when the ch axis carries real x/y
(freq, ch) spectrum —
(ch) + positions channel map sweep
anything else message inspector —

The inspector reports type, dims, axes and attrs for any topic, including
ones no plot can draw.

Cost to the graph

Sample data is reduced before it is sent — to the plot's pixel width, over a
selectable time window. Bandwidth follows columns per second rather than the
publisher's sample rate:

stream measured
8 ch @ 1 kHz 0.1 MB/s
256 ch @ 30 kHz 0.7 MB/s
2048 ch @ 1 kHz 5.8 MB/s

Optional dependency

Plotting needs the viz extra (pip install "ezmsg-dashboard[viz]"), which
adds ezmsg-tools for its AxisArray-to-plottable reduction — pure numpy, its Qt
and phosphor dependencies sit behind its own extras. Without it the message
inspector still works on every topic and only the plotting views are lost.

Two limits worth knowing

  • The dashboard can only read topics whose message class it can import.
    ezmsg unpickles in the subscribing process, so a type defined in a script's
    __main__, or from a package the dashboard's environment lacks, never
    arrives. The panel detects this (an active publisher with no messages) and
    says so rather than looking idle. Fixing it properly needs something at the
    ezmsg layer.
  • A stream wider than the browser's WebGL texture limit (8192 channels on
    current hardware, 2048 at the spec minimum) cannot be held in a plot buffer.
    The panel reports that and falls back to the inspector rather than drawing a
    subset under the whole stream's labels. Below that, wide streams are scrolled,
    512 lanes at a time.

Notes for review

Some non-obvious choices, each arrived at by fixing something that failed
silently — the code comments carry the detail:

  • The renderer's ring is a texture, not a vertex buffer. The obvious
    strided-attribute layout works to 63 channels and then draws nothing:
    vertexAttribPointer rejects a stride above 252 bytes, while uploads and draw
    calls still succeed.
  • The decimation ratio comes from a time window, not the frame rate.
    Without it a 30 kHz stream showed 33 ms and repainted entirely every frame.
  • Autoscale measures only the visible lanes, over the whole window. Scanning
    every channel lets a loud feature flatten a quiet one against the rail;
    measuring one frame rather than the window clips the trace.
  • bfloat16 is widened by reinterpreting its bits in numpy. Asking MLX to
    cast returns NaN and ~2.5e38 for an array that arrived over ezmsg's transport,
    because the cast runs on a device that cannot address the buffer — silently
    wrong rather than failing.

Testing

156 backend tests, 82 frontend unit tests, 32 Playwright e2e, ruff clean.

frontend/scripts/check-stream-panel.mjs additionally drives the dashboard in a
real browser against examples/stream_demo_graph.py and asserts each view
renders — shader compilation and "is anything drawn" cannot fail in jsdom. See
the Development Guide for how to run it.

Verified by hand against a live 256-channel 30 kHz pipeline, including
MLX-backed (float32, float16, bfloat16) payloads.

🤖 Generated with Claude Code

cboulay and others added 5 commits August 25, 2026 14:44
The dashboard could show that a publisher was sending, but not what. This
adds a `/ws/stream` endpoint that subscribes to any topic and streams its
samples to the browser.

The backend already holds a GraphContext, so it subscribes directly rather
than injecting a unit or brokering shared memory: no edge is added to the
topology and the tap does not appear in profiling. Two properties of ezmsg's
Subscriber carry the design. `leaky=True` releases publisher backpressure for
every notification it drops, so a wedged browser cannot stall the graph it is
watching; `recv_zero_copy()` lends the message without pickling it, so the
reduction happens in place and nothing is retained past the lease.

Leakiness costs contiguity, so each tap owns a ring written at message rate
and read at frame rate, and reports gaps rather than drawing across them.

Sample data is reduced before it is sent, to the client's pixel budget over a
requested time window. The window is what makes this affordable: without it a
frame's samples are spread across the whole plot, so a 30 kHz stream shows
33 ms and repaints entirely every frame. Fixing the window in time makes a
column a fixed number of samples, and the wire cost follow columns per second
rather than the publisher's rate -- 2048 channels measures at 5.8 MB/s.

Payloads travel as binary frames, `(n_out, n_channels, 2)` min/max pairs,
which is also the memory layout of the texture the browser uploads them into.

A message inspector reports type, dims, axes and attrs for any topic,
including ones no plot can draw. ezmsg unpickles in the subscribing process,
so a message class the dashboard cannot import never arrives; the panel
detects that (an active publisher with no messages) rather than looking idle.

ezmsg-tools supplies the AxisArray-to-plottable reduction and is optional:
without it the inspector still works and only the plotting views are lost.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"View Data" on a publisher row opens a dock showing what that publisher is
sending. The view follows the data -- sweep for a time axis, spectrum for a
frequency axis, channel map for electrode positions -- and a stream that
supports more than one offers a dropdown, since every view is a different
reading of the same frames.

Data frames never enter React state. They arrive tens of times a second and
go from the socket straight to the renderer through a ref; routing them
through `useState` would re-render the topology page at frame rate.

The renderer is purpose-built rather than a charting library, and two of its
choices are not the obvious ones. The ring lives in an RG32F *texture*, not a
vertex buffer: the obvious strided-attribute layout works to 63 channels and
then silently draws nothing, because `vertexAttribPointer` rejects a stride
above 252 bytes. And every channel is one instanced draw, so channel count
costs nothing in draw calls.

Channels beyond what fits are scrolled rather than refused, capped at 512
drawn lanes. Streams that fold several dimensions into channels -- 256
electrodes by 2 features -- get composite names built by walking the same
axis order the reshape uses, so a name cannot drift from its trace, and a
dropdown pins one folded axis to look at a single feature across channels.

Autoscale measures only the lanes on screen, over the whole window. Both
halves matter: scanning every channel lets a loud feature flatten a quiet one
against the rail, and measuring one frame rather than the window settles at a
fraction of the excursion and clips. Narrowing the selection rescales at
once, since the tracker otherwise only contracts gradually.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Shader compilation, the vertex layout and "is anything actually drawn" cannot
fail in a jsdom test, and they are the parts most likely to be wrong -- the
stride bug produced successful uploads, successful draw calls and a blank
canvas. So this drives the real dashboard in a real browser against a real
graph.

The demo graph publishes four streams that between them reach every path: a
labelled timeseries with electrode positions (sweep, and a channel map of the
same data), a spectrum, a channel map, and a dataclass defined in the script's
`__main__` that the dashboard deliberately cannot decode -- which checks that
the panel explains itself instead of looking idle.

Rendering is judged from a screenshot rather than `readPixels`. The canvas is
created with `preserveDrawingBuffer: false`, so reading it back after the
frame composites returns nothing, and forcing that flag on from the harness
perturbs compositing enough to blank the canvas outright -- a false failure
indistinguishable from the real one. The screenshot is what the user sees and
needs no cooperation from the app.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Covers the viewer's controls and the two limits worth knowing about before
pointing it at a real graph: that it can only read topics whose message class
the dashboard's own environment can import, and that a stream wider than the
browser's texture limit falls back to the inspector rather than drawing a
subset under the whole stream's labels.

The code tour records the reasoning behind the parts that are not obvious --
the texture-backed ring, the time base, visible-lane autoscale -- since each
was arrived at by fixing something that failed silently.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rebuilt from frontend/ so the wheel serves the data viewer. Verified with
`python -m ezmsg.dashboard.build_frontend --check`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cboulay
cboulay merged commit c23d5d8 into dev Sep 1, 2026
11 checks passed
@cboulay
cboulay deleted the feat/live-data-viewer branch September 1, 2026 17:03
@cboulay
cboulay restored the feat/live-data-viewer branch September 3, 2026 16:30
@cboulay
cboulay deleted the feat/live-data-viewer branch September 16, 2026 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant