Repository navigation
feat: live data viewer for graph publishers - #16
Merged
Merged
Conversation
The dashboard could show that a publisher was sending, but not what. This adds a `/ws/stream` endpoint that subscribes to any topic and streams its samples to the browser. The backend already holds a GraphContext, so it subscribes directly rather than injecting a unit or brokering shared memory: no edge is added to the topology and the tap does not appear in profiling. Two properties of ezmsg's Subscriber carry the design. `leaky=True` releases publisher backpressure for every notification it drops, so a wedged browser cannot stall the graph it is watching; `recv_zero_copy()` lends the message without pickling it, so the reduction happens in place and nothing is retained past the lease. Leakiness costs contiguity, so each tap owns a ring written at message rate and read at frame rate, and reports gaps rather than drawing across them. Sample data is reduced before it is sent, to the client's pixel budget over a requested time window. The window is what makes this affordable: without it a frame's samples are spread across the whole plot, so a 30 kHz stream shows 33 ms and repaints entirely every frame. Fixing the window in time makes a column a fixed number of samples, and the wire cost follow columns per second rather than the publisher's rate -- 2048 channels measures at 5.8 MB/s. Payloads travel as binary frames, `(n_out, n_channels, 2)` min/max pairs, which is also the memory layout of the texture the browser uploads them into. A message inspector reports type, dims, axes and attrs for any topic, including ones no plot can draw. ezmsg unpickles in the subscribing process, so a message class the dashboard cannot import never arrives; the panel detects that (an active publisher with no messages) rather than looking idle. ezmsg-tools supplies the AxisArray-to-plottable reduction and is optional: without it the inspector still works and only the plotting views are lost. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"View Data" on a publisher row opens a dock showing what that publisher is sending. The view follows the data -- sweep for a time axis, spectrum for a frequency axis, channel map for electrode positions -- and a stream that supports more than one offers a dropdown, since every view is a different reading of the same frames. Data frames never enter React state. They arrive tens of times a second and go from the socket straight to the renderer through a ref; routing them through `useState` would re-render the topology page at frame rate. The renderer is purpose-built rather than a charting library, and two of its choices are not the obvious ones. The ring lives in an RG32F *texture*, not a vertex buffer: the obvious strided-attribute layout works to 63 channels and then silently draws nothing, because `vertexAttribPointer` rejects a stride above 252 bytes. And every channel is one instanced draw, so channel count costs nothing in draw calls. Channels beyond what fits are scrolled rather than refused, capped at 512 drawn lanes. Streams that fold several dimensions into channels -- 256 electrodes by 2 features -- get composite names built by walking the same axis order the reshape uses, so a name cannot drift from its trace, and a dropdown pins one folded axis to look at a single feature across channels. Autoscale measures only the lanes on screen, over the whole window. Both halves matter: scanning every channel lets a loud feature flatten a quiet one against the rail, and measuring one frame rather than the window settles at a fraction of the excursion and clips. Narrowing the selection rescales at once, since the tracker otherwise only contracts gradually. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Shader compilation, the vertex layout and "is anything actually drawn" cannot fail in a jsdom test, and they are the parts most likely to be wrong -- the stride bug produced successful uploads, successful draw calls and a blank canvas. So this drives the real dashboard in a real browser against a real graph. The demo graph publishes four streams that between them reach every path: a labelled timeseries with electrode positions (sweep, and a channel map of the same data), a spectrum, a channel map, and a dataclass defined in the script's `__main__` that the dashboard deliberately cannot decode -- which checks that the panel explains itself instead of looking idle. Rendering is judged from a screenshot rather than `readPixels`. The canvas is created with `preserveDrawingBuffer: false`, so reading it back after the frame composites returns nothing, and forcing that flag on from the harness perturbs compositing enough to blank the canvas outright -- a false failure indistinguishable from the real one. The screenshot is what the user sees and needs no cooperation from the app. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Covers the viewer's controls and the two limits worth knowing about before pointing it at a real graph: that it can only read topics whose message class the dashboard's own environment can import, and that a stream wider than the browser's texture limit falls back to the inspector rather than drawing a subset under the whole stream's labels. The code tour records the reasoning behind the parts that are not obvious -- the texture-backed ring, the time base, visible-lane autoscale -- since each was arrived at by fixing something that failed silently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rebuilt from frontend/ so the wheel serves the data viewer. Verified with `python -m ezmsg.dashboard.build_frontend --check`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a live view of the data flowing through any publisher, alongside the
existing topology, settings and profiling views.
Click View Data on an expanded publisher row and a dock opens showing what
that publisher is sending.
How it works
The backend already holds a
GraphContext, so it subscribes to the topicdirectly — no unit injected, no shared-memory hop, no helper process. The tap
adds no edge to the topology and does not appear in profiling (both verified
against a live graph).
Two properties of ezmsg's
Subscribercarry the design:leaky=Truereleases publisher backpressure for every notification itdrops, so a wedged browser tab cannot stall the graph it is watching. This is
the first thing to check in any "tap a live pipeline" scheme and ezmsg already
provides it.
recv_zero_copy()lends the message without pickling it. Everythingdownstream either copies into the ring or copies explicitly.
Leakiness costs contiguity, so each tap owns a ring written at message rate and
read at frame rate, and reports gaps instead of drawing across them.
Views
The view follows the data, and a stream supporting more than one offers a
dropdown — switching is a different reading of the same frames, so it needs no
re-subscription.
(time, ch)chaxis carries real x/y(freq, ch)(ch)+ positionsThe inspector reports type, dims, axes and attrs for any topic, including
ones no plot can draw.
Cost to the graph
Sample data is reduced before it is sent — to the plot's pixel width, over a
selectable time window. Bandwidth follows columns per second rather than the
publisher's sample rate:
Optional dependency
Plotting needs the
vizextra (pip install "ezmsg-dashboard[viz]"), whichadds
ezmsg-toolsfor its AxisArray-to-plottable reduction — pure numpy, its Qtand phosphor dependencies sit behind its own extras. Without it the message
inspector still works on every topic and only the plotting views are lost.
Two limits worth knowing
ezmsg unpickles in the subscribing process, so a type defined in a script's
__main__, or from a package the dashboard's environment lacks, neverarrives. The panel detects this (an active publisher with no messages) and
says so rather than looking idle. Fixing it properly needs something at the
ezmsg layer.
current hardware, 2048 at the spec minimum) cannot be held in a plot buffer.
The panel reports that and falls back to the inspector rather than drawing a
subset under the whole stream's labels. Below that, wide streams are scrolled,
512 lanes at a time.
Notes for review
Some non-obvious choices, each arrived at by fixing something that failed
silently — the code comments carry the detail:
strided-attribute layout works to 63 channels and then draws nothing:
vertexAttribPointerrejects a stride above 252 bytes, while uploads and drawcalls still succeed.
Without it a 30 kHz stream showed 33 ms and repainted entirely every frame.
every channel lets a loud feature flatten a quiet one against the rail;
measuring one frame rather than the window clips the trace.
bfloat16is widened by reinterpreting its bits in numpy. Asking MLX tocast returns NaN and ~2.5e38 for an array that arrived over ezmsg's transport,
because the cast runs on a device that cannot address the buffer — silently
wrong rather than failing.
Testing
156 backend tests, 82 frontend unit tests, 32 Playwright e2e, ruff clean.
frontend/scripts/check-stream-panel.mjsadditionally drives the dashboard in areal browser against
examples/stream_demo_graph.pyand asserts each viewrenders — shader compilation and "is anything drawn" cannot fail in jsdom. See
the Development Guide for how to run it.
Verified by hand against a live 256-channel 30 kHz pipeline, including
MLX-backed (
float32,float16,bfloat16) payloads.🤖 Generated with Claude Code