Skip to content

Add HIL test runner support - #328

Open
LeonardPertsch wants to merge 130 commits into
masterfrom
hil-runner
Open

LeonardPertsch wants to merge 130 commits into
masterfrom
hil-runner

Conversation

@LeonardPertsch

Copy link
Copy Markdown
Collaborator

This PR adds the HIL test runner setup for ImSwitch and integrates the existing e2e tests with the hardware test workflow.

LeonardPertsch and others added 30 commits September 3, 2026 10:03
Three tests aborted rather than skipped when ImSwitch was unreachable,
which the e2e README explicitly promises they would not:

- test_motor_motion.get_axes() raised while pytest was evaluating the
  parametrize argument, so a missing rig killed collection for the whole
  suite, not just this file
- test_board_link and test_ledmatrix_smoke let the ConnectionError through
  and reported a failure, which says nothing about the hardware

All three now degrade to a skip with the reason in the message.

test_motor_endstops drops the getDigitalInReliable fallback: the firmware
echoes the qid for /digitalin_get again, so getDigitalIn matches its reply
and reads back directly. The endpoint was never implemented; calling it
only cost a 404 per read.

.gitignore covers .venv as well as venv, so the interpreter the Python
extension expects in the project root stays untracked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seven files in the e2e suite ended without a trailing newline. Editors
configured to insert one rewrite the file on save, which turns an ordinary
save into a diff and, on a buffer that is also behind the file on disk,
into a conflict prompt.

Whitespace only, one line per file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pumps on openUC2 run on a PWM channel or a stepper axis that ImSwitch
already manages, so the fluidics managers now use those instead of the
board client: LaserPumpManager sets a laser value proportional to the
rate (one direction, timed), PositionerPumpManager moves a positioner
axis (positioner units and steps/s from stepsPerUl), LaserValveManager
sets one laser value per position. The stepper, servo and serial
variants that talked to the board directly are removed; the digital
output valve stays for GPIO.

Laser channels that drive a pump or a valve are no longer offered to
jobs as light sources.
It registered seven remote functions on go.arkitekt.io and a rekuest
startup hook with the old signature (instance_id), which current
rekuest_next rejects at import time; the controller was then skipped
and every STORM endpoint returned 404 wherever arkitekt_next is
installed. The guarded imports, state, startup hook, the arkitekt_*
functions, the frame generator they used, the process_arkitekt option
and the arkitekt_available status field are gone; the local STORM
pipeline is unchanged.

startFastSTORMAcquisition now sets the acquisition active before it
starts the saving worker, which runs while that flag is set and could
exit at once.
My consolidation onto the shared writer (e00543c) left the old
"unsupported format" branch's stop-and-break under the save, so direct
saving stopped after the first frame. getChunk() returns (frames, ids)
on Hik and the virtual camera; the store set-up passed that tuple on
and failed, and a plain list of frames failed in the normaliser. The
OME-Zarr store is now created from the first (cropped) frame the worker
receives instead of a frame read before acquisition started.
STORMController.js was imported nowhere (the drawer offers STORM Local
and STORM Arkitekt), and five API wrappers existed only for it, one of
them for an endpoint that does not exist (setSTORMParameters).
Its endpoints (/ArkitektController/connect, getWorkflows, startWorkflow,
stopWorkflow) and signals never existed in the backend, and the STORM
controller no longer offers Arkitekt processing. STORM Local stays.
The engine keeps the last image of every capture and derived-image
step as a Frame with its pixel size and the stage position of its
top-left pixel (read once per frame, only when the job has steps that
need it). Tasks get it through ctx.frame(source), and ctx.variables and
ctx.results; steps of the Analysis and Scripts categories use them and
are engine-only. The validator checks that a source names a step.

The simulated microscope can carry a sample of disks at stage
positions, which its camera and prescan render, so analysis can be
tested end to end.
measure_intensity, focus_metric, detect_background, count_objects,
points_from_mask and regions_from_overview read an image the run has
seen and return plain data for expressions (result.<id>.count,
result.<id>.empty). Detected objects and overview regions come back in
stage µm, ready for a position or region loop
("${result.find.positions}"). Thresholds are Otsu unless set.

python_script and external_process run unsandboxed code: they need the
setup option aufgaben.allowScripts and, like homing, a confirmed run.

Tested on synthetic frames and on the simulated sample: objects found
at their stage positions and visited, empty fields skipped, the loop
stopped after enough cells, a prescan turned into a region loop.
skip_empty_wells: a quick look per well; wells without contrast are
skipped, the others get a Z-stack. stop_after_n_cells: tiles a region,
counts cells per tile and finishes at the target. overview_regions_zstack:
prescan, regions from the overview, objective change, tiled Z-stacks
per region. Each runs in a test on a simulated sample.
Parameters marked x-widget: code in the catalogue get a monospace,
multiline field instead of a single line.
The virtual illuminator kept one intensity for every channel, so
switching the laser off after the LED on darkened the sample; jobs on
the virtual microscope captured black frames. It now keeps one value
per channel and the camera sees their sum. getLast returned the oldest
of up to five queued frames, which could show the light or stage
position from before the caller's change; it returns the newest.
How stagescan, strobesweep, normal mode, performance mode, the stage
map prescan and Aufgaben switch light and trigger cameras today, why
each is open-loop, what the cameras could offer (trigger in, strobe
out), three ways to close the loop, and eight trigger bugs found on
the way.
The local STORM pipeline and its weak points (settings ignored, no
super-resolved image, live view never reaches the UI, fitting in the
camera thread), what webSMLM computes and how its live WebSocket
protocol works, and a proposal: a WebSocket feed for webSMLM plus a
server-side spot counter that closes an activation-laser loop.
The smart-microscopy demo over HTTP: a brightfield frame of the
virtual microscope, its intensity, the objects in it, and a position
loop over three of them that captures each.
Roadmap: Phase 5 done (frame access, analysis and script steps, the
templates, the virtual demo), pumps and valves on laser and positioner
channels, the app as an essential one with icons, and the fixes of
this round with pointers to the two analyses. agents.md and the
overview describe the analysis steps and the scripting option.
Reworks Toupcam live-cycle handling so TEC models no longer call `Stop()` during normal start/stop-live flows. The camera now parks in software-trigger mode, resumes with the requested trigger mode, and drops stale frames captured before resume. Settings that require a stopped stream (ROI, gain mode, low-noise, pixel format/binning) are funneled through a controlled reconfigure path and skip restarts when unchanged.

Also adds TEC voltage reporting and logging (`tec_voltage_v`, parked state), log-header migration for updated CSV columns, and stronger stall warnings. Documentation is updated with the TEC behavior rationale, and new unit tests cover park/resume behavior, frame gating, stop/restart conditions, and non-TEC fallback behavior. FRAME_6 setup JSONs were updated with an `ObservationCamera` and revised stage/home configuration fields.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants