Repository navigation
Feat/aufgaben controller - #329
Open
beniroquai wants to merge 69 commits into
Open
beniroquai wants to merge 69 commits into
beniroquai wants to merge 69 commits into
Conversation
Refactors `ESP32StageManager` to use dynamic per-axis dictionaries and explicit firmware `axisIds`, so stages are no longer hardcoded to A/X/Y/Z and can support remapped/custom axes (including CAN satellite axes). Movement, homing, limits, transport/sample positions, joystick settings, TMC setup, callbacks, and device reads now consistently route through stepper IDs. Updates `UC2ConfigController` to read/apply/save motor settings from the new axis-agnostic structure, and updates autofocus safety-limit initialization to prefer dict-based Z limits. Also fixes `PositionerController.movePositioner` stop detection for continuous motion (`is_stop=abs(speed)==0`), refreshes related stage-map tests, adds new unit coverage for axis mapping behavior, and updates firewall port notes in `main.py`.
Drop the unused Blockly frontend entry and its npm dependencies, and clean up docs to reflect Blockly/Hypha removal and current Arkitekt/newswitch direction. Also add the generated Aufgaben job schema and refresh setup/config schema docs with newer UC2Config, FocusLock, and Arkitekt fields.
Add a VirtualESP32Manager so the virtual microscope can simulate UC2 board services like temperature, digital I/O, fan state, bus power, and basic sensor reads. This lets board-dependent controllers and metadata paths work without real hardware. Also fix several acquisition edge cases: clamp negative illumination values in OME filenames, write TIFF axes/plane metadata that matches 2-D frames, make MDA/workflow waits stoppable and pause-aware, skip image capture for autofocus/custom useq actions, and align ESP32/virtual stage trigger, homing, and stage-scan callback behavior with the underlying uc2rest APIs. The unused HyphaManager import was removed as well.
Introduce a new `imcontrol.model.aufgaben` package for composable acquisition jobs, including the core job document schema, control node specs, task registry/decorator, hardware protocol, and a v1 primitive task catalogue. Add a safe AST-based expression engine, preflight validation with safety/ask-first reporting, and a useq bridge to import/export compatible `MDASequence` workflows. Also include bundled example jobs and public package exports/schema helpers to make the new workflow model discoverable and reusable.
Tighten experiment run control across the backend and UI: the frontend pause call now targets `ExperimentController`, waits become interruptible for pause/stop, force-stop avoids leaving old workflow threads behind, and native MDA runs can be stopped explicitly. This also fixes several acquisition output bugs by correcting OME writer indexing and timepoint sizing, preserving region-specific outputs in performance scans, passing writer options by keyword so Z/channel planes are stored correctly, and forwarding autofocus settings into focus-map and native MDA paths. Smaller cleanup includes a headless-safe temperature endpoint, configurable trigger source handling, removal of unused Hypha/Blockly workflow wiring, and new unit coverage for Aufgaben, workflow pause/resume, virtual ESP32 behavior, and single-TIFF OME writing.
Introduces the new Aufgaben workflow stack end-to-end: a dedicated API controller, execution engine, ImSwitch host adapter, simulated microscope, persistent job/run storage, and broad API/unit coverage. It also unifies OME-Zarr writing across Aufgaben, ExperimentController, and recording paths, improving NGFF/OME compliance and fixing related scan/export issues (including busy-state coordination across experiment, Arkitekt, and firmware flows). Alongside this, it removes large amounts of deprecated Qt-era and unused functionality (Timelapse/HistoScan/OpenLayers and many old controllers/managers), updates setup/schema defaults, and refreshes frontend/docs to match the new headless-focused surface.
Bio-Formats rejected every OME-TIFF ImSwitch wrote: the XML escaped the micro sign as µ, which Bio-Formats turns into an unknown unit, so Fiji fell back to a plain TIFF without channels, sizes or pyramid. The XML is now stored as UTF-8. OME-Zarr uses Zstd instead of Blosc: as fast at level 1, and Fiji's N5 reader cannot load native Blosc on macOS arm64. save_image writes one numpy image (named axes, RGB) as OME-TIFF or OME-Zarr through the same code as NgffImage, so it can replace the direct tifffile and zarr writes. Unknown pixel sizes and positions are left out instead of written as 1.0 and 0. A new test opens the files with Fiji's own Bio-Formats and N5 jars when Fiji is installed.
ExperimentController had three TIFF writers next to its OME-Zarr canvas: an in-RAM hyperstack (write_tiff, capped at 512 MiB), a multi-series tile collection called stitched.ome.tif (OmeTiffStitcher) and an appended single TIFF (SingleTiffWriter). All three are now one OME-TIFF export of the canvas at finalize, stitched.ome.tif, through the shared writer: a real mosaic with pyramid and OME-XML, streamed from zarr. When only the TIFF is wanted the canvas lives in a hidden folder that is removed afterwards. Individual tile TIFFs go through save_image, with the same file names. Lightsheet's TIFF output uses the same paths. The multi-site API test now checks that every tile of every plane of the stitched canvas holds data.
Snapshots were written two ways: the background path built OME-XML with the old single-plane builder, the synchronous path stored the metadata as a JSON description (not OME at all). Both now call save_snap_tiff: pixel size, top-left stage position and exposure go into the OME-XML, and every other acquisition attribute into an OME MapAnnotation (attrs["acquisition"] in OME-Zarr), so nothing is lost and Fiji shows it. The never-used append_tiff task and the old builder module are removed.
The data store wrote OME-TIFF with its own OMETiffWriter next to the NGFF images. It now exports each detector's NGFF image to <session>_<detector>.ome.tiff when the recording closes, so both formats carry the same metadata and pyramid. A TIFF-only recording keeps its zarr in a hidden folder that is removed after the export.
A second snap implementation next to RecordingService.snap with its own TIFF writing; nothing imported it.
The sweep steps the virtual stage with move(); since blocking virtual moves take the travel time, every step also slept distance/speed and the sweep drifted against its trigger clock. The strobe tests failed about every second run. Sweep steps are no longer blocking moves.
Raw tiles and the stitched export were written with tifffile directly (the raw tiles with tifffile's JSON description, not OME). Both are now OME-TIFFs from the shared writer with pixel size and top-left stage position; the stage map knows no Z, which is left out instead of 0.
The TIFF format appended pages with tifffile and a JSON description per frame; reconstructions were plain TIFFs. A STORM recording is now one OME-NGFF time series for both formats; with TIFF chosen it lives in a hidden folder and is exported as OME-TIFF when the recording ends. Reconstructions go through save_image.
DPC, SIM, FlowStop, Lepmon, MMCore, read-noise, stress-test, UC2Config snapshots and useq MDA frames were written with tifffile directly, some with a JSON description. They are OME-TIFFs from the shared writer now; metadata dicts travel as OME annotations. UC2Config saved mImage[0] for any 3-D frame, which for an RGB camera is one image row. RGB frames are now saved whole; only frame stacks give their first frame.
The workflow step save_frame_tiff wrote to timelapse.tif in the server's working directory with a TiffWriter method that tifffile no longer has, and close_tiff_writer read an attribute that was never set. The step now keeps the frames and close_tiff_writer writes them as one OME-TIFF time series in <data>/Workflow/. ShitScope tiles and mosaics are OME-TIFFs with their pixel size.
storage.py is now the only image writer: save_image, the canvas semantics of stitched.ome.tif, Zstd for Fiji, UTF-8 OME-XML for Bio-Formats, the deleted writers, the plugin's Ashlar fix and its OME viewer. AGENTS.md tells backend work to save images through it.
One wrapper per AufgabenController endpoint, and pure helpers for the job tree (insert, move, indent, outdent, place, rename with its loop references), node ids and defaults, and the form fields of a catalogue entry. Tested against a catalogue the backend produced.
The slice holds the catalogue, the job being edited (persisted across reloads), its validation and the run: status, open question, task progress and the last 300 events. Renaming a resource rewrites its @references.
The palette lists the catalogue's steps by category; a click adds a
step at the selection, a drag drops it anywhere. The outline shows the
job as indented branches with each step's validation issues, ask-first,
motion and light markers and the step that runs now; steps move by drag
and drop or the toolbar (up, down, in, out, duplicate, delete).
The inspector edits the selected step from its schema (numbers also
take ${expressions}; resource fields pick or create an @resource), the
run conditions and error policy, the job's resources (a positions table
that takes the current stage position, a channel table over the setup's
illumination sources, JSON for plans) and the job settings: storage
formats, layout, stitching, variables and run parameters.
Start validates first, refuses a job with errors, and opens a dialog that says what the job will do (stage, focus, light), lists the ask-first steps behind a confirmation and asks for the run parameters. The run tab follows the engine (state, step, loop path, frames, elapsed, task progress, event log) with pause, resume, finish, abort and abort with stage stop and light off; the job's questions open a dialog. The library opens saved jobs and the bundled examples, saves, deletes, reads and writes JSON files and imports and exports useq sequences, and lists recent runs. Registered as an app that needs the AufgabenController; without it the page says so.
The repository map names storage.py as ImSwitch's one image writer, and the backend role saves images through it instead of tifffile or zarr.
Untyped parameters arrived as query strings: setTrigger(trigger1=false) received the string "false", which is truthy, and the output ids reached the board as strings. Ids and values are now int, delays float, flags bool, so FastAPI converts and checks them.
Two setup blocks, pumps and valves, each a MultiManager of named devices like lasers and LEDs. Pumps take volumes in µL and rates in µL/min (negative runs backwards) and are bounded by maxRateUlMin. StepperPumpManager runs a UC2 stepper axis with stepsPerUl; pump_volume blocks on the board until the volume is moved and stop() from another thread ends it. VirtualPumpManager takes the time the volume needs (with a speedup) and counts what it moved. Valves switch between named positions: DigitalOutValveManager sets one or several board outputs, ServoValveManager turns a servo, SerialValveManager sends a command to its own controller, and VirtualValveManager records the switches.
The host offers the setup's pumps and valves as devices and the pump and valve capabilities, and forwards pump_volume, pump_run, pump_stop and set_valve to their managers. A forced abort now also stops every pump. The virtual microscope gets a peristaltic pump and an inlet valve, so the stop-flow example validates and runs without hardware.
Pumps on openUC2 run on a PWM channel or a stepper axis that ImSwitch already manages, so the fluidics managers now use those instead of the board client: LaserPumpManager sets a laser value proportional to the rate (one direction, timed), PositionerPumpManager moves a positioner axis (positioner units and steps/s from stepsPerUl), LaserValveManager sets one laser value per position. The stepper, servo and serial variants that talked to the board directly are removed; the digital output valve stays for GPIO. Laser channels that drive a pump or a valve are no longer offered to jobs as light sources.
It registered seven remote functions on go.arkitekt.io and a rekuest startup hook with the old signature (instance_id), which current rekuest_next rejects at import time; the controller was then skipped and every STORM endpoint returned 404 wherever arkitekt_next is installed. The guarded imports, state, startup hook, the arkitekt_* functions, the frame generator they used, the process_arkitekt option and the arkitekt_available status field are gone; the local STORM pipeline is unchanged. startFastSTORMAcquisition now sets the acquisition active before it starts the saving worker, which runs while that flag is set and could exit at once.
My consolidation onto the shared writer (e00543c) left the old "unsupported format" branch's stop-and-break under the save, so direct saving stopped after the first frame. getChunk() returns (frames, ids) on Hik and the virtual camera; the store set-up passed that tuple on and failed, and a plain list of frames failed in the normaliser. The OME-Zarr store is now created from the first (cropped) frame the worker receives instead of a frame read before acquisition started.
STORMController.js was imported nowhere (the drawer offers STORM Local and STORM Arkitekt), and five API wrappers existed only for it, one of them for an endpoint that does not exist (setSTORMParameters).
Its endpoints (/ArkitektController/connect, getWorkflows, startWorkflow, stopWorkflow) and signals never existed in the backend, and the STORM controller no longer offers Arkitekt processing. STORM Local stays.
The engine keeps the last image of every capture and derived-image step as a Frame with its pixel size and the stage position of its top-left pixel (read once per frame, only when the job has steps that need it). Tasks get it through ctx.frame(source), and ctx.variables and ctx.results; steps of the Analysis and Scripts categories use them and are engine-only. The validator checks that a source names a step. The simulated microscope can carry a sample of disks at stage positions, which its camera and prescan render, so analysis can be tested end to end.
measure_intensity, focus_metric, detect_background, count_objects,
points_from_mask and regions_from_overview read an image the run has
seen and return plain data for expressions (result.<id>.count,
result.<id>.empty). Detected objects and overview regions come back in
stage µm, ready for a position or region loop
("${result.find.positions}"). Thresholds are Otsu unless set.
python_script and external_process run unsandboxed code: they need the
setup option aufgaben.allowScripts and, like homing, a confirmed run.
Tested on synthetic frames and on the simulated sample: objects found
at their stage positions and visited, empty fields skipped, the loop
stopped after enough cells, a prescan turned into a region loop.
skip_empty_wells: a quick look per well; wells without contrast are skipped, the others get a Z-stack. stop_after_n_cells: tiles a region, counts cells per tile and finishes at the target. overview_regions_zstack: prescan, regions from the overview, objective change, tiled Z-stacks per region. Each runs in a test on a simulated sample.
Parameters marked x-widget: code in the catalogue get a monospace, multiline field instead of a single line.
The virtual illuminator kept one intensity for every channel, so switching the laser off after the LED on darkened the sample; jobs on the virtual microscope captured black frames. It now keeps one value per channel and the camera sees their sum. getLast returned the oldest of up to five queued frames, which could show the light or stage position from before the caller's change; it returns the newest.
How stagescan, strobesweep, normal mode, performance mode, the stage map prescan and Aufgaben switch light and trigger cameras today, why each is open-loop, what the cameras could offer (trigger in, strobe out), three ways to close the loop, and eight trigger bugs found on the way.
The local STORM pipeline and its weak points (settings ignored, no super-resolved image, live view never reaches the UI, fitting in the camera thread), what webSMLM computes and how its live WebSocket protocol works, and a proposal: a WebSocket feed for webSMLM plus a server-side spot counter that closes an activation-laser loop.
The smart-microscopy demo over HTTP: a brightfield frame of the virtual microscope, its intensity, the objects in it, and a position loop over three of them that captures each.
Roadmap: Phase 5 done (frame access, analysis and script steps, the templates, the virtual demo), pumps and valves on laser and positioner channels, the app as an essential one with icons, and the fixes of this round with pointers to the two analyses. agents.md and the overview describe the analysis steps and the scripting option.
…ImSwitch into feat/AufgabenController
Reworks Toupcam live-cycle handling so TEC models no longer call `Stop()` during normal start/stop-live flows. The camera now parks in software-trigger mode, resumes with the requested trigger mode, and drops stale frames captured before resume. Settings that require a stopped stream (ROI, gain mode, low-noise, pixel format/binning) are funneled through a controlled reconfigure path and skip restarts when unchanged. Also adds TEC voltage reporting and logging (`tec_voltage_v`, parked state), log-header migration for updated CSV columns, and stronger stall warnings. Documentation is updated with the TEC behavior rationale, and new unit tests cover park/resume behavior, frame gating, stop/restart conditions, and non-TEC fallback behavior. FRAME_6 setup JSONs were updated with an `ObservationCamera` and revised stage/home configuration fields.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.