feat: add Interval Test and Focus Test generators to the Tools menu

[switch87]

Summary

Add Interval Test and Focus Test generators to the Tools menu. Both
create an ordinary layer with workpieces and steps, as discussed in #519.

Fixes #519. Closes #550.

What's in this PR

• Tools menu (laser essentials): Create Interval Test and Create Focus
  Test, each opening a small parameters dialog built from VarSets.
• Interval Test: a row of filled squares, each with its own Engrave step
  linked through generated_workpiece_uid, with line intervals spread evenly
  from the smallest to the largest value, at one power and speed. Labels show
  the interval in mm and LPI and are engraved by one low-power Contour step.
• Focus Test: one Contour line per focus offset, labelled with the offset.
  Methods: Z axis steps (relative G91/G0 Z/G90 Command steps, back to the
  start height at the end; only with a Z axis), manual (M0 Command step
  before each line) and ramp (one long line with 10 mm ticks for a tilted
  strip, no machine support needed). Z and manual need a G-code driver and the
  Command step. The dialog starts on the first available method, so machines
  with a Z axis get Z axis steps (Focus Test: default the Method to "Z axis steps" when the machine has a Z axis #550).
• Docs page under Features → Operations.

Tests

• tests/doceditor/test_calibration_tests.py: generated layers, steps,
  available methods and GRBL G-code for both tests.
• rayforge-addon-laser/tests/test_calibration_test_params.py: parameter
  validation and layout values.
• rayforge-addon-laser/tests/ui_gtk/test_calibration_test_dialogs.py: the
  parameter dialogs.

[atkaper]

@switch87 Nice one! I have not run it on my laser yet (might try later this week), but I did run your PR without machine to try your new function. Would it be an idea to make the Method "Z- axis steps" the default if the machine definition indicates it supports a z-axis? Of course just a nice to have, but I think it would make sense.

[switch87]

@atkaper Good idea, done in 73ca5b7. The method list was already ordered by preference (Z axis steps, manual, ramp) and only contains what the machine supports, but the dialog started on the manual method regardless. It now starts on the first available method, so a machine with a Z axis and a G-code driver opens on Z axis steps. A UI test covers it, and the docs say which method is the default. Also rebased onto current main.

[atkaper]

Super! I will try this one later this week!

[switch87]

Dropped the laser_essentials .po/.pot changes from this PR, following your rule of one catalog update before each release; the new strings are still wrapped in _(). Also rebased onto current main and made the calibration tests wait for the editor to settle, since one of them left tasks behind at teardown.

[switch87]

I went over this again against the rest of the code base and pushed two small follow-up commits on top, no force-push:

• 0869c0c: the Focus Test no longer imports the private _driver_uses_gcode from the intent builder; it resolves the driver with get_driver_cls like Machine does. Added a test for a non-G-code driver.
• 2dfbf11: the two dialogs are now declared as VarSets and rendered with VarSetWidget, like the Material Test Grid settings, instead of hand-built rows. Power is the usual percent slider, the speed limit comes from the machine, and the dialog is as wide as the step settings dialog. The label strings reuse the existing Material Test ones.

I also corrected the PR description, which described the ramp and the Interval Test wrongly.

[knipknap]

Thank you for this PR! In testing I noticed some issues:

1. The dialog pops up as a modal in the middle of the screen, hiding the produced test while editing.
2. I think UX wise the test feature should be aligned with the existing Material Test Grid feature. Currently, the two behave differently: Clicking "Material Test Grid" creates a step in the workflow and a generated workpiece, without opening the dialog, whereas your new tools DO open the dialog immediately. I agree that opening the dialog is the better behavior here, but in any case, all three tests should behave the same, UX wise. I do not mind changing the approach of the Material Test Grid feature - I actually like your approach of generating a completely new layer instead, it seems more elegant than what the material test grid currently does.
3. Code architecture wise, I think the tests/generators should be aligned as well. The new test generators appear to share little code with the existing test generator, which indicates that either the pre-existing architecture is insufficient, or the new implementation just misses opportunities for generalization.

Proposed next phases

1. Your code exposed a pre-existing code smell that I introduced and builds on it: The pipeline currently filters on a generated_workpiece_uid - this is actually a layer violation, as it forces the pipeline to understand the "source" of the workpiece instead of using a generic approach. I would refactor this into a step.selected_workpiece_uids: List[str], removing the leak. The step settings UI could then display the list of workpieces in a new tab. (I would not make this editable in the first step.). Without this change, the UI implies a behavior that does not match reality: It shows raster steps on a layer implying all workpieces are processed (because the current UI assumes a step=full layer processing) when in reality there is a per-workpiece filter - it just isn't visible anywhere. In an ideal world, each step icon would show a tiny number if it filters on workpieces, but I am fine to move ahead without this; this could be a future improvement.
2. Create an iterative phase 2 implementation plan for review that starts with refactoring the pre-existing test generator for re-usability with the new tests. This may also include improving the UI experience with the nice features introduced by your PR (e.g. dedicated layer for the test, per-cell editable result, UI dialog opens immediately).
3. Implementation of the new test generators from this PR on top of the cleaned up architecture. This should also close another small gap I noticed: It could add new targets to the screenshotter so that the docs include the proper screenshots.

Specific design proposal for phase 1 (AI generated but reviewed)

• Generalize Step.generated_workpiece_uid into a workpiece filter that the intent builder no longer treats as a special concept.
• Expose it in the step settings dialog as a READ-ONLY page (the last tab) listing which workpieces the step processes. No interactive editing; only generators (Material Test Grid, and the Interval/Focus Tests from the current PR) set the filter programmatically.
• No warning when a filtered step matches no workpieces (silent skip, same as today).

1. Core model — rayforge/core/step.py

• Replace generated_workpiece_uid: str | None (line 97) with workpiece_uids: list[str], where an empty list means "all workpieces of the layer" (current None semantics).
• Add Step.selected_workpieces(workpieces) -> list[WorkPiece]: returns all workpieces when the filter is empty, otherwise the layer workpieces whose uid is in the filter, in layer order; missing uids are simply skipped (preserves today's silent behavior).
• Add set_workpiece_uids(uids) setter (dedupe, normalize, updated.send(self) only on real change) so undo commands and generators have one entry point.
• Serialization: to_dict writes workpiece_uids; _serialized_keys swaps the old key for the new; from_dict reads workpiece_uids and falls back to a legacy generated_workpiece_uid (→ [uid]) so existing .ryp projects load unchanged. Older app versions round-trip the unknown key via the existing extra mechanism.
• The filter is intentionally NOT added to get_cache_params() or the recipe varset: the pipeline graph shape (which workpiece nodes exist) and the aggregate token's upstream key list already capture it, and it must not become a recipe-extractable process setting.

2. Pipeline — rayforge/pipeline/intent_builder.py

• _workpieces_for_step (line 292) becomes a delegation: return step.selected_workpieces(workpieces). The builder keeps only the generic "a step selects its workpieces" concept; the "generated" notion disappears from the pipeline (update the module docstring wording).

4. Generators

• laser_essentials/commands/calibration_test_cmd.py: _link sets step.workpiece_uids = [wp.uid]; update the module docstring.
• laser_essentials/commands/material_test_cmd.py: line 127 same; sync_preview_from_step (line 169) reads the first uid of step.workpiece_uids instead of generated_workpiece_uid.
• laser_essentials/steps/laser_step.py: get_settings() appears to be obsolete so it could be removed. Otherwise: swaps the key to workpiece_uids.

5. UI — read-only "Workpieces" page

• New rayforge/ui_gtk/doceditor/step_settings/pages/workpieces.py: WorkpiecesPage(StepSettingsPage) with show_identity = False (same pattern as PostProcessingPage).
  • A summary row: "All workpieces on this layer" when the filter is empty, otherwise "N of M workpieces".
  • One non-interactive Adw.ActionRow per workpiece of step.layer.all_workpieces (name; check-style icon when included, dimmed/minus icon when excluded by a filter).
  • Read-only: no switches, no checkbuttons, nothing persisted.
  • Refreshes on step.updated and on layer descendant_added/descendant_removed so the list stays current while the dialog is open.
• dialog.py: after the Post Processing page, add the Workpieces page as the final tab — only when step.needs_workpieces and step.layer exist (hidden for CommandStep and detached steps). Cleanup hookup via the existing _cleanup path.

6. Tests

• tests/core/test_step.py: update the serialization tests to workpiece_uids; add a legacy-file test (from_dict with only generated_workpiece_uid yields the filter; to_dict emits only the new key).
• tests/pipeline/test_intent_builder.py (lines ~122-140): switch to the new field; keep the "generated workpiece missing → step contributes nothing" case.
• tests/doceditor/test_calibration_tests.py (line 123): assert the linkage via workpiece_uids.
• New tests/ui_gtk/ test for the page: filter set → only matching workpieces shown as included; empty filter → summary row says all; command step → page not added by the dialog (logic-level check).

7. Docs & housekeeping

• CHANGELOG.md entry.
• New user-visible strings wrapped in _(); follow docs/agents/translations.md for catalogs.
• pixi run lint first, then targeted tests: pixi run test tests/core/test_step.py tests/pipeline/test_intent_builder.py tests/doceditor/test_calibration_tests.py and pixi run uitest for the new UI test.

Out of scope (deferred by design): interactive filter editing in the UI, multi-select semantics beyond what generators set.

[knipknap]

(I will draft a PR for phase 1, as that was a pre-existing mess so I will clean this up.)

[switch87]

Thanks for testing it this thoroughly and for the detailed plan. I agree on all three points, and the workpiece filter is a much cleaner base than generated_workpiece_uid.

I'll leave phase 1 to you then. Meanwhile I'll write the phase 2 plan, refactoring the Material Test Grid generator so the three tests share one architecture and one UX (dedicated layer, dialog opens immediately and doesn't hide the test, per-cell editable result), and post it here for review before writing any code. I've turned this PR back into a draft so it doesn't sit in your review queue; phase 3 will rebuild these generators on top of the cleaned-up code, including the screenshot targets for the docs.

[switch87]

Phase 2 plan: one generator architecture for all calibration tests

This plan starts after your phase 1 (Step.workpiece_uids / selected_workpieces()) is merged. Every step below links a step to its workpieces through that filter only. The pipeline never learns about "tests" or "generated" content.

Where things stand today

• Material Test Grid. MaterialTestCmd.create_test_grid() adds one MaterialTestStep and one procedural preview workpiece to the active layer. The cells, labels, risk ordering and Speed-vs-Offset power scaling are all computed inside the raygeo material_test_grid assembler, so one step holds every cell. The test settings live on the step and are edited in the step settings dialog. sync_preview_from_step resizes the preview when they change. The asset is added with doc.add_asset() outside the undo transaction, so it stays behind after an undo. MaterialTestCmd has no tests.
• PR feat: add Interval Test and Focus Test generators to the Tools menu #539. It creates its own layer, with one standard Engrave/Contour step and one workpiece per cell, a shared labels workpiece and step, and Command steps between Focus Test lines. Its parameters exist only in a create dialog, so a test can't be changed after it is created. Workpieces go through an SVG export and import with a regex fill hack. Centering is done by hand. The layer gets the default color and no stock material.

What all three tests share: a parameter VarSet, a row or grid layout with labels, an ordered list of "cell = workpiece + step" entries (sometimes with commands in between), placement on the work area, one undo entry, and regeneration when a parameter changes.

Target shape (all in laser_essentials, nothing new in core)

• calibration/generator.py: CalibrationGenerator base class with id, title, varset(context), defaults() and layout(params, context) -> CalibrationLayout. It is pure: no document, no GTK.
• calibration/layout.py: CalibrationLayout is an ordered list of CellEntry(name, geometry or workpiece factory, step factory) and CommandEntry(name, lines). Order in the list is execution order. Shared helpers: row_positions, grid_positions and label_geometry. The text helper from feat: add Interval Test and Focus Test generators to the Tools menu #539 moves here. All geometry is in test-local mm, Y up, origin at the bottom-left of the test.
• calibration/builder.py: CalibrationBuilder turns a layout into document content. It creates the layer (unique name, unused color, default stock material, the same as AddLayerAndSetActiveCommand._create_default_layer), the assets, workpieces and steps (step.set_workpiece_uids([wp.uid])) and the commands. Everything runs as undoable commands in one transaction. Placement is converted from test-local to world coordinates in exactly one function.
• calibration/record.py: CalibrationRecord, a small asset type registered through register_asset_types. It stores the generator id, a format version, the params and the uids of the items it generated. It is saved in the .ryp like any asset.
• commands/calibration_cmd.py: CalibrationCmd (editor.calibration) with create(generator_id) and update_params(record, params). update_params regenerates the test in one undo entry, anchored at the test's current position, so a moved test stays where the user put it.
• widgets/calibration_provider.py: CalibrationPropertyProvider, the test settings shown in the right panel (see 2.6).

Steps

Each step is one small PR with its own CHANGELOG.md line. New strings go in _(), and no .po/.pot changes are included. Lint runs first (pixi run lint), then the targeted tests.

2.1 Characterization tests for the Material Test Grid (tests only)

• New rayforge-addon-laser/tests/commands/test_material_test_cmd.py. It covers:
  • create_test_grid() adds one step and one workpiece, linked through workpiece_uids, centered on axis_extents.
  • One undo removes both.
  • Changing grid_dimensions resizes the preview.
• A golden test on the assembler output for each grid mode: the (speed, power, passes, offset) of every cell and the order they run in, for the default 5×5. 2.7 uses this golden to prove equivalence.
• Users see: nothing.

2.2 Extract the generator interface (refactor, no behaviour change)

• Add calibration/generator.py and calibration/layout.py.
• MaterialTestGridGenerator (calibration/material_test_grid.py) returns a layout with a single entry: the procedural preview workpiece plus a configured MaterialTestStep.
• MaterialTestStep.recipe_varset() stays as it is because recipes depend on it ("MATERIAL_TEST" in core/recipe.py). The generator reuses it.
• Users see: nothing. 2.1 stays green, plus unit tests for the pure layout.

2.3 Shared builder and one-transaction undo

• Add calibration/builder.py. MaterialTestCmd.create_test_grid() becomes a thin call into the builder, still targeting the active layer in this step.
• Assets are added through ListItemCommand(..., "add_asset") inside the transaction. This fixes the asset left behind after an undo, and it is the only intended change. 2.1 gets a test for it.
• Users see: undo now cleans up fully. Nothing else changes.

2.4 Dedicated layer

• The builder creates a new layer named after the test, with the step in that layer's workflow, and makes it active. Undo removes the layer, its content and the asset.
• Users see: the Material Test Grid gets its own layer, like the tests in feat: add Interval Test and Focus Test generators to the Tools menu #539. Tests: layer count and contents, undo and redo, and that the original active layer is restored on undo.

2.5 Persisted parameters and regeneration

• Add CalibrationRecord and CalibrationCmd.update_params(). Regeneration replaces the items listed in the record in one undo entry, at the test's current anchor.
• For the Material Test Grid, edits on the step settings page go through update_params instead of the descendant_updated → sync_preview_from_step path, which is removed.
• Users see: nothing new, but a changed test survives save and reload with its parameters. Tests: record round-trip through to_dict/from_dict, regeneration keeps position, one undo entry per change.

2.6 Test settings in the right panel (the "dialog" opens immediately and never covers the test)

• CalibrationPropertyProvider is registered in property_provider_registry. It shows a "Test Settings" group whenever the selection contains workpieces of a calibration record, built with VarSetRowList. Changes are debounced the same way SketchPropertyProvider does it (300 ms).
• After Tools → Create … the builder selects the new test, so the panel appears straight away in the right panel and the canvas and test stay fully visible. Selecting any cell later brings it back. Removing the test is a plain undo or a layer delete. There is no Create/Cancel window.
• I prefer this to a dialog window because GTK4 can't position windows on Wayland, so any dialog can end up on top of the test again.
• The Material Test Grid's Preset, Grid, Labels and Parameters sections move from MaterialTestGridSettingsPage to the panel. The step dialog keeps Laser and Post Processing.
• Users see: all tests behave the same. Tests: provider can_handle, values to update_params, panel shown after creation (uitest).

2.7 Per-cell editable Material Test Grid (only if you agree, see question 1)

• MaterialTestGridGenerator.layout() emits:
  • one standard Engrave or Contour step per cell, with power, speed, MultiPassTransformer passes and bidir_x_offset_mm, in the assembler's risk order;
  • one labels workpiece with one low-power step.
• MaterialTestStep stays registered (hidden) so older projects load and run unchanged.
• Users see: every cell can be adjusted in its own step settings. Tests: the 2.1 golden matches the per-cell steps (values and order).
• Docs: material-test-grid.md. Screenshotter: scripts/screenshot/material_test.py captures the panel instead of the step dialog.

Phase 3 (on top of the above, replacing the code in #539)

1. Interval Test. calibration/interval_test.py as a generator (pure layout and VarSet from feat: add Interval Test and Focus Test generators to the Tools menu #539, reusing the shared row and label helpers), a Tools menu entry, tests, docs, and the screenshot target scripts/screenshot/interval_test.py registered in cli.py.
2. Focus Test. calibration/focus_test.py. Z axis, manual and ramp modes, with CommandEntry items between the lines, and modes offered depending on the driver and the Automation addon. Tests, docs, and screenshot targets for the panel in each mode.

#539 stays a draft for reference and gets closed when 3.1 is opened.

Open questions

1. Per-cell Material Test Grid: at the 20×20 maximum that is 400 steps in one workflow. Do you want 2.7 like this, a lower grid limit, or should the grid stay one assembler step (own layer and panel, but no per-cell editing)?
2. Where to keep the parameters: a CalibrationRecord asset as proposed, or something on Layer (extra or a field)?
3. Regeneration vs. hand edits: when the test parameters change, should regeneration overwrite cells the user edited by hand (proposed, with a note in the panel), or should the parameters lock once a cell has been edited?
4. Placement: keep centering on axis_extents as today, or use machine.work_area so work margins are respected?
5. Labels in per-cell mode: fine to build them in Python with text_to_geometry, or should the label layout become a raygeo helper shared with the assembler?
6. Settings in the right panel instead of a dialog window (2.6): is that what you had in mind with "dialog opens immediately", or do you prefer a dialog that is placed so it doesn't cover the test?

[github-actions]

This pull request was closed automatically because its description does not follow the pull request template. Problems found:

• section "What does this PR change?" is missing
• section "Why did you choose this specific architectural approach?" is missing
• section "How did you manually test this locally?" is missing
• section "Checklist" is missing

Please edit the pull request description to answer every section and to tick every checkbox; as soon as it complies, the pull request is reopened automatically. See CONTRIBUTING.md.

[knipknap]

Sorry, bot went wild. Tried to reopen and that is also blocked... ugh. Checking what can be done.