feat: add TPX3 two-quad 4x2 mask support

.github/workflows/ci-scripts.yml

@@ -106,6 +106,7 @@ jobs:
         if: ${{ matrix.run_validation }}
         run: |
           python3 test/test_generate_dependency_report.py
+          python3 test/generate_tpx3_4x2_demo_calibration.py --check
           python3 test/verify_coordinate_map.py
           python3 test/validate_bob_xml.py
           python3 test/validate_bob_prefix_macros.py

README.md

@@ -25,7 +25,7 @@ Notes:
 * ADCore, ADSupport, and other EPICS module versions are supplied by the surrounding EPICS release configuration (`configure/RELEASE_LIBS_INCLUDE` and local overrides); this repository does not lock them independently.
 * **Preview Images**: Uses TCP streaming (jsonimage format) for preview images. GraphicsMagick HTTP method has been removed. For backward compatibility, the GraphicsMagick implementation is preserved in the `preserve/graphicsmagick-preview` branch.
 * This has only been developed/tested on ubuntu 22.04, 20.04, 18.04, RHEL 7.9, RHEL 9.6 Linux 64-bit machines.
-* Layout support in driver and OPI is most complete for **1 chip** and **2×2 quad**; **8-chip** (e.g. 2×4 mosaic, two SPIDR boards) has IOC/DB/driver support documented in [documentation/8chip-migration.md](documentation/8chip-migration.md)—validate BPC/mask mapping and screens on your hardware.
+* Layout support in driver and OPI is most complete for **1 chip** and **2×2 quad**; **8-chip** (a 4×2 mosaic assembled from two quads) has IOC/DB/driver support documented in [documentation/8chip-migration.md](documentation/8chip-migration.md)—validate BPC/mask mapping and screens on your hardware. The single-board full-rate SpidrTurbo layout remains a distinct, unqualified mapping.
 * **Serval versions**: The master branch supports both Serval 4.x.x and 3.x.x and is recommended (no need to use the 3.3.2 branch). The "dual image" issue was resolved in Serval 4.1.5; **Serval 4.1.5 remains the recommended stable version** for 4.x. R1-7-4 acquisition-lifecycle qualification also passed with **Serval 4.1.6-EXPERIMENTAL build 1760** on the TPX3 and MPX3 emulators. Serval and emulator versions must match. Data replay has been tested and is currently supported only with older Serval (3.3.2). Serval 2.x.y is in a separate branch and is not under active development.
 * The driver has been developed using the TimePix3 Emulator and real detectors (quad-chip and single-chip).
 

RELEASE.md

@@ -79,13 +79,13 @@ R1-8-0 (in progress)
 * Keep accumulated preview-histogram Y autoscaling enabled as frame updates
   arrive. No histogram data, axis PV, or acquisition behavior changes.
 * SNS Phoebus qualification passes for 256x256 and 512x512 mask displays and
-  preview-histogram accumulation. The 1024x512 mask-display check remains
-  pending.
+  preview-histogram accumulation. The 1024x512 mask display also passes with
+  the two-quad TPX3 emulator; physical-detector qualification remains pending.
 
 ### Eight-chip IOC waveform capacity
 
 * Raise the TPX3 NDStats profile capacity to 1024x512 so the horizontal cursor
-  profile is not truncated at 512 pixels for an eight-chip 2x4 mosaic. Smaller
+  profile is not truncated at 512 pixels for an eight-chip 4x2 mosaic. Smaller
   detector geometries continue to publish their actual profile lengths.
 * Select `MASK_BPC_NELEMENTS=524288` to reserve 1024x512 record capacity for
   mask/BPC, PixelConfig-difference, and image-accumulation waveforms. This is a
@@ -95,12 +95,30 @@ R1-8-0 (in progress)
   `MASK_BPC_NELEMENTS`.
 * SNS Phoebus qualification passes for the full-width 1024-pixel horizontal
   profile, 1024x512 image accumulation, and preview-histogram accumulation.
-* The 1024x512 mask display remains unqualified. The available quad BPC
-  provides 262144 samples rather than 524288, and the current rectangular
-  geometry helper derives a 128-pixel chip (`16384` PixelConfig bytes) from
-  the 4x2 layout instead of the required 256-pixel chip (`65536` bytes).
-  Eight-chip calibration composition and rectangular mask geometry are
-  deferred to a separate change.
+* Add an opt-in synthetic 4x2 demo calibration pair: a deterministic
+  524288-byte BPC and a DACS file with unique `[Chip0]` through `[Chip7]`
+  sections, both reproducibly composed from the existing quad demo files.
+* Correct the rectangular eight-chip geometry using captured Serval metadata:
+  `RowLen=4` chips across, `NumberOfRows=512`, `NumberOfChips=8`, and
+  `PixCount=524288`. The isolated eight-chip path derives a 1024x512 raster,
+  two chip rows, and a 256-pixel chip edge (`65536` PixelConfig bytes per
+  chip), while preserving the established one- and four-chip geometry path.
+  Eight-chip mask mapping requires two distinct TPX3 quad board IDs and fails
+  closed for the separate single-board SpidrTurbo layout.
+* Replace stale global PixelConfig errors with a completion message after each
+  refresh; per-chip status fields continue to report matches and failures.
+* Map two adjacent quads as `2,1,4,7 / 3,0,5,6`, with the right module rotated
+  180 degrees, and add compiled full-grid round-trip and portable coordinate
+  tests. The distinct single-board SpidrTurbo mapping remains deferred.
+* Qualify the opt-in 4x2 files and mask workflow with the two-quad TPX3
+  emulator on 2026-10-04: BPC and DACS uploads returned HTTP 200; all eight
+  65536-byte PixelConfig responses matched their BPC slices with zero
+  mismatches; and a rectangular mask propagated through the 1024x512 mask
+  preview and masked-pixel JSON export. The 1024x512 image-accumulation and
+  histogram views continued to operate during the same qualification.
+* Physical 1024x512 mask qualification and detector-specific eight-chip
+  equalization remain pending; the checked-in calibration is explicitly
+  synthetic and must not be used as qualified hardware calibration.
 
 ### Bounded rolling accumulation
 
@@ -183,12 +201,12 @@ R1-8-0 (in progress)
 ### Validation
 
 * Full IOC build passed.
-* 346/346 deterministic protocol, lifecycle, rolling-window, numeric-range,
+* 358/358 deterministic protocol, lifecycle, rolling-window, numeric-range,
   path-containment, atomic-write, and destination-policy checks passed.
-* Coordinate-map validation passed for 11 implemented cases; five planned
-  physical SpIDR 2x4 cases remain deferred.
+* Coordinate-map validation passed for 13 implemented cases; five planned
+  physical SpIDR 4x2 cases remain deferred.
 * All Phoebus BOB files passed XML validation.
-* REUSE validation passed for 372/372 files.
+* REUSE validation passed for 398/398 files.
 * TPX3 emulator runtime qualification passed at 10 Hz for Img and PrvHst
   windows of 10 and 100 frames: requested capacity equaled current fill, memory
   plateaued at the expected bounded values, per-frame publication operated, and
@@ -315,7 +333,7 @@ Driver / user-visible version **1.7.3** (see `ADTIMEPIX_*` in `ADTimePix.h`).
   * MPX3 emulator qualification with Serval 4.1.6 completed six finite acquisitions with frame preview, integrated preview, and full image enabled. All three TCP streams closed normally after every measurement, every subsequent acquisition began from Idle, and Serval processed 139 frames with zero drops and no TCP sender errors.
 
 * **Release qualification and deferred scope**:
-  * R1-7-3 is software- and emulator-qualified. Physical post-equalization MPX3 mask/PixelConfig confirmation, broader physical reconnect testing, and 2x4 SpIDR placement/rotation/BPC ordering remain controlled-hardware follow-up work; a Serval PixelConfig match is not proof of detector-register state.
+  * R1-7-3 is software- and emulator-qualified. Physical post-equalization MPX3 mask/PixelConfig confirmation, broader physical reconnect testing, and 4x2 SpIDR placement/rotation/BPC ordering remain controlled-hardware follow-up work; a Serval PixelConfig match is not proof of detector-register state.
   * Standard acquisition start/stop and reconnect paths are covered, but exhaustive transactional partial-start and injected-fault coordination remain future work.
   * Large retained image/histogram accumulation workloads do not yet have a geometry-aware memory-capacity or unsigned-to-signed overflow contract. Validate site workloads and monitor resource use.
   * File and destination configuration assume a trusted deployment boundary; canonical writable-root confinement, destination allowlists, access control, and credential policy remain deployment-hardening work.

documentation/8chip-migration.md

@@ -1,4 +1,4 @@
-# Eight-chip (2×4 / dual SPIDR) support
+# Eight-chip 4×2 (two-quad / dual-SPIDR) support
 
 This document tracks IOC and driver changes for detectors with up to eight TimePix3 chips (e.g. two SPIDR boards × four chips).
 
@@ -12,15 +12,26 @@ This document tracks IOC and driver changes for detectors with up to eight TimeP
   - **Boards:** `TPX3_BOARDS2_ID`, `TPX3_BOARDS2_IP`, `TPX3_BOARDS_CH5`…`CH8` from `Info.Boards[1]` when available (`CH5`…`CH8` are that board’s four chip JSON objects).
 - **Database:** `ADTimePix3.template` adds `BChBoard2Id_RBV`, `IpAddr2_RBV`, `Chip5_RBV`…`Chip8_RBV`.
 - **Mask / BPC indexing (`mask_io.cpp`):** For `numChips == 8` and
-  **DetectorOrientation UP (0)** only, `pelIndex` and `bpc2ImgIndex` contain a
-  linear rectangular-mosaic mapping. Live 4x2 mask operation is not yet
-  qualified because `rowsCols()` still derives square-layout dimensions; see
-  [Known two-quad mask limitations](#known-two-quad-mask-limitations).
+  **DetectorOrientation UP (0)** only, `pelIndex` and `bpc2ImgIndex` implement
+  the two-quad 4x2 placement `2,1,4,7 / 3,0,5,6`. The right-hand quad is
+  rotated 180 degrees relative to the left-hand quad. The distinct single-board
+  full-rate SpidrTurbo mapping remains unimplemented pending golden hardware
+  vectors.
+- **Geometry:** the captured TPX3 Serval response reports `Info.RowLen=4`
+  chips across, `NumberOfRows=512`, `NumberOfChips=8`, and
+  `PixCount=524288`. The eight-chip-only geometry path derives a 4x2 grid of
+  256x256 chips and a 1024x512 raster. The established one- and four-chip path
+  is unchanged. Runtime mask mapping also requires two distinct `41...` TPX3
+  quad board IDs; the different single-board `84...` SpidrTurbo layout fails
+  closed.
+- **Synthetic calibration fixture:** `vendor/tpx3/4x2/` contains a reproducible
+  524288-byte BPC and a DACS file with unique `[Chip0]` through `[Chip7]`
+  sections for software and emulator testing. It is not hardware equalization.
 
 ## NDStats profile dimensions
 
 `profiles/tpx3/unique.cmd` sets the NDStats waveform capacities to
-`XSIZE=1024` and `YSIZE=512`, matching the largest supported 2x4 mosaic.
+`XSIZE=1024` and `YSIZE=512`, matching the largest supported 4x2 mosaic.
 The Phoebus horizontal cursor profile reads `Stats1:ProfileCursorX_RBV`, whose
 record `NELM` is set from `XSIZE`; a value of 512 therefore truncates a
 1024-pixel row profile. Smaller detector geometries remain supported because
@@ -41,42 +52,47 @@ eight-chip capacity by default:
 |-------|----------------|------------|----------------------|
 | 1 | 1×1 | 256×256 | 65536 |
 | 4 | 2×2 | 512×512 | 262144 |
-| 8 | 2×4 | 1024×512 | 524288 |
+| 8 | 4×2 | 1024×512 | 524288 |
 
 `envPaths` does **not** set this macro; it only notes that `profiles/tpx3/unique.cmd` must define it.
 
 The configured value is record capacity only. It prevents EPICS waveform
 truncation but does not create a matching calibration or validate the mapping.
 
-## Known two-quad mask limitations
-
-Runtime qualification with a 1024x512 two-quad detector passed the full-width
-horizontal profile, image-accumulation displays, and preview histogram. Mask
-qualification remains blocked by two independent issues:
-
-1. The available TPX3 calibration is a four-chip, 262144-byte BPC. Serval
-   accepts one selected pixel-configuration file, so an eight-chip detector
-   needs one validated 524288-byte BPC containing the chip blocks in final
-   Serval chip order. The DACS file likewise needs unique `[Chip0]` through
-   `[Chip7]` sections; two unchanged quad DACS files cannot simply be appended.
-2. `rowsCols()` currently calculates `chipPelWidth = NumberOfRows / RowLen`.
-   For 512 rows and four chips per row this produces 128, causing
-   `RefreshPixelConfig` to expect 16384 bytes per chip instead of 65536. The
-   rectangular geometry calculation and compiled tests require a separate
-   driver change.
-
-Do not use the eight-chip mask, BPC comparison, or PixelConfig-difference
-results until both the geometry fix and a detector-specific combined
-calibration have been validated. Calibration files are hardware-specific and
-should remain site-local; a repository change may provide deterministic
-composition and validation tooling without publishing facility calibration.
+## Two-quad calibration and qualification boundary
+
+Runtime qualification on 2026-10-04 with the 1024x512 two-quad TPX3 emulator
+passed the full-width horizontal profile, image-accumulation displays, preview
+histogram, and mask workflow. The synthetic BPC and DACS uploads both returned
+HTTP 200. All eight PixelConfig responses decoded to 65536 bytes and matched
+their BPC slices with zero mismatches. A rectangular mask propagated through
+the 1024x512 mask preview and the masked-pixel JSON export. The geometry and
+deterministic two-quad composition prerequisites are also covered by compiled
+and portable tests, but physical-detector mask qualification remains pending.
+
+The checked-in 4x2 BPC repeats the four source chip blocks for chips 4-7. The
+DACS file repeats each source chip's values under a unique destination section
+name. These files verify file sizing, upload selection, per-chip offsets, and
+software/emulator behavior only. Real calibration files are hardware-specific
+and should remain site-local unless redistribution is explicitly appropriate.
+
+Do not claim the eight-chip mask, BPC comparison, or PixelConfig-difference
+path as hardware-qualified until a detector-specific BPC/DACS pair and an
+asymmetric mask have confirmed all eight chip placements.
 
 ## Verify on hardware
 
-- **BPC ↔ image mapping:** The 8-chip branch assumes **linear** chip order (`chip = Y_CHIP * xChips + X_CHIP`) on a uniform grid. **Full-rate SpIDR 2×4** (single board, `chipboardId` `84…`) uses a different mosaic and per-chip rotations — see **[COORDINATE_MAP.md](COORDINATE_MAP.md)** (*SpIDR 2×4 — planned*). Confirm masks and PixelConfigDiff on real data before relying on 8-chip mask paths on that hardware.
+- **BPC ↔ image mapping:** The implemented 8-chip path is specifically the
+  **two-quad** placement `2,1,4,7 / 3,0,5,6`. **Full-rate SpidrTurbo 4×2**
+  (single board, `chipboardId` `84…`) uses a different mosaic and per-chip
+  rotations—see [COORDINATE_MAP.md](COORDINATE_MAP.md). Do not use the
+  two-quad mapping on that hardware.
 - **Phoebus:** `profiles/tpx3/TimePix3Detector.bob` embeds eight chip panels in a **4×2** grid (geometry maintained in the BOB). See also `profiles/tpx3/Acquire/DetectorConfig.bob` (Vth), `common/Mask/` and `common/Acquire/ImgAccumulation.bob` for **1024×512** at `PixCount_RBV == 524288`, and **`common/Detector/TimePixDetectorVoltages.bob`** (Display Builder) or **`TimePixDetectorVoltages.opi`** (legacy BOY under **`op/opi/Detector`**) for **Pwr3–Pwr5**. **`common/Detector/TimePixDetectorHealth.bob`** summarizes health and cross-links to voltages.
 - **Large images:** For 8-chip, keep `MASK_BPC_NELEMENTS` at `524288` (or larger if `PixCount` is greater), `XSIZE` at least `1024`, and `YSIZE` at least `512`. Raise `EPICS_CA_MAX_ARRAY_BYTES` and other ND plugin `NELEMENTS` values if waveforms exceed their configured limits.
 
 ## Calibration
 
-Per-chip / per-geometry calibration policy is outside this IOC change; use naming or paths that include chip count and layout id.
+The repository's `vendor/tpx3/4x2/` pair is deliberately named `synthetic` and
+is opt-in. For physical operation, use names or site-local paths that identify
+the detector and layout, with independently generated values for all eight
+chips.

documentation/CONTRIBUTING.md

@@ -99,6 +99,7 @@ The command should finish with **REUSE Specification 3.0** compliance. If you ad
 ```bash
 make -j
 make -C test runtests
+python3 test/generate_tpx3_4x2_demo_calibration.py --check
 python3 test/verify_coordinate_map.py
 python3 test/validate_bob_xml.py
 reuse lint