From 251c6248eb377440fbeb7ef804e35f9138a66f9e Mon Sep 17 00:00:00 2001 From: Mike Langmayr <1809691+mikelangmayr@users.noreply.github.com> Date: Thu, 24 Sep 2026 11:03:53 -0700 Subject: [PATCH 1/2] Drop the pull request preview deployment in favour of the build artifact --- .github/workflows/docs.yml | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-) diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 6211a93..27ee3dc 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -4,7 +4,6 @@ on: push: branches: ["main"] pull_request: - types: [opened, synchronize, reopened, closed] workflow_dispatch: # Serialize per ref so a rapid second push cannot race the first one's gh-pages commit. Only pull @@ -15,11 +14,9 @@ concurrency: permissions: contents: write - pull-requests: write jobs: build: - if: github.event.action != 'closed' runs-on: ubuntu-latest timeout-minutes: 30 @@ -62,6 +59,7 @@ jobs: sphinx-build -W -b html -d docs/_build/doctrees docs docs/_build/html touch docs/_build/html/.nojekyll + # How a pull request build is viewed rendered, since only main is published - name: Upload rendered site uses: actions/upload-artifact@v4 with: @@ -78,27 +76,3 @@ jobs: with: github_token: ${{ secrets.GITHUB_TOKEN }} publish_dir: docs/_build/html - # Preserve previews/, which lives on the same branch - keep_files: true - - # Fork pull requests get a read-only token and cannot deploy; they use the artifact instead - - name: Publish pull request preview - if: > - github.event_name == 'pull_request' && - github.event.pull_request.head.repo.full_name == github.repository - uses: rossjrw/pr-preview-action@v1 - with: - source-dir: docs/_build/html - umbrella-dir: previews - action: deploy - - remove-preview: - if: github.event.action == 'closed' - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - uses: rossjrw/pr-preview-action@v1 - with: - umbrella-dir: previews - action: remove From a189f290a1b8dd7403fdccfe9dfc19b2811d41b7 Mon Sep 17 00:00:00 2001 From: Mike Langmayr <1809691+mikelangmayr@users.noreply.github.com> Date: Thu, 24 Sep 2026 11:03:53 -0700 Subject: [PATCH 2/2] Finish the getting started, FITS, controller command and development chapters --- docs/commands/controller.md | 86 +++++++++++++++++++++++++++-------- docs/development/index.md | 43 ++++++++++++------ docs/fits/index.md | 70 ++++++++++++++++++---------- docs/getting-started/index.md | 72 +++++++++++++++++++---------- 4 files changed, 188 insertions(+), 83 deletions(-) diff --git a/docs/commands/controller.md b/docs/commands/controller.md index 7a51c96..d37c36d 100644 --- a/docs/commands/controller.md +++ b/docs/commands/controller.md @@ -1,38 +1,86 @@ # Controller commands -Commands passed through to the detector controller largely untouched. The server does not interpret -them, so this is the escape hatch for anything the higher-level commands do not cover. +Commands that reach the detector controller rather than being interpreted by the server. This is +the escape hatch for anything the higher-level commands do not cover. -:::{note} -Fills out in M3 alongside the architecture chapter. -::: +Two routes exist. `native` passes a command straight through and returns the reply. The commands +routed through `controller_cmd`, marked Archon in the [base command table](base.md), are +server-side implementations of controller-specific features. ## Archon -`native` sends an Archon command directly and returns its reply. The commands most worth knowing: +`native ` sends `` to the Archon and returns its reply without parsing it, beyond +confirming that a reply came back. The commands most worth knowing: `FRAME` -: Frame buffer status: which buffer is complete, frame numbers, timestamps, sizes. +: Frame buffer status: which buffer is complete, frame numbers, timestamps and sizes. `STATUS` : Backplane status, including module temperatures, voltages and currents. `SYSTEM` -: Module complement: what is in each slot, with type, revision and version. +: Module complement: what occupies each slot, with type, revision and version. + +`TIMER` +: The controller's free-running timer, useful for checking the link is alive. -`raw` reaches the Archon configuration memory directly, to read the loaded configuration or set keys -in it. +The server itself parses `FRAME`, `STATUS` and `SYSTEM` replies for its own bookkeeping, so use +`native` for inspection rather than as a control path. + +Beyond `native`, the Archon-specific commands are `raw` for configuration memory, `getp` and `setp` +for parameters, `inreg` for a VCPU input register, `loadtiming` and `readacf` for loading, `mode` +for camera modes, `autofetch_mode`, and `heater` and `sensor` for the thermal modules. The +[heater and sensor](#heater-and-sensor) syntax is below. ## ARC (AstroCam) -ARC controllers take three-letter DSP commands. The ones the server exposes: +:::{warning} +`native` is not implemented on an ARC build. `AstroCamInterface::native` logs "not yet implemented" +and returns success, so the command replies `DONE` while doing nothing. + +`controller_cmd` is not implemented either, so every command in the Archon list above returns +`not_supported` on an ARC build. +::: + +Passing three-letter DSP commands through to an ARC controller is therefore not currently possible +from the command interface. + +## Heater and sensor + +For Archon **Heater** and **HeaterX** modules. Both require firmware to be loaded and a +sufficiently recent backplane. -| Command | Meaning | +### heater + +``` +heater [ [target] | | PID [

] + | RAMP [ [rate]] | ILIM [val] | INPUT [A|B|C] ] +``` + +| Form | Effect | +|---|---| +| `heater ` | Get enable state and target | +| `heater [target]` | Set enable state, optionally the target | +| `heater ` | Set the target | +| `heater PID [

]` | Get or set the P, I and D terms, each 0 to 10000 | +| `heater RAMP [ [rate]]` | Get or set ramp enable and rate, 1 to 32767 | +| `heater ILIM [val]` | Get or set the current limit, 0 to 10000 | +| `heater INPUT [A\|B\|C]` | Get or set the input sensor. `C` requires HeaterX. | + +The target range defaults to backplane-dependent limits, overridable with `HEATER_TARGET_MIN` and +`HEATER_TARGET_MAX` in degrees C. See [core keys](../configuration/core.md). + +### sensor + +``` +sensor [ | AVG [ ] ] +``` + +| Form | Effect | |---|---| -| `PON` | Power on | -| `POF` | Power off | -| `RDM` | Read memory | -| `WRM` | Write memory | -| `SBN` | Set bias number | -| `SMX` | Set multiplexer | -| `TDL` | Test data link | +| `sensor ` | Get the RTD excitation current in nanoamps | +| `sensor ` | Set it, 0 to 1600000 nA | +| `sensor AVG` | Get the digital averaging count | +| `sensor AVG ` | Set it, N in {1, 2, 4, 8, ... 256} | + +Sensor `C` exists only on HeaterX modules. diff --git a/docs/development/index.md b/docs/development/index.md index 71d3397..f777ac8 100644 --- a/docs/development/index.md +++ b/docs/development/index.md @@ -7,10 +7,9 @@ pip install -r docs/requirements.txt sphinx-build -W -b html -d docs/_build/doctrees docs docs/_build/html ``` -`-d` keeps Sphinx's build cache out of the output directory, which is published verbatim. - `-W` turns warnings into errors, which is what CI uses, so a broken cross-reference fails the build -rather than shipping a dead link. +rather than shipping a dead link. `-d` keeps Sphinx's build cache out of the output directory, which +is published verbatim. The Python API page needs the compiled `camera_interface` module to be importable. Without it the build still succeeds and the page shows a note in place of the reference, so a docs-only change does @@ -18,8 +17,8 @@ not require a full C++ build locally. ## How the docs stay current -Reference tables that restate something the source already knows are generated at documentation -build time and cross-checked against the source, so the build fails when they diverge: +Reference tables that restate something the source already knows are generated at build time and +cross-checked against the source, so the build fails when they diverge: | Table | Source of truth | |---|---| @@ -40,6 +39,14 @@ diverged: `CAMERAD_SYNTAX` feeds the `help` output and still advertises commands longer implements. The dispatch is what actually answers a client. ::: +### Adding a command or key + +1. Make the change in the C++ as usual. +2. Run the docs build. It fails, naming what is now undescribed. +3. Add the entry to `docs/data/commands.yaml` or `docs/data/config_keys.yaml`. + +Nothing has to be added to a page: the tables pick it up. + ## Documentation layout ``` @@ -51,24 +58,23 @@ docs/ / one directory per chapter ``` -Pages are Markdown via MyST. Use `{source}`` `path` `` to link to a file in the repository rather -than pasting signatures into the prose. +Pages are Markdown via MyST. Link to a file in the repository with +`` {source}`camerad/camera_interface.h` `` rather than pasting signatures into the prose; the C++ +reference here is narrative by design. ## Publishing `.github/workflows/docs.yml` builds on every pull request and every push to `main`. -- A pull request publishes to `previews/pr-/` on the `gh-pages` branch, and a bot comments the - link. The preview is removed when the PR closes. -- A merge to `main` publishes to the site root. -- Every build uploads the rendered HTML as a workflow artifact, which is the fallback for pull - requests from forks, since those get a read-only token and cannot deploy. +- A merge to `main` publishes the site to the root of the `gh-pages` branch. +- Every build uploads the rendered HTML as a workflow artifact, which is how a pull request build is + viewed rendered. Download it from the run's summary page. +- `workflow_dispatch` on `main` also publishes, so the site can be restored without an empty commit. :::{important} The `gh-pages` branch needs a `.nojekyll` file at its root. Without it Pages runs the output through Jekyll, which skips directories beginning with an underscore, and the whole site loads with no CSS -because `_static/` returns 404. A marker inside a preview subdirectory is not enough; it has to be at -the branch root. Recreating `gh-pages` from scratch means adding it again. +because `_static/` returns 404. Recreating `gh-pages` from scratch means adding it again. ::: ## Testing camerad itself @@ -77,7 +83,14 @@ the branch root. Recreating `gh-pages` from scratch means adding it again. make run_unit_tests && ./bin/run_unit_tests ``` -The end-to-end tests run against the [emulator](../emulator/index.md) in CI. +The end-to-end tests run against the [emulator](../emulator/index.md) in CI, in +`.github/workflows/emulator-integration.yml`. + +:::{warning} +The build workflow compiles the default target, and an instrument only when one is named explicitly. +`hispec_tracking_camera` is the only instrument any workflow builds, so the other modules can fall +behind changes to the core without CI noticing. +::: ## Instrument submodules diff --git a/docs/fits/index.md b/docs/fits/index.md index 2b8effd..29c65a6 100644 --- a/docs/fits/index.md +++ b/docs/fits/index.md @@ -1,42 +1,58 @@ # FITS output -:::{note} -Expands in M2, when the ATC keyword table becomes generated output. Naming, cube layout and the -system keyword tables are written then. -::: - -`camerad` writes FITS through an asynchronous writer: the readout thread hands a completed frame to -a queue, and a dedicated thread writes it. Configuration is under -[frame output keys](../configuration/frame-outputs.md). +`camerad` writes FITS through an asynchronous writer: the consumer thread hands a completed frame to +a queue and returns, and a dedicated thread writes it. Enabling and locating the output is covered +in [frame output keys](../configuration/frame-outputs.md). ## Filenames -`fitsnaming` selects between two schemes: +The writer builds each name itself: -`time` -: The filename carries a timestamp, so names never collide and sort chronologically. +``` +/_.fits +``` -`number` -: The filename carries an incrementing image number, reported and set with `imnum`. +The frame number is zero-padded to eight digits. If that path already exists the writer appends +`_1`, `_2` and so on until it finds a free name, so a file is never silently overwritten. + +`FITS_AUTODIR` puts all of this inside a `YYYYMMDD` subdirectory of `FITS_OUTPUT_DIR`. + +:::{warning} +Older documentation describes a `fitsnaming` command choosing between timestamp and number +schemes, with `imnum` and `fitsname` to go with it. Those commands no longer exist. Naming is +entirely `FITS_BASENAME` plus the frame number. +::: -`autodir` adds a `YYYYMMDD` subdirectory under the image directory. Which midnight that rolls over -on follows `TM_ZONE`. +## Cubes -## Cubes and extensions +`datacube true` makes the writer accumulate frames into one file instead of writing one file per +frame. The primary HDU is header-only (`NAXIS=0`) and each frame becomes an image extension. The +cube is finalized when the exposure command finishes, which the server signals to every output +after the last frame. -`datacube` writes successive frames as planes of one cube rather than separate files. For detectors -read through several amplifiers, `mexamps` writes each amplifier as its own extension, and `mex` -controls multi-extension output generally. +`datacube` is the only runtime option the FITS writer accepts; anything else is rejected. ## Keywords -Three sources of keywords end up in a header: +Four sources end up in a header. -1. **System keywords**, written by the server: geometry, timing, exposure and controller state. -2. **Instrument keywords**, from the instrument module's header dictionary. The ATC dictionary is in - {source}`camerad/Instruments/hispec_tracking_camera/fits_header_dictionary.cpp`. -3. **User keywords**, added at runtime with `key`. `writekeys` controls whether they are written - before or after the exposure, which matters for anything whose value changes during it. +Writer keywords +: Added to every file: `FRAMENO`, `TIMESTMP` (the Archon timestamp in 0.01 microsecond units), + `DATE` (when the file was written) and `FILENAME`. `FILENAME` carries the base name only, because + a FITS card holds 68 characters and a deployment path can exceed that. + +Per-exposure keywords +: Resolved once when the exposure starts and shared by all of its frames. + +Per-frame keywords +: Rebuilt for each frame, for values that change between reads within one exposure. + +User keywords +: Added at runtime with `key KEYWORD=VALUE//COMMENT`. `key list` shows both the system and user + sets, and `key KEYWORD=.` deletes one. + +Instrument modules supply their own dictionary on top of this; see the +[tracking camera keyword table](../instruments/hispec-tracking-camera.md) for the worked example. ## Checking a header @@ -47,3 +63,7 @@ rather than going unnoticed. It needs no FITS library, and both emulator CI jobs ```bash python3 python/tests/fits_header_check.py --exptime ``` + +## Pixel format + +Frames of 2 bytes per pixel are written as `USHORT_IMG`, anything wider as `ULONG_IMG`. diff --git a/docs/getting-started/index.md b/docs/getting-started/index.md index 1a7559a..5957c21 100644 --- a/docs/getting-started/index.md +++ b/docs/getting-started/index.md @@ -1,7 +1,7 @@ # Getting started -This walks from a clean checkout to a FITS file on disk, with no detector controller attached: the -Archon emulator stands in for the hardware. +From a clean checkout to a FITS file on disk, with no detector controller attached: the Archon +emulator stands in for the hardware. ## Dependencies @@ -24,8 +24,8 @@ sudo apt-get install -y build-essential cmake ninja-build \ ## Build -`-DCONTROLLER=` is required; CMake stops with an error without it. `-DINSTRUMENT=` is optional and -selects an instrument module from `camerad/Instruments/`. +`-DCONTROLLER=` is required; CMake stops with an error without it. `-DINSTRUMENT=` selects an +instrument module from `camerad/Instruments/`. ```bash git clone --recurse-submodules \ @@ -36,10 +36,8 @@ make ``` Binaries land in `bin/` in the source tree. `make install` copies them under -`CMAKE_INSTALL_PREFIX` instead, which is the better choice for anything deployed. - -The build options are covered in full under [configuration](../configuration/index.md); the ones that -change what gets built are `ENABLE_SHM_OUTPUT`, `BUILD_PYTHON_MODULE` and `INTERFACE_TYPE`. +`CMAKE_INSTALL_PREFIX` instead, which is the better choice for anything deployed. The full set of +build options is in the [configuration reference](../configuration/index.md). :::{note} Instrument modules are git submodules. A clone without `--recurse-submodules` leaves @@ -47,20 +45,22 @@ Instrument modules are git submodules. A clone without `--recurse-submodules` le `git submodule update --init --recursive`. ::: -## Run the emulator +## Run it + +This uses `config/frame_outputs_test/frame_outputs_test.cfg`, which is the config the CI integration +test drives, so it is known to work end to end. It enables the FITS writer, points `ARCHON_IP` and +`ARCHON_PORT` at the emulator, and listens on port 3131. -The emulator reads `EMULATOR_PORT` and `EMULATOR_SYSTEM` from the same `.cfg` the server uses, so -pointing `ARCHON_IP` and `ARCHON_PORT` at it is all that separates a test rig from real hardware. The -shipped `config/demo/demo.cfg` already does this. +Start the emulator: ```bash -bin/camerad-emulator config/demo/demo.cfg -i generic +bin/camerad-emulator config/frame_outputs_test/frame_outputs_test.cfg -i generic ``` -## Run the server +Then the server, in another terminal: ```bash -bin/camerad --foreground --config config/demo/demo.cfg +bin/camerad --foreground --config config/frame_outputs_test/frame_outputs_test.cfg ``` `--config` is required. Without `--foreground` the server daemonizes. Logging always goes to a daily @@ -68,11 +68,10 @@ file under `LOGPATH`; `--foreground` additionally writes it to stderr. ## Take an exposure -`camerad-socksend` sends one command and prints the reply. Point it at the blocking port from the -`.cfg` (`BLKPORT`). +`camerad-socksend` sends one command and prints the reply. Point it at `BLKPORT`. ```bash -send() { bin/camerad-socksend -p 3031 -t 60 "$1"; } +send() { bin/camerad-socksend -p 3131 -t 60 "$1"; } send "open" # connect to the controller send "load" # load firmware named by DEFAULT_FIRMWARE @@ -81,16 +80,41 @@ send "exptime 1.5" # seconds, because this config leaves LONGEXPOSURE at its d send "expose 1" ``` -Each returns `DONE` or `ERROR`. The FITS file appears under `IMDIR`, named from `BASENAME`. +Each returns `DONE` or `ERROR`. The FITS file appears under `FITS_OUTPUT_DIR`, which this config +sets to `/tmp/ci_fits_test`, named from `FITS_BASENAME`. + +Check the header carries what the instrument promises: + +```bash +python3 python/tests/fits_header_check.py /tmp/ci_fits_test/ci_frame_outputs_*.fits --exptime 1.5 +``` :::{warning} -`DONE` means the server accepted and completed the command, not that every frame output succeeded. -The FITS writer drops frames by design when the queue backs up, so `DONE` from `expose` is not a -promise that a file was written. [Frame outputs](../fits/index.md) explains how to check. +`DONE` means the server accepted and completed the command, not that a file was written. The FITS +writer drops frames by design when its queue backs up, so the file on disk is the real confirmation. +See [frame outputs](../configuration/frame-outputs.md). ::: +:::{note} +`config/demo/demo.cfg` is a smaller example, but it enables no frame outputs at all, so an exposure +against it returns `DONE` and writes nothing. It is a starting point for a config, not a working +demonstration. +::: + +## Where things went + +| What | Where | +|---|---| +| FITS files | `FITS_OUTPUT_DIR` | +| Log file | a daily file under `LOGPATH` | +| Shared-memory stream | `SHM_DIR`, or `MILK_SHM_DIR`, or `/milk/shm` | + +Note that `IMDIR` and `BASENAME` appear in the shipped configs but are read by nothing. Output +location is set entirely by the [frame output keys](../configuration/frame-outputs.md). + ## Next +- What the commands are: [command reference](../commands/index.md) - Drive the camera from Python instead of a socket: [Python bindings](../python/index.md) -- Understand what the emulator does and does not model: [Emulator](../emulator/index.md) -- Configure a real instrument: [Instruments](../instruments/index.md) +- What the emulator does and does not model: [emulator](../emulator/index.md) +- Configure a real instrument: [instruments](../instruments/index.md)