Skip to content

700 and 800 series controllers: backup, restore and verify (1.1.0) - #1

Merged
Highsteads merged 1 commit into
Highsteads:mainfrom
autolog:700-800-series
Sep 14, 2026
Merged

Highsteads merged 1 commit into
Highsteads:mainfrom
autolog:700-800-series

Conversation

@autolog

@autolog autolog commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

700 and 800 series controllers

Adds backup, restore and verify for 700 and 800 series sticks (Zooz ZST10 700 / ZST39, Aeotec Z-Stick 7 / Z-Stick 10 Pro, UZB-7 and similar), as discussed on the forum thread. The 500-series code paths are untouched; the new generation gets its own functions in controller_image.py and its own restore body in plugin.py.

How it works

Same shape as the 500 path, but through the NVM3 commands these sticks use — a straight port of zwave-js backupNVMRaw700 / restoreNVMRaw700, whose serial frames I captured from Z-Wave JS UI and matched byte for byte:

  • SetRFReceiveMode off and StopWatchdog before any NVM access; SoftReset afterwards and wait for the unsolicited SerialAPIStarted. The reset is always sent on the way out (failure paths too), so the stick is never left with its radio off. No unplug on restore: the reset is the documented step and the read-back follows at once.
  • NVMOperations 0x2E (16-bit offsets), or ExtendedNVMOperations 0x3D when the stick advertises it in its function bitmask (needed once the NVM passes 64 KB; the ZST39 and Z-Stick 10 Pro do, the 700 stick does not).
  • Probe with a 255-byte read, use the chunk that comes back (64 on all three sticks), 48 on an empty probe.
  • Node ids parsed by payload length, because zwave-js leaves these sticks in 16-bit node id mode and it survives a soft reset (they read as node 0 otherwise).
  • Full SDK version from GetProtocolVersion (7.17.2 where the library string says 7.17), used for the same-software restore guard and shown in Show Plugin Info / the device.

Two things learnt on hardware that shape the restore path

  1. The final write is never performed. The write that reaches the NVM size is answered 0xFF (end of file) and dropped; zwave-js behaves the same and never checks. Those bytes hold nothing any file refers to. The plugin reports it and leaves it alone — and deliberately does not do what the 500 path does for the Gen5's last 16 bytes (split and retry): a 1-byte write near the end hung a ZST39 outright, no ACK, soft reset ignored, watchdog already stopped, replug needed. write_nvm_700 writes in the probed chunk and accepts EOF on the last one; a test asserts the fake's "hang" is never triggered.
  2. Byte-identical read-back is not achievable on NVM3. The firmware appends housekeeping objects into erased space when it restarts (74 bytes after one restore), and an hour of live running adds a few hundred bytes of route updates the same way; a power cycle can relocate whole pages. So compare_images_700 allows differences only inside the declined tail and where the image holds 0xFF; anything overwritten is "did NOT hold". Restore additionally checks the Home ID and the node table reported after the reset against the sidecar. Verify says how many bytes were added in free space and compares the node table too.

Two small fixes found in live use

  • A backup folder pasted with quotes around it was accepted as a relative path and the images landed inside the plugin bundle. clean_folder_path strips quotes, expands ~, and the dialog refuses anything not absolute.
  • The Home ID sanity check took the first Home…_Node… key in Indigo's Z-Wave prefs; a real prefs file carried five Home IDs from past networks and the first was not the live one, so it warned about a mismatch that was not there. It now checks against the set.

Tests

FakeStick700 in tests/fake_stick.py, built from the captured ZST39 and 700-stick payloads, covers the declined tail, the hang, 16-bit node ids, the probe fallback, housekeeping on reset, and a stick that says OK to writes and keeps none. 27 new tests; tests/run.sh green (96 tests, syntax, XML, ruff). All existing tests pass unchanged apart from the one that asserted a 700 stick is refused.

Live verification

Stick Result
Zooz ZST39 LR, 800 series, Z-Wave 7.24, 10 nodes backup (19 s) → verify → restore → verify, all clean; backups byte-identical to Z-Wave JS UI's on both 0x2E and 0x3D; an older image restored brought back its older node table
Aeotec Z-Stick 10 Pro, 800 series, Z-Wave 7.23 (its Z-Wave port; the Zigbee port is ignored) backup (13 s) → verify → restore → verify, all clean
Silicon Labs 700 series stick, Z-Wave 7.17, 108 node ids, running Indigo 2025.2's live network backup from the plugin menu inside Indigo, 16 s, byte-identical to a terminal run; Indigo reconnected in 2 s; restore not exercised on the production stick

Version bumped to 1.1.0; README, SPEC and release notes updated. I don't have a 500-series stick, so the Gen5 path was covered by the existing tests only — worth one run on yours before release.

🤖 Generated with Claude Code

The plugin now images 700 and 800 series sticks (Zooz ZST10 700 and ZST39,
Aeotec Z-Stick 7 and Z-Stick 10 Pro, Silicon Labs UZB-7 and similar) as well
as the 500 series it already handled. The 500-series code paths are unchanged.

controller_image.py
- Nvm700: NVM3 open/read/write/close over NVMOperations 0x2E, or
  ExtendedNVMOperations 0x3D (32-bit offsets) when the stick advertises it in
  its GetSerialApiCapabilities function bitmask.
- begin/end_nvm_session_700: radio off (0x10) and watchdog stopped (0xD3)
  before any NVM access; soft reset afterwards, waiting for the unsolicited
  SerialAPIStarted (0x0A). Mirrors zwave-js backupNVMRaw700, whose serial
  frames were captured from Z-Wave JS UI and matched byte for byte.
- dump_nvm_700 / write_nvm_700: probe with a 255-byte read, take the chunk
  the stick answers with (64 on all three sticks tested), fall back to 48 on
  an empty probe. The final write, the one reaching the NVM size, is answered
  "end of file" and not performed by the firmware; it is reported and left
  alone, never split and retried: a one-byte write there hung a ZST39 until
  it was replugged, with the watchdog stopped so it could not recover itself.
- compare_images_700: a 700/800 read-back is compared everywhere the image
  holds data; differences inside the declined tail or where the image is
  erased (0xFF) are the firmware's own housekeeping, anything else is a real
  mismatch. Byte-identical read-back is not achievable on NVM3.
- read_identity: node ids parsed by payload length (16-bit node id mode, which
  zwave-js sets and which survives a soft reset), the advertised function
  list, and the full SDK version from GetProtocolVersion (0x09); an extra
  restore guard on that version. Known models: ZST39 LR, Z-Stick 10 Pro and
  the Silicon Labs 700 reference ids.
- Two fixes found in live use: a backup folder pasted with quotes is cleaned
  up (a relative path put the images inside the plugin bundle), and the Home
  ID check considers every network Indigo's Z-Wave prefs remember rather than
  the first one found (a real prefs file held five).

plugin.py
- _with_stick opens a 700/800 session instead of refusing, and always sends
  the closing reset, on failure too, so a stick is never left with its radio
  off. Backup and verify pick the generation; verify tolerates additions in
  free space and compares the node table with the sidecar's.
- _restore_body_700: write, reset, read back at once (no unplug on this
  generation), verify with compare_images_700 plus Home ID and node table.

tests
- FakeStick700 with profiles from the captured ZST39 and 700-stick payloads:
  declined tail, the hang on a tail write (asserted never triggered), 16-bit
  node ids, the probe fallback, housekeeping appended on reset, a stick that
  drops writes. 27 new tests; tests/run.sh green (96).

Tested live 14-09-2026: Zooz ZST39 LR (Z-Wave 7.24) and Aeotec Z-Stick 10 Pro
(Z-Wave 7.23), each backup/verify/restore/verify; a Silicon Labs 700 stick
(Z-Wave 7.17, 108 node ids) running Indigo 2025.2's live network, backup from
the plugin menu inside Indigo, 16 s, byte-identical to a terminal run and to a
Z-Wave JS UI backup of the same stick.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Highsteads
Highsteads merged commit d0845bf into Highsteads:main Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants