700 and 800 series controllers: backup, restore and verify (1.1.0)

README.md

@@ -1,17 +1,17 @@
 # Z-Wave Controller Backup
 
-**Version:** 1.0.2 | **Author:** CliveS & Claude | **Platform:** Indigo 2025.2 or later, macOS
+**Version:** 1.1.0 | **Author:** CliveS & Claude | **Platform:** Indigo 2025.2 or later, macOS
 
 Takes a complete copy of your Z-Wave USB controller's memory, checks it, keeps it, and can put
 it back. Two clicks and about two minutes, from inside Indigo, on the Mac it already runs on.
 
 ## Why
 
 The stick that runs your Z-Wave network keeps the only copy of what that network is: the
-Home ID, the list of devices, the routes between them. A 500-series stick can lose that list.
-When it does, every device is still on the network and still bound to the same Home ID, but
-the stick no longer knows any of them, and Indigo's only documented remedy is to exclude and
-include every device again. mat on the Indigo forum had that happen to 106 of them.
+Home ID, the list of devices, the routes between them. A stick can lose that list. When it
+does, every device is still on the network and still bound to the same Home ID, but the stick
+no longer knows any of them, and Indigo's only documented remedy is to exclude and include
+every device again. mat on the Indigo forum had that happen to 106 of them.
 
 With a copy of the stick's memory on disk the remedy is a two-minute restore. This plugin
 makes that copy from inside Indigo, needing nothing but the stick you already have.
@@ -51,22 +51,35 @@ is when the stick's contents change. The plugin reminds you.
 
 ## Which controllers
 
-Any 500-series Z-Wave stick plugged into the Indigo Mac: the Aeotec Z-Stick Gen5 and Gen5+,
-the Z-Wave.me UZB, the HomeSeer SmartStick+, the Zooz ZST10 500 and similar. The plugin finds
-the port from Indigo's own Z-Wave settings, so there is nothing to configure.
+Any 500, 700 or 800 series Z-Wave stick plugged into the Indigo Mac. 500 series: the Aeotec
+Z-Stick Gen5 and Gen5+, the Z-Wave.me UZB, the HomeSeer SmartStick+, the Zooz ZST10 500 and
+similar. 700 and 800 series: the Zooz ZST10 700 and ZST39, the Aeotec Z-Stick 7 and Z-Stick
+10 Pro, the Silicon Labs UZB-7 and similar. The plugin finds the port from Indigo's own Z-Wave
+settings, so there is nothing to configure, and works out which generation it is talking to.
 
-Not yet: 700 and 800 series sticks (Z-Stick 7, Z-Stick 10 Pro, ZST39, ZWA-2). Their memory is
-read a different way and the author has none to test against. The plugin recognises one and
-says so rather than guessing. Sticks connected over the network rather than USB are also out.
+The 700 and 800 series keep their memory as a small filesystem rather than a flat block, and
+are read and written through a different Serial API command. Two things follow, both explained
+in the Event Log when they happen: the stick is reset at the end of every visit (that is the
+documented way to leave its memory tidy, and Indigo reconnects to it as normal), and a restore
+does not need the unplug-and-replug a 500-series stick needs. A backup takes about twenty
+seconds.
+
+Sticks connected over the network rather than USB are out.
 
 ## The one rule about restoring
 
 An image only goes back onto the same stick, or another stick of the same model running the
 same software version. It is a raw copy, and the layout of the memory changed between
 Aeotec's software versions (an original Gen5 on 1.01 and a Gen5+ on 1.02 are not
-interchangeable). The plugin checks this and refuses otherwise. Converting an image for a
-different generation of stick is a job for zwave-js, and needs the newer Gen5 software
-first; mat's forum write-up (topic 29135) covers that route.
+interchangeable), and between the Silicon Labs SDK versions a 700 or 800 series stick runs.
+The plugin checks this and refuses otherwise.
+
+Moving a network onto a different generation of stick (500 to 700, 700 to 800) is a
+conversion, not a raw restore, and is a job for zwave-js: its `nvmedit convert` tool, or the
+non-raw Restore NVM in Z-Wave JS UI, takes exactly the `.bin` this plugin writes. On a 500
+series stick that route needs the newer Gen5 software first; mat's forum write-up (topic
+29135) covers it. Two sticks holding the same Home ID must never be plugged in at the same
+time, with or without software driving them.
 
 ## Tested
 
@@ -81,6 +94,24 @@ accept a write to the last 16 bytes of its memory. Those bytes are blank in ever
 so the plugin reads them back instead of forcing the write, confirms they already match, and
 carries on. It is expected, not a fault.
 
+700 and 800 series, live on 14 September 2026 on a second system: a Zooz ZST39 LR (800 series,
+Z-Wave 7.24, ten nodes), an Aeotec Z-Stick 10 Pro (800 series, Z-Wave 7.23, its Z-Wave side;
+the Zigbee side is a separate port the plugin ignores) and a Silicon Labs 700 series stick
+(Z-Wave 7.17, 108 node ids) running Indigo 2025.2's network. Backups matched a Z-Wave JS UI
+backup of the same stick byte for byte, on both of the Serial API commands the plugin uses. A
+restore of an older image onto the ZST39 brought back its older node table, and a restore of
+the current image put it back; the Z-Stick 10 Pro went through backup, verify, restore and
+verify the same way; each restore verified by read-back.
+
+Two things learnt there, and now built in: a 700/800 stick answers the very last write of a
+restore with "end of file" and does not perform it (zwave-js sees the same; those bytes hold
+nothing any of its files refer to), and it must not be asked to write those bytes in smaller
+pieces, because a one-byte write there hung the ZST39 outright until it was replugged. The
+plugin leaves that tail alone and says so. And after any reset the stick may append a few
+bytes of its own housekeeping into free space, so a 700/800 read-back is compared everywhere
+the image holds data rather than byte for byte, and the node table and Home ID are checked
+too.
+
 ## Installation
 
 1. Download the plugin:
@@ -91,8 +122,9 @@ carries on. It is expected, not a fault.
    Backup). It needs no settings.
 
 Images go to `Z-Wave Controller Backups` next to the Indigo folder unless you choose another
-folder in the plugin's Configure dialog. They are about 256 KB each, so keep them all, and
-let your normal backup carry the folder off the Mac.
+folder in the plugin's Configure dialog. They are 256 KB each for a 500 series stick and about
+40 to 48 KB for a 700 or 800 series one, so keep them all, and let your normal backup carry the
+folder off the Mac.
 
 ## If something goes wrong
 
@@ -105,13 +137,25 @@ the thread is the one place to look.
 
 ## Credits
 
+The 700 and 800 series support was contributed by **Autolog**, tested on a ZST39, a Z-Stick 10 Pro
+and the 700 series stick running a second Indigo system, with Claude Opus 5 at the keyboard.
+
 mat's forum write-up of recovering a Gen5 that had lost its node table is what started this,
 and his zwave-js scripts remain the fallback route for anyone without the plugin. The Serial
 API details were checked against the zwave-js project's source (MIT). Nothing from either is
 bundled here; the plugin is plain Python with no dependencies.
 
 ## What's new
 
+**1.1.0** (14-Sep-2026) — 700 and 800 series sticks (Zooz ZST10 700 and ZST39, Aeotec Z-Stick 7
+and Z-Stick 10 Pro, UZB-7 and similar): backup, restore and verify through the NVM3 commands
+those sticks use, tested live on a ZST39 LR and a 700 series stick against Z-Wave JS UI. No
+unplug needed on this generation; the closing reset does the job. Node ids are read correctly
+from a stick that zwave-js has left in 16-bit mode. Show Plugin Info and the device report the
+full SDK version. Two small fixes found on the way: a backup folder pasted with quotes around it
+is cleaned up rather than landing the images inside the plugin bundle, and the Home ID check
+looks at every network Indigo's settings remember instead of the first one it finds.
+
 **1.0.2** (13-Sep-2026) — Show Plugin Info now reports the controller itself: model, Home ID,
 software version, how many nodes the last image holds, which file it is, and whether the network
 has changed since. It previously showed only the folder, port and backup time.
@@ -125,6 +169,6 @@ with read-back verification, verify, one controller device, and the stale-backup
 
 ## Authors & licence
 
-Vibed into existence by **CliveS**, who knew what he wanted, argued until he got it, and tested it on a real house. Typed at inhuman speed by **Claude** (Anthropic), who mostly did as it was told.
+Vibed into existence by **CliveS**, who knew what he wanted, argued until he got it, and tested it on a real house. Typed at inhuman speed by **Claude** (Anthropic), who mostly did as it was told. 700 and 800 series by **Autolog** and Claude, on a second real house.
 
 © 2026 CliveS · [MIT licence](LICENSE) — copy it, fork it, bend it, break it, fix it, ship it. If it breaks, you get to keep both pieces.

SPEC.md

@@ -219,3 +219,57 @@ Python 3.13 stdlib only (termios/fcntl/json/hashlib), no requirements.txt, versi
   request (the stick was mid-reply), so the plugin now pauses two seconds, drains and ACKs
   what the stick still has to say, and backs off on collisions; the Restore dialog's pop-up
   preselects the newest image.
+
+
+## 700 / 800 series — added in v1.1.0 (14-Sep-2026)
+
+Verified live on a Zooz ZST39 LR (800 series, "Z-Wave 7.24", SDK 7.24.2, ids 027A:0004:0610,
+NVM 40960 bytes) and a Silicon Labs 700 series stick (Z-Wave 7.17, SDK 7.17.1, ids
+0000:0004:0004, NVM 49152 bytes, 108 node ids, Indigo 2025.2's live network), against
+zwave-js 15.29 whose backupNVMRaw700 serial frames were captured from Z-Wave JS UI and matched.
+
+**Scope change:** "700/800-series sticks" moves from *Out* to *In*. Conversion between
+generations stays out (zwave-js `nvmedit convert`; documented, not done).
+
+| Use | Function | Notes |
+|---|---|---|
+| full SDK version ("7.17.1") | 0x09 GetProtocolVersion | payload[1..3]; only asked when the bitmask advertises it |
+| function bitmask | 0x07 GetSerialApiCapabilities | bytes 8..39, bit n-1 = function n; decides 0x2E vs 0x3D |
+| radio off / on | 0x10 SetRFReceiveMode | [00] before any NVM access, response [01] = done |
+| watchdog off | 0xD3 StopWatchdog | no response frame; the closing reset starts it again |
+| NVM open/read/write/close | 0x2E NVMOperations | [op, len, offset16]; reply [status, len, offset16, data] |
+| the same, 32-bit offsets | 0x3D ExtendedNVMOperations | used when advertised (800 series, NVM over 64 KB); open reply carries a supported-ops bitmask |
+| reset | 0x08 SoftReset | mandatory after any NVM visit; the stick answers with an unsolicited 0x0A SerialAPIStarted ~110 ms later |
+
+Statuses: 0x00 ok, 0xFF end of file (also ok), 0x01 error, 0x02 operation mismatch, 0x03
+operation interference, 0x04 sub-command not supported. Reads ask for 255 bytes and take the
+chunk that comes back (64 on both sticks); an empty reply to the probe drops the chunk to 48.
+Writes go in the probed chunk. Both sticks answer node ids in 16-bit mode if zwave-js has
+put them there, so MemoryGetId and GetSUCNodeId are parsed by payload length.
+
+**Session:** identity -> radio off -> watchdog off -> open (size) -> close -> [reads / writes]
+-> soft reset -> wait for SerialAPIStarted. The reset is always sent on the way out, on failure
+too, so the stick is never left with its radio off. No replug on restore: the reset is the
+documented step and the read-back follows it at once.
+
+**Restore facts:** the write whose end reaches the NVM size is answered 0xFF and NOT performed
+(zwave-js behaves the same and never checks). Splitting it: 32 bytes at size-64 ok, 32 at
+size-32 refused, 16/8/4/2-byte writes down to size-2 ok, then a 1-byte write at size-2 hung
+the ZST39 (no ACK, soft reset ignored, watchdog stopped, replug needed). Hence: never retry the
+tail in smaller pieces; report it as declined; verify around it.
+
+**Verification:** byte-identical read-back is not achievable on NVM3 — the firmware appends
+housekeeping objects into erased (0xFF) space on restart (74 bytes after one restore) and can
+relocate pages after a power cycle. A 700/800 restore is verified by `compare_images_700`:
+differences are allowed only inside the declined tail and where the image holds 0xFF; anything
+else is "did NOT hold". The Home ID and the node table reported after the reset must equal the
+sidecar's. Verify Last Backup uses the same comparison and says how many housekeeping bytes
+appeared.
+
+**Guards added:** `protocolVersionFull` equal when both sides have it (the library string
+"Z-Wave 7.17" is coarser than the layout). Known models: ZST39 LR (027A:0004:0610) and the
+Silicon Labs 700 reference ids (0000:0004:0004).
+
+**Tests added:** `FakeStick700` with ZST39 and 700 profiles from the captured payloads,
+including the declined tail, the hang on a tail write (asserted never triggered), 16-bit node
+ids, the probe fallback, the housekeeping append on reset, and a stick that drops writes.

Z-Wave Controller Backup.indigoPlugin/Contents/Info.plist

@@ -25,7 +25,7 @@
 		<string>Highsteads</string>
 	</dict>
 	<key>PluginVersion</key>
-	<string>1.0.2</string>
+	<string>1.1.0</string>
 	<key>ServerApiVersion</key>
 	<string>3.0</string>
 </dict>

Z-Wave Controller Backup.indigoPlugin/Contents/Server Plugin/MenuItems.xml

@@ -27,7 +27,7 @@
         <ButtonTitle>Restore</ButtonTitle>
         <ConfigUI>
             <Field id="lbl_warn" type="label">
-                <Label>This writes an image back into the controller's memory. It only goes onto the same model running the same software version as the stick the image came from; the plugin refuses anything else. Afterwards you unplug the stick, plug it back in, and the plugin checks the image took before telling you to switch Z-Wave back on.</Label>
+                <Label>This writes an image back into the controller's memory. It only goes onto the same model running the same software version as the stick the image came from; the plugin refuses anything else. Afterwards the plugin resets the stick (a 500-series stick also needs an unplug and replug, which the Event Log asks for), reads it back to check the image took, and tells you to switch Z-Wave back on.</Label>
             </Field>
             <Field id="imageFile" type="menu">
                 <Label>Image:</Label>

Z-Wave Controller Backup.indigoPlugin/Contents/Server Plugin/PluginConfig.xml

@@ -4,7 +4,7 @@
         <Label>Backs up the memory of your Z-Wave USB controller (Home ID, node table, routes) to a file, and can write it back. Backups run from the plugin menu and need Z-Wave switched off for about two minutes: the plugin waits for you to do that and tells you when to switch it back on. Nothing here needs a password or a key.</Label>
     </Field>
     <Field id="backupFolder" type="textfield">
-        <Label>Backup folder (blank = Z-Wave Controller Backups, next to the Indigo folder):</Label>
+        <Label>Backup folder, full path without quotes (blank = Z-Wave Controller Backups, next to the Indigo folder):</Label>
     </Field>
     <Field id="portOverride" type="textfield">
         <Label>Serial port (blank = the port Indigo's Z-Wave interface uses):</Label>