Update to Litestream 0.5.17 - #78
Open
cole-robertson wants to merge 2 commits into
Open
Conversation
Commands.run joined argv into one shell string, ignored the exit status
and discarded stderr, so a failing command returned "" (or [] after
table parsing) and looked like success. Arguments with spaces broke, and
argv reached the shell unescaped.
The runner now uses Open3.popen3 with an argv array, raises
CommandFailedException with the exit status and stderr on failure, and
takes an explicit output mode: :table (the existing header/rows
parsing), :raw, or :json when a caller passes json: true (Litestream
>= 0.5). In JSON mode the two opt-in restore skips, which print one
logfmt line on stdout with exit 0, come back as {"skipped" => true,
"message" => ...} so callers can tell "did nothing" from data.
timeout: runs the command in its own process group and TERMs then KILLs
it on expiry, raising CommandTimeoutException with the child reaped.
The LITESTREAM_INSTALL_DIR note prints once per process.
The bundled binary moves from 0.3.13 to 0.5.17. Release tarballs are pinned by sha256 in Upstream::CHECKSUMS and verified in memory before extraction; a sidecar exe/<platform>/litestream.sha256 records what was packaged so a cached binary from an earlier version is re-downloaded instead of shipped. The darwin zip assets are gone upstream, so rubyzip goes too. Litestream 0.5 replaced generations, snapshots and wal with TXID-numbered LTX files. Commands gains ltx and status; the three removed methods and rake tasks stay for one release as tombstones that explain the change. The dashboard reads status (local, no daemon) and ltx (replica) through the same commands, isolates a command failure to the one database, and lets anything else raise. Generator template, dummy config and README use the 0.5 shape: one replica: per database, root snapshot: block.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second of two, stacked on #76 (the runner). Depends on #76: this branch contains that commit too, so review the second commit here, or wait for #76 to merge and this diff becomes just the bump. Contains only the version bump and what it forces; the dashboard redesign is a later PR, and so is IPC support, which #73 already implements and which should land on top of this as an optional enrichment rather than the only path.
What changes
Upstream::CHECKSUMS), verified in memory before extraction. A sidecarexe/<platform>/litestream.sha256records what was packaged, so a cached binary from an earlier version is re-downloaded rather than shipped. No zip assets upstream any more, sorubyzipis dropped.generations,snapshots,walare gone in 0.5;ltxandstatusreplace them.statusexpands a relative database path: Litestream 0.5 filtersstatusby the absolute path whileltxandrestorematch the config string, so without this the documented--database=storage/production.sqlite3form printed an empty table. The three removed methods and rake tasks remain for one release as tombstones that raise with a pointer toltx.status -json(local, no daemon) andltx -level all -json(replica). A command failure is isolated to that database; anything else raises. Process detection via systemctl/ps is unchanged and still matches a 0.5 daemon. This deliberately does not use the 0.5 IPC socket: it is off by default and only reachable from the process tree runningreplicate, so on Kamal or Docker with a separate replicate role a socket-based dashboard shows nothing. The socket does expose better data where it is reachable (a real last-sync timestamp, and a daemon-confirmed replica TXID fromsync -wait), which is what Litestream v0.5.9 update #73 builds on. The two compose well: keep these CLI calls as the baseline that always works, and read the socket to enrich them when the config enables it and the file exists.replica:per database, root-levelsnapshot:block. README's restore section lists the 0.5 flags; an "Upgrading from 0.3" section covers the config edit, thedatabaseskey rename (replicas→replica), the tombstones, and rollback.Not touched: Puma plugin, engine, VerificationJob,
executable, auth, version.rb (left for the release commit).Verified
Fresh Rails 8 app consuming the packaged native gem (bundled binary, no override): install generator, Puma plugin replicating to a local replica, dashboard over HTTP with and without basic auth and with the daemon stopped, the restore button, every rake task including the tombstones, and
VerificationJob.85 tests, 0 failures (the 39 tests for the removed commands are replaced by
ltx/status/tombstone tests); standardrb clean; fork CI green.Native Gems workflow on the fork: all six platform packages build; linux-x86_64, darwin-arm64 and plain-ruby installs pass. (
darwin-x86_64-installnever starts because the workflow asks for the retiredmacos-13runner; same on upstream.)rake gem:x86_64-linuxfrom a clean tree: download, checksumOK, gem built; installed into a fresh, isolated GEM_HOME,litestream version→ 0.5.17. Planting a wrong sidecar checksum triggers a re-download.Real 0.5.17 binary against a replica holding both 0.3
generations/and 0.5ltx/history:statusandltxin JSON and table modes, the tombstones, andLitestream.databasesin the dummy app returning status plus 16 LTX rows.Running in production at Rebulk (Nightrail Cloud): the readiness probe (
status+ltx, six databases, every five minutes) and the weekly restore drill go through this gem.