Repository navigation
Keep train in its weights folder, fail on yanked crates, correct SECURITY.md - #27
Merged
Merged
Conversation
train asked the app's save_path where to write. That function turns to the per-user data directory when there is no weights folder in the working directory, and train ignored a failure to make that folder. So when the folder could not be made (a file of that name in the way, or a working directory it can't write to), train wrote over the weights saved from the app, or wrote nothing where no per-user folder existed. It printed no error either way and went on to report a saved checkpoint. Where it had written nothing, it then scored whichever other copy of the file loaded. train now writes only to weights/ in the working directory, as SECURITY.md and the README say it does. It makes the folder before it trains and stops with the error if it can't, or if a file in it can't be written. Two tests run it, one with a file in the folder's place and one with a folder in a weights file's place. Both require the error, and the first requires untouched per-user weights; each fails without the fix. Making the folder first would leave an empty one behind when the layout named is not one train trains, and the app saves into such a folder when started there. train now stops with an error at such a name, where it used to do nothing and report success, and a third test requires that and no folder. Signed-off-by: Jim Schreckengast <schreck@gmail.com>
A Hybrid weights file names its base tuning by number, 0 to 47. A larger number, which only a damaged or edited file can hold, is read as the last tuning and the file still loads. No document said so and no test covered it. docs/architecture.md now says it where the loader is described, and tests/weights_files.rs requires it. Signed-off-by: Jim Schreckengast <schreck@gmail.com>
deny.toml only warned about a locked version that had been yanked, in the log of a job that passed. That is how async-executor 1.14.0 stayed in the lock file for two weeks after it was yanked for a use-after-free. A yanked version now fails the check, and the comment says how to record one that can't be updated yet. The release workflow is unchanged: it checks licenses, bans and sources, not advisories, and relies on the pull request's check. Dependabot's weekly updates cover the dependencies named in Cargo.toml, and its security updates reach the rest of the lock file only where GitHub has an advisory, which a yank alone is not. docs/architecture.md and docs/assurance-case.md now say that instead of "the Cargo dependencies". Signed-off-by: Jim Schreckengast <schreck@gmail.com>
Statements in SECURITY.md and docs/assurance-case.md said more than is so. "Only its own files" named trained weights as what the app reads. The desktop app also reads the fonts installed on the system and, on Linux, desktop settings such as the mouse-pointer theme, and loads the system's windowing and graphics libraries. SECURITY.md says so now. The first trust boundary of the assurance case names these as input that the dependencies parse, and its remaining risks name the font parsers. "The app runs no commands" holds for the app's own code. The window libraries start the default browser for the link in the About window and, on Linux under Wayland, three desktop tools for the title bar. The assurance case now says which, and that nothing from a weights file is part of what they run. "Every release file is signed" suggested a signature on each file. Each package comes with a signed build-provenance attestation; the macOS disk image and app and the Windows executable and installer also carry a platform signature, and the Linux packages do not. SECURITY.md, the assurance case, the README and packaging/README.md now put it that way. Signed-off-by: Jim Schreckengast <schreck@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Five older issues that the review of #26 turned up, none caused by it. One commit each, except the last, which covers two.
trainwrote outside itsweightsfolder. It asked the app'ssave_pathwhere to write, which turns to the per-user data directory when there is noweightsfolder in the working directory, and it ignored a failure to make that folder. So when the folder could not be made,trainwrote over the weights saved from the app, or wrote nothing, with no error either way. It now writes only toweights/in the working directory, makes the folder before it trains, and stops with the error if it can't make the folder or write a file. It also stops with an error when no signalized layout has the name it is given, where it used to do nothing and report success. Three tests cover this; the first two fail without the fix.docs/architecture.mdandtests/weights_files.rsnow do.deny.tomlnow hasyanked = "deny", so a locked version that has been yanked fails the required check instead of printing a warning in a passing job, which is howasync-executor1.14.0 stayed in the lock file after it was yanked. The cost: an upstream yank will fail that check on every pull request until the lock file is updated or the version is recorded indeny.tomlwith a reason. The release workflow is unchanged; it checks licenses, bans and sources, not advisories. The documents now say what Dependabot covers: weekly updates for the dependencies named inCargo.toml, and security updates for a locked version that GitHub has an advisory for.SECURITY.mdanddocs/assurance-case.mdthat said more than is so. The desktop app also reads installed fonts and, on Linux, desktop settings such as the mouse-pointer theme. The window libraries it depends on start the default browser for the link in the About window and, on Linux under Wayland, three desktop tools for the title bar. Not every release file carries a platform signature: each package has a signed build-provenance attestation, and the macOS disk image and app and the Windows executable and installer are also code-signed. The README andpackaging/README.mdsaid "signed" of every package too and now match.Traffic and results are unchanged: the only source file that changes is
src/bin/train.rs, in where it saves and when it stops.Checklist
cargo test --locked --releasepasses (zero vehicle overlaps)cargo fmt --allcargo clippy --locked --release --all-targets -- -D warningsis clean (and forwasm32-unknown-unknownif the app changed)THIRD_PARTY_LICENSES.mdis regenerated (none)git commit -s) under the Developer Certificate of OriginNot tested before opening: Linux and Windows. The tests (112 pass), Clippy, the crate package,
cargo deny --locked checkandpackaging/licenses.sh --checkran on macOS with Rust 1.100. The three newtraintests depend on how each platform reports a file where a folder is expected, and they check only the part of the message that is the same everywhere. What the documents now say about Linux (the pointer theme, the programs the window libraries start) comes from reading the dependencies' source, not from running the app there.By submitting this pull request, I agree that my contribution is dual licensed under MIT OR Apache-2.0, as described in CONTRIBUTING.md.