Repository navigation
guard/dlp: prune redacted copies older than a day - #384
Open
NopeCharon wants to merge 1 commit into
Open
NopeCharon wants to merge 1 commit into
NopeCharon wants to merge 1 commit into
Conversation
In redact mode, a read of a file that holds a secret is rewritten to point at a sanitised copy under ~/.solongate/.redacted, named by a hash of the source path. Nothing ever removed those copies: the directory is one per machine, shared by every project, and every distinct file that tripped redact mode added a permanent entry. On a machine that runs agents against many repositories it grew without bound, and it kept whatever the redaction left of each file (everything except the matched spans) long after the read that needed it. dlp.SweepRedacted now removes regular files in that directory whose modification time is more than 24 hours old, and the guard calls it on every hook invocation next to sweepLegacyScratch. The copy is read as soon as the client runs the rewritten call, but a client can hold that call behind a permission prompt first, so the margin is a day rather than seconds. Reading the same file again rewrites its copy in place, which renews its age, so a copy is never swept out from under the call it was just made for. Anything that is not a regular file is left alone, since the guard only ever writes files there. The sweep is one ReadDir per hook call (a single failed syscall when the directory does not exist) and adds no dependency to the decision path. Errors are ignored: a copy that cannot be removed now is tried again on the next call. Fixes solongate#190. Signed-off-by: NopeCharon <115794254+NopeCharon@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #190.
In redact mode, a read of a file that holds a secret is rewritten to point at a sanitised copy under
~/.solongate/.redacted, named by a hash of the source path. Nothing ever removed those copies. The directory is one per machine, shared by every project, and every distinct file that tripped redact mode added a permanent entry. So it grew for the life of the machine and kept whatever the redaction left of each file (everything but the matched spans) long after the read that needed it.Change
packages/guard/dlp/redact.go: newSweepRedacted(), which removes regular files in.redactedwhose modification time is more thanredactedTTL(24 hours) old. The directory path moves into a smallredactedDir()helper thatdlpRedactCopynow uses too.packages/guard/main.go: the guard callsdlp.SweepRedacted()on every hook invocation, right aftersweepLegacyScratch(), so stale copies also go away on machines where redact mode has since been turned off.Why a day: the rewritten call reads the copy as soon as the client runs it, but a client can hold that call behind a permission prompt first, so the margin needs to be much longer than seconds. Reading the same file again rewrites its copy in place, which renews its age, so a copy is never swept out from under the call it was just made for. Anything in the directory that is not a regular file is left alone, since the guard only ever writes files there.
Decision path
This touches
packages/guard. It adds no dependency (onlytimefrom the standard library). The cost per hook call is oneReadDir(a single failed syscall when the directory does not exist) plus anInfo()per entry. Errors are ignored: a copy that cannot be removed now is tried again on the next call. It does not change any allow/deny decision or what an existing policy means.Tests
In
packages/guard/dlp/redact_test.go:TestSweepRedactedWithNoDirectoryIsANoop: no directory means no error, and the sweep does not create one.TestSweepRedactedRemovesOnlyExpiredCopies: a copy backdated past the TTL is removed, a fresh one is kept, and an old subdirectory is left alone.TestSweepRedactedKeepsACopyThatWasJustRewritten: a copy made bydlpRedactCopyand then backdated is renewed by a second read of the same file, and survives the sweep.I ran
go vet ./...(also withGOOS=windows) andgo test -count=1 ./...inpackages/guardlocally on macOS. I did not try this by hand against an agent.