Skip to content

guard/dlp: prune redacted copies older than a day - #384

Open
NopeCharon wants to merge 1 commit into
solongate:mainfrom
NopeCharon:fix/prune-redacted-copies
Open

NopeCharon wants to merge 1 commit into
solongate:mainfrom
NopeCharon:fix/prune-redacted-copies

Conversation

@NopeCharon

Copy link
Copy Markdown
Contributor

Fixes #190.

In redact mode, a read of a file that holds a secret is rewritten to point at a sanitised copy under ~/.solongate/.redacted, named by a hash of the source path. Nothing ever removed those copies. The directory is one per machine, shared by every project, and every distinct file that tripped redact mode added a permanent entry. So it grew for the life of the machine and kept whatever the redaction left of each file (everything but the matched spans) long after the read that needed it.

Change

  • packages/guard/dlp/redact.go: new SweepRedacted(), which removes regular files in .redacted whose modification time is more than redactedTTL (24 hours) old. The directory path moves into a small redactedDir() helper that dlpRedactCopy now uses too.
  • packages/guard/main.go: the guard calls dlp.SweepRedacted() on every hook invocation, right after sweepLegacyScratch(), so stale copies also go away on machines where redact mode has since been turned off.

Why a day: the rewritten call reads the copy as soon as the client runs it, but a client can hold that call behind a permission prompt first, so the margin needs to be much longer than seconds. Reading the same file again rewrites its copy in place, which renews its age, so a copy is never swept out from under the call it was just made for. Anything in the directory that is not a regular file is left alone, since the guard only ever writes files there.

Decision path

This touches packages/guard. It adds no dependency (only time from the standard library). The cost per hook call is one ReadDir (a single failed syscall when the directory does not exist) plus an Info() per entry. Errors are ignored: a copy that cannot be removed now is tried again on the next call. It does not change any allow/deny decision or what an existing policy means.

Tests

In packages/guard/dlp/redact_test.go:

  • TestSweepRedactedWithNoDirectoryIsANoop: no directory means no error, and the sweep does not create one.
  • TestSweepRedactedRemovesOnlyExpiredCopies: a copy backdated past the TTL is removed, a fresh one is kept, and an old subdirectory is left alone.
  • TestSweepRedactedKeepsACopyThatWasJustRewritten: a copy made by dlpRedactCopy and then backdated is renewed by a second read of the same file, and survives the sweep.

I ran go vet ./... (also with GOOS=windows) and go test -count=1 ./... in packages/guard locally on macOS. I did not try this by hand against an agent.

In redact mode, a read of a file that holds a secret is rewritten to
point at a sanitised copy under ~/.solongate/.redacted, named by a hash
of the source path. Nothing ever removed those copies: the directory is
one per machine, shared by every project, and every distinct file that
tripped redact mode added a permanent entry. On a machine that runs
agents against many repositories it grew without bound, and it kept
whatever the redaction left of each file (everything except the matched
spans) long after the read that needed it.

dlp.SweepRedacted now removes regular files in that directory whose
modification time is more than 24 hours old, and the guard calls it on
every hook invocation next to sweepLegacyScratch. The copy is read as
soon as the client runs the rewritten call, but a client can hold that
call behind a permission prompt first, so the margin is a day rather
than seconds. Reading the same file again rewrites its copy in place,
which renews its age, so a copy is never swept out from under the call
it was just made for. Anything that is not a regular file is left alone,
since the guard only ever writes files there.

The sweep is one ReadDir per hook call (a single failed syscall when
the directory does not exist) and adds no dependency to the decision
path. Errors are ignored: a copy that cannot be removed now is tried
again on the next call.

Fixes solongate#190.

Signed-off-by: NopeCharon <115794254+NopeCharon@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

guard/dlp (redacted-copy files in ~/.solongate/.redacted are never cleaned up)

1 participant