Skip to content

feat(run_task): [tasks.<lang>] env, and run/verbose test filters - #541

Merged
atlas-from-plumb merged 6 commits into
mainfrom
atlas/fix-537-538-run-task-env-filter
Sep 30, 2026
Merged

atlas-from-plumb merged 6 commits into
mainfrom
atlas/fix-537-538-run-task-env-filter

Conversation

@atlas-from-plumb

Copy link
Copy Markdown
Collaborator

Fixes #537
Fixes #538

Defect

#537. A [tasks.<lang>] slot could not set an environment variable. plumb's CI runs make test, which sets GOTMPDIR=$(CURDIR)/.testcache so t.TempDir() lands inside the checkout; run_task test ran go test with the daemon's environment, so temp dirs landed in the system temp dir. A test depending on that difference passed under run_task and failed on CI (#518), and agents fell back to a shell to set the variable.

#538. run_task's target fills one {target} placeholder, in practice a package. There was no way to run one test or a pattern (go test -run 'X|Y', pytest -k, a cargo filter) or to see skipped tests, and mutation_test paid for the whole package on every mutant (~2.5 min for internal/tools).

Design

[tasks.<lang>] env (table of string → string)

  • Applied by RunTaskArgv for run_task and mutation_test's compile and test steps, and by the plumb build|test|… CLI. Entries replace inherited values of the same name. They are in place before the automatic GOWORK=off decision (fix: run worktree hooks and mutation_test in the right tree [PLAN-442] #506), so an explicit GOWORK there is used as is, the same rule an inherited GOWORK follows. A GOENV/PATH set there also informs the decision. PWD is still pinned to the command's directory.
  • {workspace} and {working_dir} expand at run time, not at resolution. TaskCommand carries the templates and a Root, and mutation_test's re-root moves Root with the command, so GOTMPDIR follows a mutant into its worktree. Any other {…} token is rejected at load.
  • GOTMPDIR/TMPDIR directories inside the workspace (checked after symlink resolution) are created, the job make test's $(TESTCACHE) prerequisite does. Without this, every fresh worktree's run_task test would fail before a test ran. Nothing outside the workspace is created.
  • run_task prints env: K=V …. A value is withheld when its name marks it as a credential (…TOKEN, …SECRET, …PASSWORD, …) or internal/redact recognises it. plumb config show has a tasks.<lang> section listing each entry with its provenance.
  • Composition follows [git] env: project over global per name, the same for all three TOML spellings. cloneTasks now deep-copies Env, so a project sub-table cannot write into the base config.

Trust / security policy

  • An env var decides what a command runs (PATH, GOFLAGS=-toolexec=…, LD_LIBRARY_PATH), so a project env is trust-gated like a command. Every entry enters the plumb trust hash as a TaskCommandSpec{Slot: "env.<NAME>"}, which is injective because real slot names cannot contain . and a forged quoted key fails validation. The [TASKS.go] ENV = … spelling is hashed too. A project env makes every slot of the language project-supplied, including the shipped defaults (the working_dir rule).
  • plumb trust lists the entries and adds a warning to keys that steer execution (PATH, GOFLAGS, GOTOOLCHAIN, GIT_*, NODE_OPTIONS, PYTHONPATH, RUSTC_WRAPPER, DYLD_*, LD_LIBRARY_PATH, …).
  • Refused outright, in every layer: LD_PRELOAD, LD_AUDIT, DYLD_INSERT_LIBRARIES, DYLD_FORCE_FLAT_NAMESPACE. They run extra code in every process a command starts, and no build or test needs them. Names must match ^[A-Za-z_][A-Za-z0-9_]*$, and NUL is refused in values. env is classified ClassTrustGated and is not agent-writable.

{run} / {verbose} (#538)

  • New placeholders modelled on {target:<default>}, where an absent value adds nothing: {run} / {run:<flag>} → <flag> <filter>, and {verbose:<flag>} → <flag>. Shipped defaults: go test {verbose:-v} {run:-run} {target:./...}, pytest {verbose:-v} {run:-k} {target:}, cargo test {target:} {run:--}. Rust puts the filter after -- because cargo's one positional is already {target}, and it gets no verbose placeholder because cargo already lists every test. With nothing asked for, all of them build the argv they built before.
  • Reconciliation extends the existing equivalence. A stored go test {target:./...} (the previous default) or go test ./... reconciles to the new default, and the response notes it.
  • The filter is one argv element and no shell sees it. It allows letters, digits, space and ._/:@|^$*+?()[]-, up to 256 characters, and may not start with - or a space, since a bare {run} must not smuggle in -exec=….
  • A run on a command without {run} is refused, quoting the stored command and the placeholder to add (as {target} is). An unplaceable verbose is noted. On composite verify, verbose reaches both steps, while target/run are not applied and a note says so.
  • New arguments: run_task run + verbose, mutation_test test_run (test command only; the compile gate stays unscoped). TaskResolverFn now takes a TaskRequest struct.

plumb's own config. .plumb/config.toml is committed (.gitignore now excludes .plumb/* except config.toml) with [tasks.go.env] GOTMPDIR = "{workspace}/.testcache", matching make test. Heads-up: a project env gates the Go slots, so plumb trust is needed once per plumb checkout before run_task runs Go commands there.

Housekeeping: the mutation_test arguments moved to mutationtest_args.go and the run_task notes to conn_tasks_notes.go to keep files ≤600 lines. tools/list pinned payload is 44,958 / 45,000 bytes: run_task wording was trimmed rather than raising the cap, and the mutation_test description is back under 2,000 chars.

Verification

  • Red on origin/main: the new test files fail to compile against main (TasksConfig.Env, RunTaskArgv(…, env, …), TaskRequest do not exist).
  • Green: GOWORK=off GOTMPDIR=$PWD/.testcache go test ./... -count=1 passes on the whole module, with .plumb/config.toml present as on CI. golangci-lint run ./... (v2.13.2) reports 0 issues. make check-size check-brief check-changelog passes.
  • Mutation (each mutant reverts one piece; all KILLED, with scoped tests, file restored via git checkout):
    • Env: dropped from the trust hash; composition removed; Env clone removed; denylist dropped; unknown placeholder accepted; env decoded as an extra slot; steering keys not flagged; name validation weakened; validation not wired; TaskEnvKeyOf prefix check dropped. Also: env not applied in runArgv; temp dir not created; temp dir created outside the workspace; credential redaction removed; mutation_test step ignoring env; rerootedRoot call removed or keeping the old tree; CLI streamArgv env dropped; project env not gating provenance; env not carried by the resolver.
    • Run/verbose: leading - allowed; test_run dropped; run flag dropped; verbose ignored; run silently dropped without placeholder; previous default not reconciled; target/run swapped; verbose note and composite run note dropped; trust-gated label dropped; go default losing {run}.
    • The first round had two SURVIVED mutants: the rerootedRoot call site and the CLI env. Tests for both were added in follow-up commits, and they were then KILLED.

🤖 Generated with Claude Code

atlas-from-plumb and others added 5 commits October 1, 2026 04:47
A [tasks.<lang>] slot could not set an environment variable, so `run_task
test` could not reproduce plumb's own CI: `make test` sets GOTMPDIR inside
the checkout and `go test` under run_task used the system temp directory, so
a test depending on that passed locally and failed on CI (#537). And
{target} fills one positional, so there was no way to run one test or a
pattern, see skipped tests, or scope a mutant to the tests that should kill
it (#538).

[tasks.<lang>] env sets variables on every command of the language
(run_task, mutation_test's compile and test steps, plumb build|test|...).
They go on top of the inherited environment and before the automatic
GOWORK=off decision, so an explicit GOWORK wins, as an inherited one does.
{workspace} and {working_dir} expand at run time, so they follow a command
mutation_test re-roots into another worktree, and a GOTMPDIR/TMPDIR inside
the workspace is created, as make's $(TESTCACHE) prerequisite does.
run_task reports the entries with credentials redacted, and plumb config
show lists them with provenance.

An env var changes what a command runs as surely as the command does, so a
project's env is trust-gated like one: every entry is in the plumb trust
hash, it makes every slot of the language project-supplied (the working_dir
rule), plumb trust flags keys such as PATH, GOFLAGS or GIT_* that steer
execution, and the loader-injection variables are refused outright.

run_task gains `run` and `verbose`, mutation_test gains `test_run`, filling
new {run:<flag>} and {verbose:<flag>} placeholders that add nothing when not
asked for. The shipped go/python/rust test defaults carry them, and a stored
earlier default reconciles to the new one by the existing equivalence. The
filter is one argv element (no shell), admits | and regexp characters, and
may not start with -. A filter on a command without {run} is refused, as a
target is; a verbose it cannot place is noted.

plumb's own .plumb/config.toml is now committed with GOTMPDIR set the way
`make test` sets it.

Fixes #537
Fixes #538

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plumb-Session: giant-bison
Hand mutation of the first commit left two pieces unpinned: removing the
rerootedRoot call in rerootCommand survived (nothing ran a re-rooted command
with an env), and the `plumb build|test` path's env had no test at all. Each
test now drives the real path and carries a control.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plumb-Session: giant-bison
Dropping the prefix check survived every existing test, because no real slot
contains a dot. The check is still what keeps a trust-hash entry from ever
being read as some other dotted key, so it gets a direct test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plumb-Session: giant-bison
…ng a refusal is about

From the independent review of #541:
- BLOCKING: committing plumb's .plumb/config.toml (GOTMPDIR) made every
  Go run_task in every plumb checkout and worktree refuse until `plumb
  trust`, because a project env marked the shipped defaults
  project-supplied. A GOTMPDIR or TMPDIR set to {workspace} or
  {workspace}/<relative path> moves temporary files and changes neither
  what runs nor where, so on its own it no longer needs trust. It is
  still hashed and applied. Any other key, or a value that can leave
  the workspace, is still gated.
- The refusal now names the setting that made a slot project-supplied
  (an env key, working_dir, or an overridden command) instead of saying
  a shipped `go build ./...` "comes from this project".
- .gitignore: `.plumb/*` was anchored to the root, so .plumb/ state in
  subdirectories stopped being ignored; it is `**/.plumb/*` again, with
  only the root config.toml kept.
- More steering keys are flagged by `plumb trust` (GOENV, GOPROXY/SUMDB,
  CC/CXX, HOME, XDG_CONFIG_HOME, CGO_*, CARGO_TARGET_*, npm config,
  NODE_PATH, PERL5LIB, RUBYLIB, RUSTC_WORKSPACE_WRAPPER, CARGO_HOME).
- A leading '@' is refused in a {run} filter (an argument file to tools
  that read @file), and PrepareTaskEnv creates nothing when there is no
  workspace root.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plumb-Session: velvet-eagle
From the second review round of #541:
- The exemption covered TMPDIR as well. TMPDIR reaches every tool of
  every language, and only GOTMPDIR is needed to mirror CI, so the
  exemption is now GOTMPDIR only (least privilege). A project TMPDIR is
  gated again.
- The project specs come from map iteration, so a project with several
  gated settings could have its refusal name a different one from run
  to run. They are now sorted before the scan.
  TestTaskProvenance_NamesTheSameSettingEveryTime pins both the order
  and that an exempt GOTMPDIR never ends the scan early; the latter
  was only caught 3 times in 10 before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plumb-Session: velvet-eagle

@golimpio golimpio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two independent review rounds. Round 1's blocker (the committed config broke run_task until plumb trust) is fixed with a GOTMPDIR-only exemption and a refusal that names the setting. Round 2 found nothing blocking, and its small findings are folded in. Each change is proven by mutation.

@atlas-from-plumb
atlas-from-plumb enabled auto-merge (rebase) September 30, 2026 19:55

@golimpio golimpio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approve after updating with main (#542).

@atlas-from-plumb
atlas-from-plumb merged commit dd275e1 into main Sep 30, 2026
9 checks passed
atlas-from-plumb added a commit that referenced this pull request Sep 30, 2026
…ng a refusal is about

From the independent review of #541:
- BLOCKING: committing plumb's .plumb/config.toml (GOTMPDIR) made every
  Go run_task in every plumb checkout and worktree refuse until `plumb
  trust`, because a project env marked the shipped defaults
  project-supplied. A GOTMPDIR or TMPDIR set to {workspace} or
  {workspace}/<relative path> moves temporary files and changes neither
  what runs nor where, so on its own it no longer needs trust. It is
  still hashed and applied. Any other key, or a value that can leave
  the workspace, is still gated.
- The refusal now names the setting that made a slot project-supplied
  (an env key, working_dir, or an overridden command) instead of saying
  a shipped `go build ./...` "comes from this project".
- .gitignore: `.plumb/*` was anchored to the root, so .plumb/ state in
  subdirectories stopped being ignored; it is `**/.plumb/*` again, with
  only the root config.toml kept.
- More steering keys are flagged by `plumb trust` (GOENV, GOPROXY/SUMDB,
  CC/CXX, HOME, XDG_CONFIG_HOME, CGO_*, CARGO_TARGET_*, npm config,
  NODE_PATH, PERL5LIB, RUBYLIB, RUSTC_WORKSPACE_WRAPPER, CARGO_HOME).
- A leading '@' is refused in a {run} filter (an argument file to tools
  that read @file), and PrepareTaskEnv creates nothing when there is no
  workspace root.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plumb-Session: velvet-eagle
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants