Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions governance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ instance file does not exist yet, an invoke creates it - and any
`*-values.yaml` file IS an Argo application, so point actions only at
instances this deployment actually runs.

An action that switches a component off also switches off the engine's secret
group for it (`secrets.<group>.enabled`, as `hunts-pause` does), and its
counterpart switches both back on. The component's Secret is only minted while
it is on, and the engine pod waits on any Secret it still reads.

Authoring a new action: validate first with
`POST /api/v1/governance/admin/actions/validate` (returns every violation and
the would-be diff without committing), then define via
Expand Down
11 changes: 9 additions & 2 deletions governance/actions/hunts-pause.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,19 @@
name: hunts-pause
description: >-
Stop the hunt runner (huntRunner.enabled=false on the engine instance) to
shed ClickHouse load during an incident. Checkpoint watermarks are advanced
only on committed runs, so paused hunts resume with no gap. Revert with
shed ClickHouse load during an incident. The engine's hunt-runner secret
group goes off with it (secrets.hunt-runner.enabled=false), because the
runner's password Secret is only minted while the runner is on and the
engine pod would wait on it. Checkpoint watermarks are advanced only on
committed runs, so paused hunts resume with no gap. Revert with
hunts-resume.
required_action: action:invoke:hunts-pause
changes:
- cls: helmvars
name: dfe-engine-default-values
path: huntRunner.enabled
value: false
- cls: helmvars
name: dfe-engine-default-values
path: secrets.hunt-runner.enabled
value: false
9 changes: 7 additions & 2 deletions governance/actions/hunts-resume.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
name: hunts-resume
description: >-
Restart the hunt runner after a hunts-pause. Hunts resume from their last
committed checkpoint watermark - no window is lost.
Restart the hunt runner after a hunts-pause, and put the engine's hunt-runner
secret group back so the engine adopts the runner's password again. Hunts
resume from their last committed checkpoint watermark - no window is lost.
required_action: action:invoke:hunts-resume
changes:
- cls: helmvars
name: dfe-engine-default-values
path: huntRunner.enabled
value: true
- cls: helmvars
name: dfe-engine-default-values
path: secrets.hunt-runner.enabled
value: true
Loading