Skip to content

Feat/incident rca app - #538

Open
toasterman234 wants to merge 5 commits into
nerdsane:mainfrom
toasterman234:feat/incident-rca-app
Open

toasterman234 wants to merge 5 commits into
nerdsane:mainfrom
toasterman234:feat/incident-rca-app

Conversation

@toasterman234

@toasterman234 toasterman234 commented Oct 22, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 0/5

This PR needs fixes before merging because sensor runs can lose detections and seed retries can leave historical records unfinished.

Fix All in Claude CodeFindings

  1. P1 Later incidents are missed ▶
  2. P1 Overlapping runs erase detections ▶
  3. P1 Seed retries skip unfinished records ▶
  4. P1 Recurrences reuse old verification ▶
  5. P1 Historical incidents never load ▶
  6. P1 Empty inputs pass required checks ▶
  7. P1 Proposing a fix creates nothing ▶
  8. P1 Verification accepts unfinished fixes ▶
  9. P2 Selected server is ignored ▶
  10. P2 App violates kernel-only rule ▶
Fix with agent prompt
### Issue 1
os-apps/incident-rca/sensor/file_incident.py:44-46
`file_incident.py` searches only the first page of incidents. The server returns 100 rows by default. When a matching incident is on a later page, the sensor creates another incident instead of reopening or reusing it. `seed_loader.py` has the same problem when checking existing titles.

Follow `@odata.nextLink` before deciding that no match exists.

### Issue 2
os-apps/incident-rca/sensor/tailnet-sensor.py:102-104
If sensor runs overlap during an outage, they can erase undelivered detections. Each run reads the shared queue, keeps its own copy while SSH requests run, then replaces the whole file. If two runs read an empty queue, the second write drops the first run's detection.

Lock the queue across updates and save changes without exposing a partly written file.

### Issue 3
os-apps/incident-rca/seeds/seed_loader.py:63-65
Once creation succeeds, a later action failure leaves that incident unfinished. The next run skips it solely because its title exists, so retrying cannot finish the load. The loader also exits successfully after reporting partial loads.

Resume existing unfinished seeds from their current state and return a failure when any seed fails.

### Issue 4
os-apps/incident-rca/incident.ioa.toml:undefined-127
`Reopen` keeps every flag from the previous closure. After a recurrence, an investigator can run `BeginInvestigation → CompleteDocumentation → StateHypothesis → ProposeFix → BeginVerification → Close` without attaching new evidence, reviewing prior incidents again, or recording new verification. Reset the investigation and verification flags when reopening, while keeping the previous record as history.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 5
os-apps/incident-rca/seeds/tunnel-incidents.json:undefined-1
The installer never reads `seeds/tunnel-incidents.json`. It only loads `seed-data/*.toml` files containing `[[instance]]` blocks, so installing this app creates none of the eight historical incidents promised in `APP.md`. Convert the corpus to the supported seed format and include the actions needed to reach `Closed`.

### Issue 6
os-apps/incident-rca/incident.ioa.toml:83-85
The recording actions set their completion flags even when the caller supplies `{}` or empty strings. Declaring `params` and CSDL `Nullable="false"` does not require content here: the runtime accepts inputs without constraints. An incident can therefore close without evidence or a hypothesis, and a postmortem can enter review without root-cause text.

Add nonempty constraints to the evidence, prior-review, hypothesis, and verification inputs. Apply the same check to `AddRootCause` in the postmortem spec.

### Issue 7
os-apps/incident-rca/incident.ioa.toml:91-96
`ProposeFix` only changes the incident's status. It has no creation effect or trigger, so it never creates the `Remediation` promised by its hint and `APP.md`. Callers reach `FixProposed` with nothing available for a human to approve. Add an explicit step that creates and links the proposed remediation.

### Issue 8
os-apps/incident-rca/incident.ioa.toml:99-104
`BeginVerification` accepts any `RemediationRef` without checking the referenced remediation. For example, `{"RemediationRef":"not-real"}` moves an incident from `FixProposed` to `Verifying`; a failed or rejected remediation works too. The investigator can then advance to closure without a successful fix. Require a remediation linked to this incident that has reached `Succeeded` before allowing this transition.

### Issue 9
os-apps/incident-rca/sensor/tailnet-sensor.py:65-68
`--base` is accepted and passed to `file_incident`, but that function ignores it and always sends detections to the hardcoded Mac over SSH. Someone selecting another server still files to the original deployment, or queues failures if that Mac is unavailable.

Honor the selected destination or remove the misleading option.

### Issue 10
os-apps/incident-rca/app.toml:1-3
This adds an application-specific incident workflow to the kernel repository. The repository guide explicitly requires “Kernel code only” and says app and agent logic belongs in `temperpaw`. The incident state machine and Cedar policies are app behavior, not kernel support. Put the new app bundle in its owning repository before merging to satisfy that requirement.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds an incident investigation workflow with human-approved fixes and postmortems.

Follow-up changes:

  • Tunnel failures become incidents that need evidence before closure.
  • Fix proposals need human approval before they can be carried out.
  • Postmortems move from a root-cause draft to a published record.
  • Tunnel faults reach the incident app even when the tunnel is down.
  • Observe can be built and run as a production container.
  • Observe pages no longer include empty update-time placeholders.

New findings cover incomplete list reads, lost queue updates, unfinished seed retries, and an ignored server option.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Tunnel classifier] --> B{Fault detected?}
  B -->|Yes| C[Detection plus queued items]
  B -->|No| D[Queued items]
  C --> E[SSH to Mac helper]
  D --> E
  E --> F{Delivery succeeds?}
  F -->|No| G[Keep detection in local queue]
  F -->|Yes| H[List incidents]
  H --> I{Matching incident?}
  I -->|Open| J[Reuse incident]
  I -->|Closed| K[Reopen incident]
  I -->|None| L[Create incident]
Loading

Reviews (2) · Last reviewed commit: "As-built updates: operator identity in C..." · Reviewed by Greptile

kind = "input"
from = ["Closed"]
to = "Reopened"
effect = ["recurrence_count += 1"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Recurrences reuse old verification

Reopen keeps every flag from the previous closure. After a recurrence, an investigator can run BeginInvestigation → CompleteDocumentation → StateHypothesis → ProposeFix → BeginVerification → Close without attaching new evidence, reviewing prior incidents again, or recording new verification. Reset the investigation and verification flags when reopening, while keeping the previous record as history.

Knowledge Base Used: Execution semantics and action evaluation

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/incident.ioa.toml
Line: 127

Comment:
**Recurrences reuse old verification**

`Reopen` keeps every flag from the previous closure. After a recurrence, an investigator can run `BeginInvestigation → CompleteDocumentation → StateHypothesis → ProposeFix → BeginVerification → Close` without attaching new evidence, reviewing prior incidents again, or recording new verification. Reset the investigation and verification flags when reopening, while keeping the previous record as history.

**Knowledge Base Used:** [Execution semantics and action evaluation](https://app.greptile.com/arni-labs/-/custom-context/knowledge-base/nerdsane/temper/-/docs/execution-semantics.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code Fix in Codex Fix in Cursor

@@ -0,0 +1 @@
[{"detected_at":"2026-10-07T00:00:00-05:00","evidence_refs":["memory/2026-10-21.md (incident pattern summary)"],"failure_mode":"ENROLLMENT_DOWN","root_cause_hypothesis":"VM-side tailscaled lost enrollment/connection state; `tailscale status` reported Not connected.","seed_id":"tunnel-2026-10-07","source_signal":"af-obsidian-icloud-sync failure + manual check","title":"Tailscale tunnel down — VM enrollment dropped","verification_evidence":"`tailscale up` restored the tunnel; missed sync completed green 2026-10-07 15:05 CDT and subsequent runs stayed green.","why_prior_fixes_failed":"First recorded incident of this mode; no prior fix."},{"detected_at":"2026-10-08T00:00:00-05:00","evidence_refs":["memory/2026-10-14.md (flap pattern summary)"],"failure_mode":"TUNNEL_DOWN","root_cause_hypothesis":"Unknown. `tailscale status` claimed Connected while the egress proxy refused CONNECT; jobs held last-good state and backfilled on recovery. A separate same-day dropout was restored by `tailscale up`.","seed_id":"tunnel-2026-10-08","source_signal":"captures bundle/projector failures","title":"Tailscale tunnel outage (~12h), capture readers failed, self-recovered","verification_evidence":"Self-recovered; healthy runs backfilled with no captures lost (verified in the Oct 14 pattern summary).","why_prior_fixes_failed":"`tailscale up` treats enrollment loss; this mode presents as Connected, so the enrollment fix does not address it."},{"detected_at":"2026-10-10T15:29:00-05:00","evidence_refs":["memory/2026-10-10.md","memory/2026-10-21.md (incident pattern summary)"],"failure_mode":"TUNNEL_DOWN","root_cause_hypothesis":"UNKNOWN — operator fix not recorded. VM-side state: `tailscale status` and `tailscale up` both claimed Connected; the port-3130 proxy accepted TCP but closed CONNECT without a response. Ben fixed it from his side at ~15:37 CDT ('I put some elbow grease on it'); Astro asked what the fix was and no answer is on record.","seed_id":"tunnel-2026-10-10","source_signal":"af-obsidian-icloud-sync failure (15:32 CDT run)","title":"Tailscale tunnel down — proxy closed CONNECTs; operator fixed from Mac side","verification_evidence":"Sync ran green immediately after the operator fix (mirrored 15:38 CDT); subsequent runs green.","why_prior_fixes_failed":"The VM-side fix (`tailscale up`) did not restore the data plane; the effective fix was Mac-side and is undocumented — which is itself the finding this corpus exists to prevent."},{"detected_at":"2026-10-11T07:32:00-05:00","evidence_refs":["memory/2026-10-11.md"],"failure_mode":"TUNNEL_DOWN","root_cause_hypothesis":"Transient proxy CONNECT failure; the sync itself had completed and a retry read succeeded. Possibly the same underlying instability as 2026-10-08/10, unproven.","seed_id":"tunnel-2026-10-11","source_signal":"af-obsidian-icloud-sync run diagnostics (07:32 CDT)","title":"Transient tunnel blip during log inspection","verification_evidence":"Subsequent reads in the same run confirmed the 07:32:22 sync entry and no FAILED lines.","why_prior_fixes_failed":"No fix applied; recovered before intervention."},{"detected_at":"2026-10-14T12:32:00-05:00","evidence_refs":["memory/2026-10-14.md"],"failure_mode":"TUNNEL_DOWN","root_cause_hypothesis":"Proxy closed connection during CONNECT on both the sync attempt and the log tail; VM-side `tailscale up` restored service this time. Same-day failures at 04:09 and 08:54 self-recovered.","seed_id":"tunnel-2026-10-14","source_signal":"af-obsidian-icloud-sync failure (12:32 CDT run)","title":"Tailscale tunnel down at 12:32 run; `tailscale up` restored","verification_evidence":"Missed sync ran green at 12:34 CDT (mirrored to iCloud); Ben notified.","why_prior_fixes_failed":"Recurrence of the 10-08/10-10 mode; no structural fix has ever been applied — recoveries only."},{"detected_at":"2026-10-20T16:17:00-05:00","evidence_refs":["memory/2026-10-20.md","memory/2026-10-21.md (incident pattern summary)"],"failure_mode":"TUNNEL_DOWN","root_cause_hypothesis":"Proxy closed CONNECTs while `tailscale up` claimed Connected — the 10-10 presentation, but this time it self-recovered without the operator's Mac-side fix.","seed_id":"tunnel-2026-10-20-a","source_signal":"af-obsidian-icloud-sync failure (16:17 CDT run)","title":"Tailscale tunnel down at 16:17 run; self-recovered by 20:02","verification_evidence":"The 20:02 run connected and completed green (mirrored 20:04 CDT); Ben sent the correction.","why_prior_fixes_failed":"Recurrence; still no structural fix. The self-recovery weakens the 'Mac-side fault' theory from 10-10 and is recorded as such."},{"detected_at":"2026-10-20T20:02:00-05:00","evidence_refs":["memory/2026-10-20.md"],"failure_mode":"TUNNEL_DOWN","root_cause_hypothesis":"Tail of the 16:17 outage: first attempt failed with proxy-closed CONNECT, immediate retry succeeded.","seed_id":"tunnel-2026-10-20-b","source_signal":"af-obsidian-icloud-sync (20:02 CDT run, first SSH attempt)","title":"Transient tunnel failure on first attempt of 20:02 run","verification_evidence":"Same run completed: mirrored to iCloud 20:04 CDT.","why_prior_fixes_failed":"No fix applied; recovered on retry."},{"detected_at":"2026-10-22T15:32:00-05:00","evidence_refs":["memory/2026-10-22.md"],"failure_mode":"TUNNEL_DOWN","root_cause_hypothesis":"`tailscale up` reported 'Tailscale stopped sending network configuration. Reconnecting.' while the proxy still closed CONNECTs; the Mac was reachable again by ~15:43 CDT with no intervention. Recurrence count for this mode is now high enough that 'transient' is no longer a satisfying classification.","seed_id":"tunnel-2026-10-22","source_signal":"af-obsidian-icloud-sync failure (15:32 CDT run)","title":"Tailscale tunnel down at 15:32 run; self-recovered within ~10 minutes","verification_evidence":"Mac reachable at ~15:43 CDT (SSH recon for the incident-rca build); sync runs resume on schedule.","why_prior_fixes_failed":"Every prior recovery was either enrollment repair, an undocumented operator fix, or self-recovery. Nothing has addressed the proxy/data-plane instability itself."}] No newline at end of file

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Historical incidents never load

The installer never reads seeds/tunnel-incidents.json. It only loads seed-data/*.toml files containing [[instance]] blocks, so installing this app creates none of the eight historical incidents promised in APP.md. Convert the corpus to the supported seed format and include the actions needed to reach Closed.

Knowledge Base Used: Platform lifecycle and deployment

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/seeds/tunnel-incidents.json
Line: 1

Comment:
**Historical incidents never load**

The installer never reads `seeds/tunnel-incidents.json`. It only loads `seed-data/*.toml` files containing `[[instance]]` blocks, so installing this app creates none of the eight historical incidents promised in `APP.md`. Convert the corpus to the supported seed format and include the actions needed to reach `Closed`.

**Knowledge Base Used:** [Platform lifecycle and deployment](https://app.greptile.com/arni-labs/-/custom-context/knowledge-base/nerdsane/temper/-/docs/platform-lifecycle.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

Comment on lines +83 to +85
guard = "prior_incidents_reviewed"
effect = ["has_root_cause_hypothesis = true"]
params = ["RootCauseHypothesis", "WhyPriorFixesFailed"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Empty inputs pass required checks

The recording actions set their completion flags even when the caller supplies {} or empty strings. Declaring params and CSDL Nullable="false" does not require content here: the runtime accepts inputs without constraints. An incident can therefore close without evidence or a hypothesis, and a postmortem can enter review without root-cause text.

Add nonempty constraints to the evidence, prior-review, hypothesis, and verification inputs. Apply the same check to AddRootCause in the postmortem spec.

Knowledge Base Used:

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/incident.ioa.toml
Line: 83-85

Comment:
**Empty inputs pass required checks**

The recording actions set their completion flags even when the caller supplies `{}` or empty strings. Declaring `params` and CSDL `Nullable="false"` does not require content here: the runtime accepts inputs without constraints. An incident can therefore close without evidence or a hypothesis, and a postmortem can enter review without root-cause text.

Add nonempty constraints to the evidence, prior-review, hypothesis, and verification inputs. Apply the same check to `AddRootCause` in the postmortem spec.

**Knowledge Base Used:**
- [Specification authoring and compilation](https://app.greptile.com/arni-labs/-/custom-context/knowledge-base/nerdsane/temper/-/docs/specification-authoring.md)
- [Execution semantics and action evaluation](https://app.greptile.com/arni-labs/-/custom-context/knowledge-base/nerdsane/temper/-/docs/execution-semantics.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

Comment on lines +91 to +96
name = "ProposeFix"
kind = "internal"
from = ["HypothesisStated"]
to = "FixProposed"
guard = "has_root_cause_hypothesis"
hint = "Propose the fix (creates a Remediation in Proposed state). Unreachable without prior review and a hypothesis — that is the point."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Proposing a fix creates nothing

ProposeFix only changes the incident's status. It has no creation effect or trigger, so it never creates the Remediation promised by its hint and APP.md. Callers reach FixProposed with nothing available for a human to approve. Add an explicit step that creates and links the proposed remediation.

Knowledge Base Used: Specification authoring and compilation

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/incident.ioa.toml
Line: 91-96

Comment:
**Proposing a fix creates nothing**

`ProposeFix` only changes the incident's status. It has no creation effect or trigger, so it never creates the `Remediation` promised by its hint and `APP.md`. Callers reach `FixProposed` with nothing available for a human to approve. Add an explicit step that creates and links the proposed remediation.

**Knowledge Base Used:** [Specification authoring and compilation](https://app.greptile.com/arni-labs/-/custom-context/knowledge-base/nerdsane/temper/-/docs/specification-authoring.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

Comment on lines +99 to +104
name = "BeginVerification"
kind = "input"
from = ["FixProposed"]
to = "Verifying"
params = ["RemediationRef"]
hint = "Begin verification after the linked Remediation reaches Succeeded. Verification is against explicit criteria, not vibes."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Verification accepts unfinished fixes

BeginVerification accepts any RemediationRef without checking the referenced remediation. For example, {"RemediationRef":"not-real"} moves an incident from FixProposed to Verifying; a failed or rejected remediation works too. The investigator can then advance to closure without a successful fix. Require a remediation linked to this incident that has reached Succeeded before allowing this transition.

Knowledge Base Used:

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/incident.ioa.toml
Line: 99-104

Comment:
**Verification accepts unfinished fixes**

`BeginVerification` accepts any `RemediationRef` without checking the referenced remediation. For example, `{"RemediationRef":"not-real"}` moves an incident from `FixProposed` to `Verifying`; a failed or rejected remediation works too. The investigator can then advance to closure without a successful fix. Require a remediation linked to this incident that has reached `Succeeded` before allowing this transition.

**Knowledge Base Used:**
- [Specification authoring and compilation](https://app.greptile.com/arni-labs/-/custom-context/knowledge-base/nerdsane/temper/-/docs/specification-authoring.md)
- [Execution semantics and action evaluation](https://app.greptile.com/arni-labs/-/custom-context/knowledge-base/nerdsane/temper/-/docs/execution-semantics.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

Comment on lines +1 to +3
name = "incident-rca"
description = "Governed incident + root-cause-analysis workflow: detected failures are investigated against the full prior-incident corpus, documented, fixed only through human-approved remediations, and verified closed. First deployment: the VM↔Mac mini Tailscale tunnel."
version = "0.1.0"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 App violates kernel-only rule

This adds an application-specific incident workflow to the kernel repository. The repository guide explicitly requires “Kernel code only” and says app and agent logic belongs in temperpaw. The incident state machine and Cedar policies are app behavior, not kernel support. Put the new app bundle in its owning repository before merging to satisfy that requirement.

Context Used: CLAUDE.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/app.toml
Line: 1-3

Comment:
**App violates kernel-only rule**

This adds an application-specific incident workflow to the kernel repository. The repository guide explicitly requires “Kernel code only” and says app and agent logic belongs in `temperpaw`. The incident state machine and Cedar policies are app behavior, not kernel support. Put the new app bundle in its owning repository before merging to satisfy that requirement.

**Context Used:** CLAUDE.md ([source](https://github.com/nerdsane/temper/blob/main/CLAUDE.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code Fix in Codex Fix in Cursor

Comment on lines +44 to +46
_, listing = call(tok, "GET", "/tdata/Incidents")
same_mode = [v for v in listing.get("value", [])
if v.get("fields", {}).get("FailureMode") == item["mode"]]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Later incidents are missed

file_incident.py searches only the first page of incidents. The server returns 100 rows by default. When a matching incident is on a later page, the sensor creates another incident instead of reopening or reusing it. seed_loader.py has the same problem when checking existing titles.

Follow @odata.nextLink before deciding that no match exists.

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/sensor/file_incident.py
Line: 44-46

Comment:
**Later incidents are missed**

`file_incident.py` searches only the first page of incidents. The server returns 100 rows by default. When a matching incident is on a later page, the sensor creates another incident instead of reopening or reusing it. `seed_loader.py` has the same problem when checking existing titles.

Follow `@odata.nextLink` before deciding that no match exists.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

Comment on lines +102 to +104
with open(args.queue, "w") as f:
for item in remaining:
f.write(json.dumps(item) + "\n")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Overlapping runs erase detections

If sensor runs overlap during an outage, they can erase undelivered detections. Each run reads the shared queue, keeps its own copy while SSH requests run, then replaces the whole file. If two runs read an empty queue, the second write drops the first run's detection.

Lock the queue across updates and save changes without exposing a partly written file.

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/sensor/tailnet-sensor.py
Line: 102-104

Comment:
**Overlapping runs erase detections**

If sensor runs overlap during an outage, they can erase undelivered detections. Each run reads the shared queue, keeps its own copy while SSH requests run, then replaces the whole file. If two runs read an empty queue, the second write drops the first run's detection.

Lock the queue across updates and save changes without exposing a partly written file.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

Comment on lines +63 to +65
if seed["title"] in existing_titles:
print(f"skip (exists): {seed['seed_id']}")
continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Seed retries skip unfinished records

Once creation succeeds, a later action failure leaves that incident unfinished. The next run skips it solely because its title exists, so retrying cannot finish the load. The loader also exits successfully after reporting partial loads.

Resume existing unfinished seeds from their current state and return a failure when any seed fails.

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/seeds/seed_loader.py
Line: 63-65

Comment:
**Seed retries skip unfinished records**

Once creation succeeds, a later action failure leaves that incident unfinished. The next run skips it solely because its title exists, so retrying cannot finish the load. The loader also exits successfully after reporting partial loads.

Resume existing unfinished seeds from their current state and return a failure when any seed fails.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

Comment on lines +65 to +68
proc = subprocess.run(
[os.path.expanduser("~/workspace/bin/mac-ssh.sh"), "bencharney@100.71.118.10",
f"echo '{payload}' | base64 -d | python3 ~/temper-server/file_incident.py"],
capture_output=True, text=True, timeout=120)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Selected server is ignored

--base is accepted and passed to file_incident, but that function ignores it and always sends detections to the hardcoded Mac over SSH. Someone selecting another server still files to the original deployment, or queues failures if that Mac is unavailable.

Honor the selected destination or remove the misleading option.

Prompt To Fix With AI
This is a comment left during a code review.
Path: os-apps/incident-rca/sensor/tailnet-sensor.py
Line: 65-68

Comment:
**Selected server is ignored**

`--base` is accepted and passed to `file_incident`, but that function ignores it and always sends detections to the hardcoded Mac over SSH. Someone selecting another server still files to the original deployment, or queues failures if that Mac is unavailable.

Honor the selected destination or remove the misleading option.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant