Skip to content

feat(cloudformation): Add DevOps Agent alarm investigations template - #38

Open
LorenzoBoccaccia wants to merge 1 commit into
aws:mainfrom
LorenzoBoccaccia:add-alarm-investigations-cfn
Open

feat(cloudformation): Add DevOps Agent alarm investigations template#38
LorenzoBoccaccia wants to merge 1 commit into
aws:mainfrom
LorenzoBoccaccia:add-alarm-investigations-cfn

Conversation

@LorenzoBoccaccia

@LorenzoBoccaccia LorenzoBoccaccia commented Aug 3, 2026

Copy link
Copy Markdown

Description

Forwards one CloudWatch alarm to a DevOps Agent generic webhook so the alarm opens an
investigation. Also moves the existing skill-policies template into its own directory
and adds a README for each.

No code. An EventBridge rule scoped to a single alarm ARN does the
filtering, its input transformer builds the incident payload, and an API destination
posts it through a connection holding the webhook API key as
Authorization: Bearer <key>. An IAM role grants the rule
events:InvokeApiDestination on that one destination.

incidentId is the EventBridge event id, which is stable across retries, so
redeliveries dedup instead of opening duplicate investigations.

API key rather than HMAC because EventBridge connections support only Basic, API key
and OAuth and cannot sign per request; HMAC would need a signing Lambda. The README
covers the trade-off.

Type of change

  • New skill
  • New custom agent
  • Update to an existing skill or agent
  • Documentation or infrastructure change

Testing

Not a skill, so Agent Skill Eval does not apply. Run tests:

  • validate-template: 5 parameters, CAPABILITY_IAM.
  • Deployed twice, one stack per alarm. Each creates four resources: connection, API
    destination, rule, IAM role.
  • Forced an alarm to ALARM. Rule metrics in AWS/Events: TriggeredRules 1,
    InvocationAttempts 1, SuccessfulInvocationAttempts 1, RetryInvocationAttempts 0,
    FailedInvocations 0.
  • The event created an INVESTIGATION task in the Agent Space, IN_PROGRESS, carrying
    the alarm ARN. Its reference.referenceId is the EventBridge event id, which is the
    dedup path.
  • EventBridgeRuleTag: deployed with a value, which tags the rule and the role, and
    empty, which falls back to the stack name.
  • Test resources were tagged and deleted afterwards.

One behaviour to know about: an alarm ARN that already appears in an earlier incident
arrives LINKED to it, whether that incident is open or closed. Only a
first-of-its-kind ARN starts its own investigation. That is DevOps Agent correlation
rather than template behaviour, but it means the same alarm will add to previous investigatios.

@LorenzoBoccaccia
LorenzoBoccaccia force-pushed the add-alarm-investigations-cfn branch from d636396 to 6189121 Compare August 18, 2026 09:08
@LorenzoBoccaccia
LorenzoBoccaccia force-pushed the add-alarm-investigations-cfn branch 2 times, most recently from 90b32a0 to 9239bc0 Compare August 19, 2026 07:51
@LorenzoBoccaccia
LorenzoBoccaccia force-pushed the add-alarm-investigations-cfn branch 2 times, most recently from 883c2ea to a301d6c Compare September 10, 2026 13:54
Forward a single CloudWatch alarm to a DevOps Agent generic webhook so the
alarm opens an investigation. An EventBridge rule scoped to the alarm ARN
matches ALARM state changes, an input transformer builds the incident
payload, and an API destination POSTs it using an API key connection. No
Lambda function is involved.

Move devops-agent-skill-policies.yaml into its own directory and give it a
README so each template is self-documenting, and update the links that
referenced the old path.
@LorenzoBoccaccia
LorenzoBoccaccia force-pushed the add-alarm-investigations-cfn branch from a301d6c to c43418b Compare September 10, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants