Skip to content

feat: add aiml-bedrock-security-assessment skill - #78

Open
aditya-vikram-parakala wants to merge 1 commit into
aws:mainfrom
aditya-vikram-parakala:feature/aiml-bedrock-security-assessment
Open

feat: add aiml-bedrock-security-assessment skill#78
aditya-vikram-parakala wants to merge 1 commit into
aws:mainfrom
aditya-vikram-parakala:feature/aiml-bedrock-security-assessment

Conversation

@aditya-vikram-parakala

Copy link
Copy Markdown

Summary

Adds the aiml-bedrock-security-assessment skill: a read-only Amazon Bedrock security posture assessment that runs 33 checks (BR-01..BR-33) across an account, regions, and associated accounts, and produces severity-rated findings plus a consolidated report. The DevOps Agent runs the read-only checks directly; no infrastructure is deployed.

This is the Bedrock domain of a per-domain AI/ML security posture family (scoped down from a broader 174-check proposal per reviewer feedback to phase by domain and prove the determinism + eval model on one domain first). Coverage: guardrail depth (content filters, sensitive-information/PII, contextual grounding, automated reasoning, tiers), KMS encryption (custom/imported models, knowledge bases, invocation logs, batch output), VPC private connectivity, agent and action-group IAM least privilege, agent guardrail association and idle-session TTL, model-invocation logging, CloudTrail, CloudWatch alarms, service-quota throttling, and Inspector Lambda scanning.

Owns vs. defers (boundaries with existing skills)

  • Owns — proactive, point-in-time security misconfiguration posture of Bedrock control-plane resources.
  • Defers to bedrock-adoption-readiness — production-adoption readiness (ZDR/data-retention, quota/capacity headroom, observability maturity, bedrock-mantle).
  • Defers to aiml-access-diagnostics — reactive diagnosis of a specific access denial (authorization-chain tracing).
  • Out of scope — SageMaker, AgentCore (AgentCore observability is owned by agentcore-observability-setup), Responsible AI GRC, OWASP Top 10 for LLM.

Verify vs. prescribe (determinism contract)

Every check carries a Verifiability classification so the report never marks an unread control as Passed:

  • Verifiable (22) — a read-only call returns the exact config; deterministic verdict.
  • Heuristic (10) — readable but inferred (e.g. IAM least-privilege); the verdict cites concrete evidence, and ambiguous/denied reads become N/A, never Passed.
  • Prescribe-only (1 — BR-14 stale access) — requires iam:GenerateServiceLastAccessedDetails, a Generate* verb blocked by the DevOps Agent read-only guardrail; always reported N/A with out-of-band remediation, never Passed/Failed.

Testing

  • Agent Skill Eval — Audit: 100/100, Grade A, 0 critical / 0 warning / 0 info, passed: true.
  • Agent Skill Eval — Functional: overall 0.9727, passed: true (outcome 0.971, process 1.0, style 0.971, efficiency 0.948).
  • Agent Skill Eval — Trigger: should_trigger:false negatives 6/6 (no over-trigger onto SageMaker, AgentCore, reactive access-denial, or adoption-readiness). should_trigger:true activation is implied by the functional pass per CONTRIBUTING; it registers via the Skill tool when the skill is installed in the evaluation environment.
  • cfn-lint on cloudformation/devops-agent-skill-policies.yaml: 0 errors (only pre-existing warnings unrelated to this change).
  • mkdocs build --strict with the skill staged, on the pinned mkdocs-material==9.6.14: EXIT 0, 0 warnings; README uses only absolute GitHub URLs so the docs deploy does not break.

IAM / CloudFormation

Added a gated read-only inline policy EnableAIMLBedrockSecurityAssessment (default true, Condition-gated, with SkillPolicySummary updated) to cloudformation/devops-agent-skill-policies.yaml. It grants only the reads not covered by AIDevOpsAgentAccessPolicy:

  • bedrock:Get*/List* control-plane reads (21 actions — note the bedrock: IAM prefix applies to Agents/Knowledge Bases/Flows/Prompts; bedrock-agent is only the SDK client name),
  • cloudtrail:Get*/ListTrails, servicequotas:GetServiceQuota/GetAWSDefaultServiceQuota, inspector2:BatchGetAccountStatus.

IAM, Organizations, EC2, CloudWatch, S3, KMS, and Lambda reads ride AIDevOpsAgentAccessPolicy. All actions are read-only. BR-14 is deliberately not granted (prescribe-only).

Taskei

V2341889467 — https://taskei.amazon.dev/tasks/V2341889467

Checklist

  • SKILL.md has version and author in metadata
  • name in frontmatter matches directory name
  • description clearly states what/when to activate (1004/1024 chars)
  • README.md includes non-production disclaimer
  • CHANGELOG.md present (## 1.0.0, version-synced)
  • Agent Skill Eval audit passed (100/A) and functional passed (0.97)
  • Registered in llms.txt
  • .skilleval.yaml present (canonical STR-016 ignore)
  • README reference links are absolute GitHub URLs (mkdocs --strict safe)
  • Gated read-only CFN inline policy added, SkillPolicySummary updated
  • No internal Amazon tool names in submitted content
  • No customer data, account IDs, or credentials
  • Working against latest main

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache License 2.0.

Add a read-only Amazon Bedrock security posture assessment skill that runs
33 checks (BR-01..BR-33) across guardrail coverage, KMS encryption, VPC
private connectivity, agent and action-group IAM least privilege, model-
invocation logging, CloudTrail, CloudWatch alarms, service-quota throttling,
and Inspector Lambda scanning, producing severity-rated findings and a
consolidated report.

This is the Bedrock domain of a per-domain AI/ML security posture family
(carved out of a broader proposal per reviewer feedback to phase by domain).
It defers production-adoption readiness to bedrock-adoption-readiness,
reactive access-denial diagnosis to aiml-access-diagnostics, and AgentCore
observability to agentcore-observability-setup.

Every check carries a Verifiability classification (Verifiable / Heuristic /
Prescribe-only) so the report never marks an unread or access-denied control
as Passed. BR-14 (stale access) is Prescribe-only and always N/A, since it
would require iam:GenerateServiceLastAccessedDetails, a Generate* verb blocked
by the DevOps Agent read-only guardrail.

Register the skill in llms.txt and add a gated read-only inline policy
(EnableAIMLBedrockSecurityAssessment, default true) to the CloudFormation
skill policies for the Bedrock/cloudtrail/servicequotas/inspector2 reads not
covered by AIDevOpsAgentAccessPolicy.

sim: https://taskei.amazon.dev/tasks/V2341889467
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant