Skip to content

spec: issue-3540 Skills used in a Chainloop Trace session - #3541

Merged
jiparis merged 4 commits into
chainloop-dev:mainfrom
jiparis:issue-3540-spec-trace-skills-used
Oct 7, 2026
Merged

jiparis merged 4 commits into
chainloop-dev:mainfrom
jiparis:issue-3540-spec-trace-skills-used

Conversation

@jiparis

@jiparis jiparis commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

Spec issue-3540 proposes that Chainloop Trace records the skills that an agent used in a session, and puts each full skill into the attestation.

  • Each used skill becomes a spec entry of the new kind skill.
  • The CLI copies the skill folder (SKILL.md, scripts, and other files) when the session uses the skill. At push time, it uploads two EVIDENCE materials: SKILL.md, which a viewer shows, and a reproducible archive of the full folder, which a viewer offers as the download. This covers project, user, organization, and plugin skills, after secret redaction.
  • Spec entries get a new optional metadata object, with fields that depend on the kind. For a skill, it holds the skill source and the use counts: total, started by the model, started by the user, and from subagents.
  • The CLI makes the skill entries. This departs from D-001 of Spec issue-3495 for skills only.

The first version supports Claude Code and OpenCode. The change is additive and keeps the schema version.

Open questions for reviewers

None. The Decision Record holds the choices made during drafting and review.

Refs #3540

This spec was written with the help of Claude Code.

Refs chainloop-dev#3540

Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: 62aaeaaa-033c-4fb3-abac-9ee026634f60
@chainloop-platform

chainloop-platform Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

PR validation — ✅ 3 passing

Status Policy Material Messages
✅ Passed pr-min-approvals pr-info -
✅ Passed pr-description-required pr-info -
✅ Passed pr-user-story-linked pr-info -

View attestation ↗

AI Session Checks — 🟢 91% · ⚠️ 1 failing

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟢 91% 1 ⚠️ 1 100% AI / 0% Human 2 +404 / -223 3h47m37s

🟢 91% — 100% AI — ⚠️ 1 policies failing

Oct 7, 2026 07:51 UTC · 3h47m37s · $14.36 · 340 in / 198.8k out · claude-code 2.1.292 (claude-opus-5-5)

View session details ↗

Change Summary

AI Session Overall Score

🟢 91% — Spec-only session, well aligned; quality and verification were largely not applicable.

AI Session Analysis Breakdown

🟢 93% · user-trust-signal

🟢 User corrections stayed calm and collaborative through review and follow-up decisions. · High Impact

🟢 92% · alignment

🟢 AI opened the issue, drafted the spec, and opened the PR as asked. · High Impact

🟢 90% · scope-discipline

🟢 Changes stayed on the issue-3540 spec and its rename within docs/specs. · High Impact

🟢 86% · context-and-planning

No notes.

abstained · solution-quality

🟡 Solution quality was not assessed because the session only drafted and revised a spec. · Low Severity

abstained · verification

🟡 Verification was not assessed because the change was docs-only, not behavioral code. · Low Severity

Missing criteria: solution-quality, verification


File Attribution

████████████████████ 100% AI / 0% Human

Status Attribution File Lines
deleted ai docs/specs/006-trace-skills-used.md +210 / -210
modified ai docs/specs/issue-3540-trace-skills-used.md +194 / -13

Policies (4, 1 failing)

Status Policy Material Messages
✅ Passed ai-config-ai-agents-allowed ai-coding-session-62aaea -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-62aaea -
⚠️ Failed ai-config-no-secrets ai-coding-session-62aaea
  • Secret (generic-[REDACTED:generic-password) detected in session content [turn=514, source=tool_result, line=26]: | ⚠️ Failed | ai-config-no-secrets | [ai-coding-session-62aaea](https://app.chainloop.dev/u/chainloop/projects/chainloop/versi...
  • Secret (generic-password) detected in session content [turn=344, source=tool_result, line=12]: INF redacted secrets from the AI coding session before upload count=10 rules=["generic-[REDACTED:generic-password]"]
  • Secret (generic-password) detected in session content [turn=514, source=tool_result, line=26]: | ⚠️ Failed | ai-config-no-secrets | [ai-coding-session-62aaea](https://app.chainloop.dev/u/chainloop/projects/chainloop/versi...
  • Secret (generic-password) detected in session content [turn=623, source=assistant-text, line=20]: - ai-config-no-secrets: a false positive. The scanner matched the CLI's own log line from an earlier push, rules=["generic-[REDACTED:generic-password]"], which was captured in this session's t...
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-62aaea -

Security Checks — ✅ 5 passing

✅ secret-scan

Status Policy Messages
✅ Passed secrets-detection -

✅ sast-scan

Status Policy Messages
✅ Passed owasp-top10-2025 -
✅ Passed sast -
✅ Passed cwe-top25 -
✅ Passed cwe-top26-40-cusp -
Scans not applied (3)
Scan Reason
vulnerability-scan no manifest/lockfile changed
github-actions-scan no workflow files changed
iac-scan no IaC files changed

View attestation ↗

Security context

✅ Nothing this change touches has a recorded security-fix history.

View in Chainloop ↗ · How this works ↗


Powered by Chainloop and Chainloop Trace

@jiparis
jiparis requested a review from a team October 7, 2026 08:57
Comment thread docs/specs/006-trace-skills-used.md Outdated
Refs chainloop-dev#3540

Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: 62aaeaaa-033c-4fb3-abac-9ee026634f60
@migmartri

Copy link
Copy Markdown
Member

Heads up: the spec naming convention changed. Specs are now named after their parent GitHub issue instead of a sequential number, because parallel spec PRs kept taking the same number.

Please rename this spec before merging:

  • File: docs/specs/issue-3540-trace-skills-used.md
  • Heading: # Spec issue-3540: Skills used in a Chainloop Trace session
  • PR title: spec: issue-3540 Skills used in a Chainloop Trace session

Existing specs on main are being renamed in #3545.

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri

Refs chainloop-dev#3540

Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: 62aaeaaa-033c-4fb3-abac-9ee026634f60
@jiparis jiparis changed the title spec: 006 Skills used in a Chainloop Trace session spec: issue-3540 Skills used in a Chainloop Trace session Oct 7, 2026
@jiparis

jiparis commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

Renamed in ef55926: docs/specs/issue-3540-trace-skills-used.md, heading # Spec issue-3540: ..., and the PR title. Links to the other specs use the names from #3545.

Refs chainloop-dev#3540

Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: 62aaeaaa-033c-4fb3-abac-9ee026634f60
@jiparis
jiparis merged commit fcb6c22 into chainloop-dev:main Oct 7, 2026
16 of 17 checks passed
@jiparis
jiparis deleted the issue-3540-spec-trace-skills-used branch October 7, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

spec Design spec

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants