Skip to content

Prepare the site for Agentic Evals use cases - #32

Merged
pramodbn27 merged 2 commits into
mainfrom
evals-usecases-builder
Oct 26, 2026
Merged

pramodbn27 merged 2 commits into
mainfrom
evals-usecases-builder

Conversation

@pramodbn27

Copy link
Copy Markdown
Member

Agentic Evals is adding hosted use cases at https://deepagentlabs.io/agentic-evals/usecases/, built with build_docs.py --section usecases like Control Tower's.

  • Builder: legacy-links.js is now added only to the Evals docs section. Before this, a use-cases build referenced a missing script and check_docs_site.py failed. The Evals docs output is unchanged (verified locally: docs 20 pages and use cases 5 pages both pass check_docs_site.py).
  • Use-cases index: a "More use cases / Agentic Evals" section after Control Tower, with the three pages and their view counters. The counter paths are added to usecase-views.js and update_usecase_views.py; the index loads usecase-views.js?v=4 so cached copies refresh.

The Evals PR depends on the builder change because its docs workflow checks out this repo's main. The index links go live once that PR deploys.

Checked: scripts/test_usecase_views.py passes. The new section renders with no horizontal overflow at 360 and 1440 px. (test_product_theme.py fails the same way on main, since it needs sibling checkouts.)

The builder added legacy-links.js to every Agentic Evals build. A use-cases
section has no such file, so its pages pointed at a missing script and
check_docs_site.py failed. The docs build is unchanged.
A "More use cases / Agentic Evals" section after Control Tower links the
three Evals use cases, and the view counters include their pages. The
counter script URL moves to ?v=4 so cached copies pick up the new paths.
@pramodbn27
pramodbn27 requested a review from a team as a code owner October 26, 2026 06:17
Copilot AI lite review requested due to automatic review settings October 26, 2026 06:17
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T06:18:33.098297Z c300092 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@pramodbn27
pramodbn27 merged commit 18000f1 into main Oct 26, 2026
3 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Existing unversioned use-case page loaders can serve stale tracking code that omits the new Evals paths.

1 open issue
What changed in this PR

This pull request prepares the site for Agentic Evals hosted use cases by updating the documentation builder, use-case index, and page-view tracking.

Changes:

  • Restricts legacy-links.js to Agentic Evals documentation builds.
  • Adds three Agentic Evals use cases to the main index.
  • Extends view-counter snapshots and browser tracking for the new paths.
  • Updates the index’s counter-script cache version.
File Reviewed changes Final review comments
usecases/​index.html Adds Agentic Evals metadata, links, counters, and cache-busted tracking script. Nit — The title “Catch a right reply reached the wrong way” is grammatically incomplete; revise it to a clearer phrase or the exact page title. Votes: 2
scripts/​update_usecase_views.py Adds the three Agentic Evals paths to scheduled view-total collection. No final comments.
scripts/​build_docs.py Loads legacy-link handling only for Agentic Evals documentation builds. No final comments.
assets/​js/​usecase-views.js Adds Agentic Evals paths to browser-side tracking and displayed counters. Moderate — Existing use-case pages load the unversioned script URL, so cached copies may omit the new Evals paths and skip tracking. Bump the loader URL on all use-case pages, including the Evals template, or otherwise provide deployment cache busting. Votes: 1

Review effort: Lite

Comment thread usecases/index.html
<a class="nav-cta" href="https://github.com/DeepAgentLabs">GitHub <span aria-hidden="true">↗</span></a>
</nav>
</header><main id="main"><section class="uc-hero uc-wrap"><p class="uc-kicker">DEEPAGENTLABS / USE CASES</p><div class="uc-heading"><h1>From an agent run<br>to <em>operational confidence.</em></h1><p>See how the ecosystem comes together. Hands-on stories, one continuous loop: observe, test, evaluate, improve.</p></div><div class="uc-note"><span class="uc-live-dot"></span> Interactive concept demos <span>Fictional data · Runs in your browser · No setup needed</span></div></section><section class="uc-wrap uc-card-grid" aria-label="Choose a use case"><a class="uc-card" href="/usecases/1/"><span class="uc-card-num">01 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-0"><span>Developer</span><b>How are my agents doing?</b><span>↓ MCP · Lens · Evals</span></div><p class="uc-kicker">CONNECT & OPERATE</p><h2>Operate your agents through MCP</h2><p>One conversation. Your entire AgenticOps ecosystem. Follow a support incident from the first question to a consolidated operations report.</p><strong class="uc-card-link">Ask. Investigate. Understand. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/1/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/2/"><span class="uc-card-num">02 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-1"><span class="uc-dot">Production run</span><b>↻ &nbsp; Replay + inject fault</b><span>Timeout → fallback → recovery</span></div><p class="uc-kicker">REPLAY & IMPROVE</p><h2>Replay, stress-test, optimize</h2><p>Turn a captured production run into a repeatable experiment. Introduce a failure, inspect the evidence, and validate a better recovery strategy.</p><strong class="uc-card-link">Break it here. Improve it safely. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/2/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/3/"><span class="uc-card-num">03 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-2"><span>Baseline &nbsp; → &nbsp; Candidate</span><b>78 <small>→</small> 94</b><span>Quality improves. Safety still gates.</span></div><p class="uc-kicker">EVALUATE & COMPARE</p><h2>Evaluate today. Validate your next release.</h2><p>Score a recorded execution, then replay its scenario against a candidate release. See improvements and regressions before making a release decision.</p><strong class="uc-card-link">Make the release decision visible. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/3/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/4/"><span class="uc-card-num">04 <span>INTERACTIVE DASHBOARD ↗</span></span><div class="uc-mini mini-lens"><span>Orchestrator → subagents → tools</span><b>▰ ▰▰ &nbsp; 2,840 tokens</b><span>Distributed tracing · Cost · Diagnostics</span></div><p class="uc-kicker">OBSERVE & DIAGNOSE</p><h2>See inside every agent run.</h2><p>Explore an AgenticLens observability console. Inspect nested traces, token usage by agent, model costs, retries and evidence-backed findings.</p><strong class="uc-card-link">Follow the trace. Find the bottleneck. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/4/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/5/"><span class="uc-card-num">05 <span>ANIMATED SIMULATION ↗</span></span><div class="uc-mini mini-sidecar-loop"><span>Agent → Sidecar → external calls</span><b>Check. Review. Approve. Execute.</b><span>Guardrails · Intent · Critique · Approval</span></div><p class="uc-kicker">SUPERVISE & EXECUTE</p><h2>A checkpoint before every call.</h2><p>Watch Sidecar supervise an agent’s tools, MCP and vector database calls. Automated checks precede every call; critical actions also require scripted human approval.</p><strong class="uc-card-link">Watch the supervision loop. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/5/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/6/"><span class="uc-card-num">06 <span>INTERACTIVE FLEET CONSOLE ↗</span></span><div class="uc-mini mini-tower"><span>Development · Staging · Production</span><b>Inspect → Review → Control</b><span>Fleet evidence · Approvals · Emergency brake</span></div><p class="uc-kicker">OPERATE & GOVERN</p><h2>An AI-native operations command center.</h2><p>Investigate a failure with MCP + Copilot, approve a remediation runbook and verify recovery. Or stop an agent, block new calls and revoke access.</p><strong class="uc-card-link">Take the operator’s seat. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/6/" title="Page views currently unavailable">— views</span></a></section><section class="uc-wrap uc-more-usecases" aria-labelledby="sidecar-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTIC-SIDECAR</p><h2 id="sidecar-examples-heading">Explore more Sidecar examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/agentic-sidecar/usecases/prevent-unauthorized-refunds/"><span>AGENTIC-SIDECAR</span><strong>Prevent unauthorized refunds</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/prevent-unauthorized-refunds/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-sidecar/usecases/governed-customer-support-agent/"><span>AGENTIC-SIDECAR</span><strong>Governed customer support agent</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/governed-customer-support-agent/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-sidecar/usecases/govern-langgraph-agent-workflows/"><span>AGENTIC-SIDECAR</span><strong>Govern LangGraph agent workflows</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/govern-langgraph-agent-workflows/" title="Page views currently unavailable">— views</span></a></div></section><section class="uc-wrap uc-more-usecases" aria-labelledby="tower-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTICOPS-CONTROL-TOWER</p><h2 id="tower-examples-heading">Explore more Control Tower examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/control-tower/usecases/fleet-health-snapshot/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Find unhealthy agents and capability gaps</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/fleet-health-snapshot/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/lens-failure-to-incident/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Turn a failed AgenticLens run into an incident</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/lens-failure-to-incident/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/ecosystem-evidence-posture/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>One posture view for every producer</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/ecosystem-evidence-posture/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/fleet-change-with-approval/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Change every staging agent with a second approver</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/fleet-change-with-approval/" title="Page views currently unavailable">— views</span></a></div></section><p class="uc-wrap uc-views-note">Page views since counters were added · Counts update in about five minutes · Powered by <a href="https://hits.sh/" target="_blank" rel="noopener noreferrer">Hits</a></p><section class="uc-wrap uc-bottom"><p class="uc-kicker">ONE STORY, MANY PERSPECTIVES</p><h2>Meet Northstar Support.</h2><p>A fictional support team uses a triage agent, an order specialist and a resolution agent to handle a delayed order. Explore how the team investigates a slow run, tests recovery and checks its next release.</p><div class="uc-loop">Capture <span>→</span> Standardize <span>→</span> Replay <span>→</span> Stress <span>→</span> Evaluate <span>→</span> Improve</div><p class="uc-muted">These stories illustrate a possible ecosystem workflow. Tool names, artifacts, scores and outcomes are illustrative; they do not demonstrate live integrations or guarantee current product support.</p></section></main> </body></html>
</header><main id="main"><section class="uc-hero uc-wrap"><p class="uc-kicker">DEEPAGENTLABS / USE CASES</p><div class="uc-heading"><h1>From an agent run<br>to <em>operational confidence.</em></h1><p>See how the ecosystem comes together. Hands-on stories, one continuous loop: observe, test, evaluate, improve.</p></div><div class="uc-note"><span class="uc-live-dot"></span> Interactive concept demos <span>Fictional data · Runs in your browser · No setup needed</span></div></section><section class="uc-wrap uc-card-grid" aria-label="Choose a use case"><a class="uc-card" href="/usecases/1/"><span class="uc-card-num">01 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-0"><span>Developer</span><b>How are my agents doing?</b><span>↓ MCP · Lens · Evals</span></div><p class="uc-kicker">CONNECT & OPERATE</p><h2>Operate your agents through MCP</h2><p>One conversation. Your entire AgenticOps ecosystem. Follow a support incident from the first question to a consolidated operations report.</p><strong class="uc-card-link">Ask. Investigate. Understand. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/1/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/2/"><span class="uc-card-num">02 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-1"><span class="uc-dot">Production run</span><b>↻ &nbsp; Replay + inject fault</b><span>Timeout → fallback → recovery</span></div><p class="uc-kicker">REPLAY & IMPROVE</p><h2>Replay, stress-test, optimize</h2><p>Turn a captured production run into a repeatable experiment. Introduce a failure, inspect the evidence, and validate a better recovery strategy.</p><strong class="uc-card-link">Break it here. Improve it safely. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/2/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/3/"><span class="uc-card-num">03 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-2"><span>Baseline &nbsp; → &nbsp; Candidate</span><b>78 <small>→</small> 94</b><span>Quality improves. Safety still gates.</span></div><p class="uc-kicker">EVALUATE & COMPARE</p><h2>Evaluate today. Validate your next release.</h2><p>Score a recorded execution, then replay its scenario against a candidate release. See improvements and regressions before making a release decision.</p><strong class="uc-card-link">Make the release decision visible. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/3/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/4/"><span class="uc-card-num">04 <span>INTERACTIVE DASHBOARD ↗</span></span><div class="uc-mini mini-lens"><span>Orchestrator → subagents → tools</span><b>▰ ▰▰ &nbsp; 2,840 tokens</b><span>Distributed tracing · Cost · Diagnostics</span></div><p class="uc-kicker">OBSERVE & DIAGNOSE</p><h2>See inside every agent run.</h2><p>Explore an AgenticLens observability console. Inspect nested traces, token usage by agent, model costs, retries and evidence-backed findings.</p><strong class="uc-card-link">Follow the trace. Find the bottleneck. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/4/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/5/"><span class="uc-card-num">05 <span>ANIMATED SIMULATION ↗</span></span><div class="uc-mini mini-sidecar-loop"><span>Agent → Sidecar → external calls</span><b>Check. Review. Approve. Execute.</b><span>Guardrails · Intent · Critique · Approval</span></div><p class="uc-kicker">SUPERVISE & EXECUTE</p><h2>A checkpoint before every call.</h2><p>Watch Sidecar supervise an agent’s tools, MCP and vector database calls. Automated checks precede every call; critical actions also require scripted human approval.</p><strong class="uc-card-link">Watch the supervision loop. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/5/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/6/"><span class="uc-card-num">06 <span>INTERACTIVE FLEET CONSOLE ↗</span></span><div class="uc-mini mini-tower"><span>Development · Staging · Production</span><b>Inspect → Review → Control</b><span>Fleet evidence · Approvals · Emergency brake</span></div><p class="uc-kicker">OPERATE & GOVERN</p><h2>An AI-native operations command center.</h2><p>Investigate a failure with MCP + Copilot, approve a remediation runbook and verify recovery. Or stop an agent, block new calls and revoke access.</p><strong class="uc-card-link">Take the operator’s seat. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/6/" title="Page views currently unavailable">— views</span></a></section><section class="uc-wrap uc-more-usecases" aria-labelledby="sidecar-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTIC-SIDECAR</p><h2 id="sidecar-examples-heading">Explore more Sidecar examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/agentic-sidecar/usecases/prevent-unauthorized-refunds/"><span>AGENTIC-SIDECAR</span><strong>Prevent unauthorized refunds</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/prevent-unauthorized-refunds/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-sidecar/usecases/governed-customer-support-agent/"><span>AGENTIC-SIDECAR</span><strong>Governed customer support agent</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/governed-customer-support-agent/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-sidecar/usecases/govern-langgraph-agent-workflows/"><span>AGENTIC-SIDECAR</span><strong>Govern LangGraph agent workflows</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/govern-langgraph-agent-workflows/" title="Page views currently unavailable">— views</span></a></div></section><section class="uc-wrap uc-more-usecases" aria-labelledby="tower-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTICOPS-CONTROL-TOWER</p><h2 id="tower-examples-heading">Explore more Control Tower examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/control-tower/usecases/fleet-health-snapshot/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Find unhealthy agents and capability gaps</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/fleet-health-snapshot/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/lens-failure-to-incident/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Turn a failed AgenticLens run into an incident</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/lens-failure-to-incident/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/ecosystem-evidence-posture/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>One posture view for every producer</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/ecosystem-evidence-posture/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/fleet-change-with-approval/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Change every staging agent with a second approver</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/fleet-change-with-approval/" title="Page views currently unavailable">— views</span></a></div></section><section class="uc-wrap uc-more-usecases" aria-labelledby="evals-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTIC-EVALS</p><h2 id="evals-examples-heading">Explore more Evals examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/agentic-evals/usecases/block-an-unsafe-release/"><span>AGENTIC-EVALS</span><strong>Block a release when one safety case fails</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-evals/usecases/block-an-unsafe-release/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-evals/usecases/right-reply-wrong-trajectory/"><span>AGENTIC-EVALS</span><strong>Catch a right reply reached the wrong way</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-evals/usecases/right-reply-wrong-trajectory/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-evals/usecases/gate-on-judge-reliability/"><span>AGENTIC-EVALS</span><strong>Don&#39;t let a flaky LLM judge approve a release</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-evals/usecases/gate-on-judge-reliability/" title="Page views currently unavailable">— views</span></a></div></section><p class="uc-wrap uc-views-note">Page views since counters were added · Counts update in about five minutes · Powered by <a href="https://hits.sh/" target="_blank" rel="noopener noreferrer">Hits</a></p><section class="uc-wrap uc-bottom"><p class="uc-kicker">ONE STORY, MANY PERSPECTIVES</p><h2>Meet Northstar Support.</h2><p>A fictional support team uses a triage agent, an order specialist and a resolution agent to handle a delayed order. Explore how the team investigates a slow run, tests recovery and checks its next release.</p><div class="uc-loop">Capture <span>→</span> Standardize <span>→</span> Replay <span>→</span> Stress <span>→</span> Evaluate <span>→</span> Improve</div><p class="uc-muted">These stories illustrate a possible ecosystem workflow. Tool names, artifacts, scores and outcomes are illustrative; they do not demonstrate live integrations or guarantee current product support.</p></section></main> </body></html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants