Repository navigation
Prepare the site for Agentic Evals use cases - #32
Merged
Merged
Conversation
The builder added legacy-links.js to every Agentic Evals build. A use-cases section has no such file, so its pages pointed at a missing script and check_docs_site.py failed. The docs build is unchanged.
A "More use cases / Agentic Evals" section after Control Tower links the three Evals use cases, and the view counters include their pages. The counter script URL moves to ?v=4 so cached copies pick up the new paths.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
🟡 Changes recommended
Existing unversioned use-case page loaders can serve stale tracking code that omits the new Evals paths.
1 open issue
What changed in this PR
This pull request prepares the site for Agentic Evals hosted use cases by updating the documentation builder, use-case index, and page-view tracking.
Changes:
- Restricts
legacy-links.jsto Agentic Evals documentation builds. - Adds three Agentic Evals use cases to the main index.
- Extends view-counter snapshots and browser tracking for the new paths.
- Updates the index’s counter-script cache version.
| File | Reviewed changes | Final review comments |
|---|---|---|
usecases/index.html |
Adds Agentic Evals metadata, links, counters, and cache-busted tracking script. | Nit — The title “Catch a right reply reached the wrong way” is grammatically incomplete; revise it to a clearer phrase or the exact page title. Votes: 2 |
scripts/update_usecase_views.py |
Adds the three Agentic Evals paths to scheduled view-total collection. | No final comments. |
scripts/build_docs.py |
Loads legacy-link handling only for Agentic Evals documentation builds. | No final comments. |
assets/js/usecase-views.js |
Adds Agentic Evals paths to browser-side tracking and displayed counters. | Moderate — Existing use-case pages load the unversioned script URL, so cached copies may omit the new Evals paths and skip tracking. Bump the loader URL on all use-case pages, including the Evals template, or otherwise provide deployment cache busting. Votes: 1 |
Review effort: Lite
| <a class="nav-cta" href="https://github.com/DeepAgentLabs">GitHub <span aria-hidden="true">↗</span></a> | ||
| </nav> | ||
| </header><main id="main"><section class="uc-hero uc-wrap"><p class="uc-kicker">DEEPAGENTLABS / USE CASES</p><div class="uc-heading"><h1>From an agent run<br>to <em>operational confidence.</em></h1><p>See how the ecosystem comes together. Hands-on stories, one continuous loop: observe, test, evaluate, improve.</p></div><div class="uc-note"><span class="uc-live-dot"></span> Interactive concept demos <span>Fictional data · Runs in your browser · No setup needed</span></div></section><section class="uc-wrap uc-card-grid" aria-label="Choose a use case"><a class="uc-card" href="/usecases/1/"><span class="uc-card-num">01 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-0"><span>Developer</span><b>How are my agents doing?</b><span>↓ MCP · Lens · Evals</span></div><p class="uc-kicker">CONNECT & OPERATE</p><h2>Operate your agents through MCP</h2><p>One conversation. Your entire AgenticOps ecosystem. Follow a support incident from the first question to a consolidated operations report.</p><strong class="uc-card-link">Ask. Investigate. Understand. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/1/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/2/"><span class="uc-card-num">02 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-1"><span class="uc-dot">Production run</span><b>↻ Replay + inject fault</b><span>Timeout → fallback → recovery</span></div><p class="uc-kicker">REPLAY & IMPROVE</p><h2>Replay, stress-test, optimize</h2><p>Turn a captured production run into a repeatable experiment. Introduce a failure, inspect the evidence, and validate a better recovery strategy.</p><strong class="uc-card-link">Break it here. Improve it safely. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/2/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/3/"><span class="uc-card-num">03 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-2"><span>Baseline → Candidate</span><b>78 <small>→</small> 94</b><span>Quality improves. Safety still gates.</span></div><p class="uc-kicker">EVALUATE & COMPARE</p><h2>Evaluate today. Validate your next release.</h2><p>Score a recorded execution, then replay its scenario against a candidate release. See improvements and regressions before making a release decision.</p><strong class="uc-card-link">Make the release decision visible. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/3/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/4/"><span class="uc-card-num">04 <span>INTERACTIVE DASHBOARD ↗</span></span><div class="uc-mini mini-lens"><span>Orchestrator → subagents → tools</span><b>▰ ▰▰ 2,840 tokens</b><span>Distributed tracing · Cost · Diagnostics</span></div><p class="uc-kicker">OBSERVE & DIAGNOSE</p><h2>See inside every agent run.</h2><p>Explore an AgenticLens observability console. Inspect nested traces, token usage by agent, model costs, retries and evidence-backed findings.</p><strong class="uc-card-link">Follow the trace. Find the bottleneck. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/4/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/5/"><span class="uc-card-num">05 <span>ANIMATED SIMULATION ↗</span></span><div class="uc-mini mini-sidecar-loop"><span>Agent → Sidecar → external calls</span><b>Check. Review. Approve. Execute.</b><span>Guardrails · Intent · Critique · Approval</span></div><p class="uc-kicker">SUPERVISE & EXECUTE</p><h2>A checkpoint before every call.</h2><p>Watch Sidecar supervise an agent’s tools, MCP and vector database calls. Automated checks precede every call; critical actions also require scripted human approval.</p><strong class="uc-card-link">Watch the supervision loop. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/5/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/6/"><span class="uc-card-num">06 <span>INTERACTIVE FLEET CONSOLE ↗</span></span><div class="uc-mini mini-tower"><span>Development · Staging · Production</span><b>Inspect → Review → Control</b><span>Fleet evidence · Approvals · Emergency brake</span></div><p class="uc-kicker">OPERATE & GOVERN</p><h2>An AI-native operations command center.</h2><p>Investigate a failure with MCP + Copilot, approve a remediation runbook and verify recovery. Or stop an agent, block new calls and revoke access.</p><strong class="uc-card-link">Take the operator’s seat. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/6/" title="Page views currently unavailable">— views</span></a></section><section class="uc-wrap uc-more-usecases" aria-labelledby="sidecar-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTIC-SIDECAR</p><h2 id="sidecar-examples-heading">Explore more Sidecar examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/agentic-sidecar/usecases/prevent-unauthorized-refunds/"><span>AGENTIC-SIDECAR</span><strong>Prevent unauthorized refunds</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/prevent-unauthorized-refunds/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-sidecar/usecases/governed-customer-support-agent/"><span>AGENTIC-SIDECAR</span><strong>Governed customer support agent</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/governed-customer-support-agent/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-sidecar/usecases/govern-langgraph-agent-workflows/"><span>AGENTIC-SIDECAR</span><strong>Govern LangGraph agent workflows</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/govern-langgraph-agent-workflows/" title="Page views currently unavailable">— views</span></a></div></section><section class="uc-wrap uc-more-usecases" aria-labelledby="tower-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTICOPS-CONTROL-TOWER</p><h2 id="tower-examples-heading">Explore more Control Tower examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/control-tower/usecases/fleet-health-snapshot/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Find unhealthy agents and capability gaps</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/fleet-health-snapshot/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/lens-failure-to-incident/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Turn a failed AgenticLens run into an incident</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/lens-failure-to-incident/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/ecosystem-evidence-posture/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>One posture view for every producer</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/ecosystem-evidence-posture/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/fleet-change-with-approval/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Change every staging agent with a second approver</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/fleet-change-with-approval/" title="Page views currently unavailable">— views</span></a></div></section><p class="uc-wrap uc-views-note">Page views since counters were added · Counts update in about five minutes · Powered by <a href="https://hits.sh/" target="_blank" rel="noopener noreferrer">Hits</a></p><section class="uc-wrap uc-bottom"><p class="uc-kicker">ONE STORY, MANY PERSPECTIVES</p><h2>Meet Northstar Support.</h2><p>A fictional support team uses a triage agent, an order specialist and a resolution agent to handle a delayed order. Explore how the team investigates a slow run, tests recovery and checks its next release.</p><div class="uc-loop">Capture <span>→</span> Standardize <span>→</span> Replay <span>→</span> Stress <span>→</span> Evaluate <span>→</span> Improve</div><p class="uc-muted">These stories illustrate a possible ecosystem workflow. Tool names, artifacts, scores and outcomes are illustrative; they do not demonstrate live integrations or guarantee current product support.</p></section></main> </body></html> | ||
| </header><main id="main"><section class="uc-hero uc-wrap"><p class="uc-kicker">DEEPAGENTLABS / USE CASES</p><div class="uc-heading"><h1>From an agent run<br>to <em>operational confidence.</em></h1><p>See how the ecosystem comes together. Hands-on stories, one continuous loop: observe, test, evaluate, improve.</p></div><div class="uc-note"><span class="uc-live-dot"></span> Interactive concept demos <span>Fictional data · Runs in your browser · No setup needed</span></div></section><section class="uc-wrap uc-card-grid" aria-label="Choose a use case"><a class="uc-card" href="/usecases/1/"><span class="uc-card-num">01 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-0"><span>Developer</span><b>How are my agents doing?</b><span>↓ MCP · Lens · Evals</span></div><p class="uc-kicker">CONNECT & OPERATE</p><h2>Operate your agents through MCP</h2><p>One conversation. Your entire AgenticOps ecosystem. Follow a support incident from the first question to a consolidated operations report.</p><strong class="uc-card-link">Ask. Investigate. Understand. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/1/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/2/"><span class="uc-card-num">02 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-1"><span class="uc-dot">Production run</span><b>↻ Replay + inject fault</b><span>Timeout → fallback → recovery</span></div><p class="uc-kicker">REPLAY & IMPROVE</p><h2>Replay, stress-test, optimize</h2><p>Turn a captured production run into a repeatable experiment. Introduce a failure, inspect the evidence, and validate a better recovery strategy.</p><strong class="uc-card-link">Break it here. Improve it safely. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/2/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/3/"><span class="uc-card-num">03 <span>INTERACTIVE DEMO ↗</span></span><div class="uc-mini mini-2"><span>Baseline → Candidate</span><b>78 <small>→</small> 94</b><span>Quality improves. Safety still gates.</span></div><p class="uc-kicker">EVALUATE & COMPARE</p><h2>Evaluate today. Validate your next release.</h2><p>Score a recorded execution, then replay its scenario against a candidate release. See improvements and regressions before making a release decision.</p><strong class="uc-card-link">Make the release decision visible. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/3/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/4/"><span class="uc-card-num">04 <span>INTERACTIVE DASHBOARD ↗</span></span><div class="uc-mini mini-lens"><span>Orchestrator → subagents → tools</span><b>▰ ▰▰ 2,840 tokens</b><span>Distributed tracing · Cost · Diagnostics</span></div><p class="uc-kicker">OBSERVE & DIAGNOSE</p><h2>See inside every agent run.</h2><p>Explore an AgenticLens observability console. Inspect nested traces, token usage by agent, model costs, retries and evidence-backed findings.</p><strong class="uc-card-link">Follow the trace. Find the bottleneck. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/4/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/5/"><span class="uc-card-num">05 <span>ANIMATED SIMULATION ↗</span></span><div class="uc-mini mini-sidecar-loop"><span>Agent → Sidecar → external calls</span><b>Check. Review. Approve. Execute.</b><span>Guardrails · Intent · Critique · Approval</span></div><p class="uc-kicker">SUPERVISE & EXECUTE</p><h2>A checkpoint before every call.</h2><p>Watch Sidecar supervise an agent’s tools, MCP and vector database calls. Automated checks precede every call; critical actions also require scripted human approval.</p><strong class="uc-card-link">Watch the supervision loop. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/5/" title="Page views currently unavailable">— views</span></a><a class="uc-card" href="/usecases/6/"><span class="uc-card-num">06 <span>INTERACTIVE FLEET CONSOLE ↗</span></span><div class="uc-mini mini-tower"><span>Development · Staging · Production</span><b>Inspect → Review → Control</b><span>Fleet evidence · Approvals · Emergency brake</span></div><p class="uc-kicker">OPERATE & GOVERN</p><h2>An AI-native operations command center.</h2><p>Investigate a failure with MCP + Copilot, approve a remediation runbook and verify recovery. Or stop an agent, block new calls and revoke access.</p><strong class="uc-card-link">Take the operator’s seat. <span>→</span></strong><span class="uc-view-count" data-usecase-views="/usecases/6/" title="Page views currently unavailable">— views</span></a></section><section class="uc-wrap uc-more-usecases" aria-labelledby="sidecar-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTIC-SIDECAR</p><h2 id="sidecar-examples-heading">Explore more Sidecar examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/agentic-sidecar/usecases/prevent-unauthorized-refunds/"><span>AGENTIC-SIDECAR</span><strong>Prevent unauthorized refunds</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/prevent-unauthorized-refunds/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-sidecar/usecases/governed-customer-support-agent/"><span>AGENTIC-SIDECAR</span><strong>Governed customer support agent</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/governed-customer-support-agent/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-sidecar/usecases/govern-langgraph-agent-workflows/"><span>AGENTIC-SIDECAR</span><strong>Govern LangGraph agent workflows</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-sidecar/usecases/govern-langgraph-agent-workflows/" title="Page views currently unavailable">— views</span></a></div></section><section class="uc-wrap uc-more-usecases" aria-labelledby="tower-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTICOPS-CONTROL-TOWER</p><h2 id="tower-examples-heading">Explore more Control Tower examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/control-tower/usecases/fleet-health-snapshot/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Find unhealthy agents and capability gaps</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/fleet-health-snapshot/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/lens-failure-to-incident/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Turn a failed AgenticLens run into an incident</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/lens-failure-to-incident/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/ecosystem-evidence-posture/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>One posture view for every producer</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/ecosystem-evidence-posture/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/control-tower/usecases/fleet-change-with-approval/"><span>AGENTICOPS-CONTROL-TOWER</span><strong>Change every staging agent with a second approver</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/control-tower/usecases/fleet-change-with-approval/" title="Page views currently unavailable">— views</span></a></div></section><section class="uc-wrap uc-more-usecases" aria-labelledby="evals-examples-heading"><p class="uc-kicker">MORE USE CASES / AGENTIC-EVALS</p><h2 id="evals-examples-heading">Explore more Evals examples.</h2><div class="uc-resource-grid"><a href="https://deepagentlabs.io/agentic-evals/usecases/block-an-unsafe-release/"><span>AGENTIC-EVALS</span><strong>Block a release when one safety case fails</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-evals/usecases/block-an-unsafe-release/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-evals/usecases/right-reply-wrong-trajectory/"><span>AGENTIC-EVALS</span><strong>Catch a right reply reached the wrong way</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-evals/usecases/right-reply-wrong-trajectory/" title="Page views currently unavailable">— views</span></a><a href="https://deepagentlabs.io/agentic-evals/usecases/gate-on-judge-reliability/"><span>AGENTIC-EVALS</span><strong>Don't let a flaky LLM judge approve a release</strong><small>Open use case <span aria-hidden="true">↗</span></small><span class="uc-view-count" data-usecase-views="/agentic-evals/usecases/gate-on-judge-reliability/" title="Page views currently unavailable">— views</span></a></div></section><p class="uc-wrap uc-views-note">Page views since counters were added · Counts update in about five minutes · Powered by <a href="https://hits.sh/" target="_blank" rel="noopener noreferrer">Hits</a></p><section class="uc-wrap uc-bottom"><p class="uc-kicker">ONE STORY, MANY PERSPECTIVES</p><h2>Meet Northstar Support.</h2><p>A fictional support team uses a triage agent, an order specialist and a resolution agent to handle a delayed order. Explore how the team investigates a slow run, tests recovery and checks its next release.</p><div class="uc-loop">Capture <span>→</span> Standardize <span>→</span> Replay <span>→</span> Stress <span>→</span> Evaluate <span>→</span> Improve</div><p class="uc-muted">These stories illustrate a possible ecosystem workflow. Tool names, artifacts, scores and outcomes are illustrative; they do not demonstrate live integrations or guarantee current product support.</p></section></main> </body></html> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Agentic Evals is adding hosted use cases at https://deepagentlabs.io/agentic-evals/usecases/, built with
build_docs.py --section usecaseslike Control Tower's.legacy-links.jsis now added only to the Evals docs section. Before this, a use-cases build referenced a missing script andcheck_docs_site.pyfailed. The Evals docs output is unchanged (verified locally: docs 20 pages and use cases 5 pages both passcheck_docs_site.py).usecase-views.jsandupdate_usecase_views.py; the index loadsusecase-views.js?v=4so cached copies refresh.The Evals PR depends on the builder change because its docs workflow checks out this repo's
main. The index links go live once that PR deploys.Checked:
scripts/test_usecase_views.pypasses. The new section renders with no horizontal overflow at 360 and 1440 px. (test_product_theme.pyfails the same way onmain, since it needs sibling checkouts.)