From 94d3689182f44a361f39a198294eb8012fb8af79 Mon Sep 17 00:00:00 2001 From: Ray Clanan <115944+rclanan@users.noreply.github.com> Date: Tue, 20 Oct 2026 05:35:09 -0400 Subject: [PATCH] docs(reality): surface mock fidelity preview and evidence scope --- index.html | 5 +++-- note-api-drift-detection-and-contract-alerting.html | 9 +++++++++ src/pages/index.html | 5 +++-- .../note-api-drift-detection-and-contract-alerting.html | 9 +++++++++ 4 files changed, 24 insertions(+), 4 deletions(-) diff --git a/index.html b/index.html index e66a9c6..b25412a 100644 --- a/index.html +++ b/index.html @@ -304,8 +304,9 @@

Latency & fault injec
-

OpenAPI conformance

-

Validate requests against your spec with deep schema resolution across path, query, header, cookie and body.

+

Reality: mock fidelity preview

+

Find changed fields, types and operations before an outdated mock gives you a passing test. Definition comparisons and the fidelity panel are in development.

+ Explore the fidelity preview
diff --git a/note-api-drift-detection-and-contract-alerting.html b/note-api-drift-detection-and-contract-alerting.html index 818dbfb..e6eb1e5 100644 --- a/note-api-drift-detection-and-contract-alerting.html +++ b/note-api-drift-detection-and-contract-alerting.html @@ -157,6 +157,15 @@

Most integration failures are not sudden outages. They are slow contract drift that nobody sees until deployment day.

Our current model is to run spec-driven checks continuously and treat schema mismatch as an alerting event, not a debugging surprise. This catches nullable flips, enum additions, and response-shape regressions while teams still have rollback room.

The practical win is faster triage: failures now include endpoint, expected shape, observed payload, and severity metadata so operators can route to the right service owner immediately.

+

Reality: keep mocks aligned with the API

+

Mock fidelity is in development. The definition comparison, fixture checks and admin panel below require an upcoming release. Follow the fidelity implementation for availability.

+

Keep the definition used to generate your mock, then compare it with the current OpenAPI or AsyncAPI contract. Add a recent upstream capture to check the traffic you actually observed. The report identifies changed fields, types and operations so an old response shape cannot quietly become your test's source of truth.

+ 
mockforge verify-mocks --spec upstream-current.yaml \
+  --mock-spec mock-generated-from.yaml \
+  --capture-db upstream-recordings.db --fail-on-drift \
+  --output fidelity.json
+

Per-route results distinguish Verified, Drift detected, Not verified and Not sampled. Verification covers the supplied contract and captured sample. Missing capture data or unsupported formats stay visible as incomplete evidence. A matching definition alone does not prove that every production behavior was exercised.

+

Use the CI gate to make drift fail the build, inspect field-level changes in the admin panel, and refresh the affected mocks from the source you checked. Captures remain read-only, with timestamps in the report so the team can see which evidence supported the result.

Docs GitHub diff --git a/src/pages/index.html b/src/pages/index.html index e09c3f7..89da978 100644 --- a/src/pages/index.html +++ b/src/pages/index.html @@ -214,8 +214,9 @@

Latency & fault injec
-

OpenAPI conformance

-

Validate requests against your spec with deep schema resolution across path, query, header, cookie and body.

+

Reality: mock fidelity preview

+

Find changed fields, types and operations before an outdated mock gives you a passing test. Definition comparisons and the fidelity panel are in development.

+ Explore the fidelity preview
diff --git a/src/pages/note-api-drift-detection-and-contract-alerting.html b/src/pages/note-api-drift-detection-and-contract-alerting.html index 672316e..9258ec6 100644 --- a/src/pages/note-api-drift-detection-and-contract-alerting.html +++ b/src/pages/note-api-drift-detection-and-contract-alerting.html @@ -25,6 +25,15 @@

Most integration failures are not sudden outages. They are slow contract drift that nobody sees until deployment day.

Our current model is to run spec-driven checks continuously and treat schema mismatch as an alerting event, not a debugging surprise. This catches nullable flips, enum additions, and response-shape regressions while teams still have rollback room.

The practical win is faster triage: failures now include endpoint, expected shape, observed payload, and severity metadata so operators can route to the right service owner immediately.

+

Reality: keep mocks aligned with the API

+

Mock fidelity is in development. The definition comparison, fixture checks and admin panel below require an upcoming release. Follow the fidelity implementation for availability.

+

Keep the definition used to generate your mock, then compare it with the current OpenAPI or AsyncAPI contract. Add a recent upstream capture to check the traffic you actually observed. The report identifies changed fields, types and operations so an old response shape cannot quietly become your test's source of truth.

+ 
mockforge verify-mocks --spec upstream-current.yaml \
+  --mock-spec mock-generated-from.yaml \
+  --capture-db upstream-recordings.db --fail-on-drift \
+  --output fidelity.json
+

Per-route results distinguish Verified, Drift detected, Not verified and Not sampled. Verification covers the supplied contract and captured sample. Missing capture data or unsupported formats stay visible as incomplete evidence. A matching definition alone does not prove that every production behavior was exercised.

+

Use the CI gate to make drift fail the build, inspect field-level changes in the admin panel, and refresh the affected mocks from the source you checked. Captures remain read-only, with timestamps in the report so the team can see which evidence supported the result.

Docs GitHub