conformance

wcp-conformance

A standalone conformance test suite for the Workstation Capability Protocol (WCP). It certifies any WCP endpoint — the wcpd reference server or a third-party implementation — against the published contract: the manifest schema (schemas/manifest.schema.json), the per-operation schemas (schemas/ops/<op.id>.json), and the MCP binding described in the WCP spec. This package depends only on that published standard; it never imports or relies on wcpd internals.

The suite checks two things:

• Core conformance — the mandatory baseline every WCP server must meet: a valid manifest and the required wcp.system operations (e.g. describe).
• Per-family conformance — for each capability family the server advertises (wcp.fs, wcp.exec, wcp.window, ...), whether its operations round-trip against their schemas and enforce scopes, sandboxing, and error codes correctly. A server that only implements a subset of families (e.g. a headless container with just fs + exec) can still be certified for exactly what it implements.

Results are structured as a list of CheckResults rolled up into a ConformanceReport (see src/wcp_conformance/results.py), which can be rendered as human-readable text (report.summary()) or serialized to JSON (report.as_dict()).

Install

This package has its own virtualenv, independent of reference/wcpd.

cd conformance
python3 -m venv .venv
.venv/bin/pip install -e ".[dev]"

Test

cd conformance
.venv/bin/python -m pytest -q

Usage

wcp-conformance --help

The CLI needs a bearer token to call the target server. Exactly one of two token modes must be supplied:

1. Pre-minted tokens (works against any WCP server, including third-party implementations). The operator mints tokens out of band and passes them in:

   wcp-conformance \
     --endpoint https://workstation.example.com/mcp \
     --token-admin "$ADMIN_TOKEN" \
     --token-none "$ZERO_SCOPE_TOKEN" \
     --token-star "$STAR_SCOPE_TOKEN" \
     --json report.json

   • --token-admin: a full-scope token, used to exercise normal operation of every advertised family.
   • --token-none: a zero-scope token, used to verify the server correctly denies unauthorized calls.
   • --token-star: a token scoped to the wildcard * (but not the restricted wcp:input:synthesize scope), used to verify wildcard scopes don't implicitly grant input synthesis.

2. Local HS256 minting (convenient against reference/HS256-based servers such as wcpd). The suite mints its own tokens with pyjwt, given the server's shared secret:

   wcp-conformance \
     --endpoint http://localhost:8977/mcp \
     --jwt-secret "$WCP_JWT_SECRET" \
     --workstation-id dev-ws \
     --json report.json

By default the suite only ever invokes operations the target's manifest marks read_only; it never performs a mutating action expected to succeed. --include-mutating and --sandbox-path PATH (default .) are available for exercising more of the surface — see docs/certifying.md for details, including a worked example with --include-mutating --sandbox-path /tmp/wcp-probe.

The CLI prints report.summary() to stdout (unless --quiet), optionally writes a machine-readable report to the path given by --json, and exits 0 if report.ok is true, 1 if a required check failed, or 2 on a usage error (e.g. no token source supplied).

For the full certifying guide — both token modes, what "read-only by default" means, and how to read core_conformant/family_verdicts in the report — see docs/certifying.md.