Repository navigation
feat(conformance): Socrate fixtures for mocks and a nightly live conformance suite - #119
Merged
Merged
Conversation
…suite Each consumer wrote its own Socrate mock and found the differences in production (#101). The new conformance package embeds what Socrate really issues: the claim sets of its access, ID and refresh tokens (authorization code with and without MFA, refresh, client_credentials, AUDIENCE_MODE=dual, namespaced custom claims, DPoP-bound), its token, userinfo and introspection responses, and its discovery and JWKS documents, with placeholders for volatile values. Names, Raw, Load, Sign, JWKS, Discovery, NewKey and Match let a mock serve and sign Socrate-shaped documents and check its own output. The package imports nothing else from backendkit. Live tests behind the conformance build tag match every fixture against a running Socrate (hosted sign-in and consent, TOTP enrolment, a DPoP proof) and test jwtauth, socrate, bff (authorize, callback, refresh, logout) and pep against it, each from its own package's tests. go test ./... is unchanged. scripts/conformance-local.sh builds Socrate from a checkout, starts it on a scratch database, seeds it with conformance/testdata/socrate-seed (throwaway secrets) and runs them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKRxaeYxyDhmt42cehLsGA
Builds Socrate from ovander/go-oauth2 (main, or a ref given on dispatch) with the Go its go.mod names, starts it on a PostgreSQL 16 service, seeds it and runs the conformance suite through scripts/conformance-local.sh. Runs nightly, on demand, and on pull requests that change the suite; it is informational and not a required check. A failed nightly or manual run opens or updates an issue labelled conformance from a separate job, the only one with issues: write; the server log goes to the job summary and an artifact. Refs #101 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKRxaeYxyDhmt42cehLsGA
Resolve the CHANGELOG conflict with #117: keep both Unreleased entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKRxaeYxyDhmt42cehLsGA
Resolve the CHANGELOG conflict with #118: the conformance entry stays under Added, the pending-login fix under Fixed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKRxaeYxyDhmt42cehLsGA
ovander
pushed a commit
that referenced
this pull request
Oct 11, 2026
Moves [Unreleased] under [1.26.0] - 2026-10-11 (#117 scopes, #118 pending-login UPDATE check, #119 conformance suite), updates the compare links and the README install snippet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKRxaeYxyDhmt42cehLsGA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Closes #101. Each consumer has built its own Socrate mock and found the differences in production (GPWA, then Lakebridge with
test/socrate-mock). This PR adds two things:1. A
conformancepackage. It imports no other backendkit package, so there is no new import edge. It embeds 25 fixtures of what Socrate really issues:client_credentials, plain, dual-audience, with custom claims and DPoP-bound;client_credentials, inactive);Volatile values are placeholders documented in the package. The API:
A consumer's mock loads or signs these instead of hand-writing claims. That's the "Done when" of #101: Lakebridge can drop
test/socrate-mock.2. A live suite behind the
conformancebuild tag. Plaingo test ./...and the required CI jobs are unaffected. Each package tests itself against a real Socrate, importing only what it already may:client_credentials, a DPoP-bound token, and the dual-audience and claim-mapped clients.Decideanswers (deny,no_applicable_rule,pep_mode_accepted), and the Enforcer in off mode and withMinimumModeenforce..github/workflows/conformance.yml:17 3 * * *),workflow_dispatchwith asocrate_refinput, and pull requests that change the suite. It is informational and not a required check.ovander/go-oauth2(mainby default) with that repo's Go version, on a Postgres 16 service, then runsscripts/conformance-local.sh. The script boots Socrate on loopback, seeds three clients and one user per package with secrets generated per run, and runsgo test -tags conformance -run '^TestConformance' ./....reportjob, the only one withissues: write, opens or comments on aconformanceissue. It never reports for pull requests.How it was tested
Live suite run end to end locally against Socrate built from go-oauth2
main(dcf8f01): 16 tests and 6 subtests pass, 0 fail.A deliberately broken fixture made the live check fail, as it should.
All fixtures are checked live. The one exception is the DPoP-off
discovery, which a unit test pins to the live-checkeddiscovery_dpopminus its DPoP key.This PR changes the suite, so it also triggers the workflow, which is the first run on GitHub.
go mod tidy && git diff --exit-code go.sumleavesgo.sumunchangedgo build ./...passesgo vet ./...passes, also with-tags conformancego test -race -count=1 -timeout=120s ./...passesgolangci-lint run ./...(v2.14.0) reports no issue, with and without--build-tags conformancegovulncheck ./...: not run locally (vuln.go.devis blocked here); CI runs it. No dependency change.actionlintpasses on the workflowA line is added under
## [Unreleased]inCHANGELOG.mdCompatibility
conformancepackage; no other package changes.conformanceissue rather than in production.socrate.IntrospectResponse.Roleis a field Socrate never sends. That would be a separate change.🤖 Generated with Claude Code
https://claude.ai/code/session_01GKRxaeYxyDhmt42cehLsGA
Generated by Claude Code