From 2b400654e1a5aba387d86b133a093dafd8b0e240 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 21 Sep 2026 03:02:14 +0900 Subject: [PATCH 1/7] docs(assessment): propose delivery ownership boundary --- ...ment-assignment-result-handoff-boundary.md | 157 ++++++++++++++++++ 1 file changed, 157 insertions(+) create mode 100644 docs/adr/0429-assessment-assignment-result-handoff-boundary.md diff --git a/docs/adr/0429-assessment-assignment-result-handoff-boundary.md b/docs/adr/0429-assessment-assignment-result-handoff-boundary.md new file mode 100644 index 000000000..506ba2c52 --- /dev/null +++ b/docs/adr/0429-assessment-assignment-result-handoff-boundary.md @@ -0,0 +1,157 @@ +# ADR 0429: Assessment delivery owns assignment and result handoff, not instrument truth + +- Status: Proposed +- Date: 2026-09-21 +- Issue: #429 +- Protected baseline reviewed: `develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f` + +## Problem + +Orgmetra already names `assessment_assignment` in its canonical logical database-object vocabulary, and selection/validation flows require assessment evidence. Protected product truth does not, however, identify an executable owner for the business lifecycle that requests an assessment, binds it to one accountable HR purpose and subject, tracks an administration occurrence, and accepts an immutable result reference from an external assessment/scoring owner. + +The existing context map cannot safely absorb this by implication. `talent_acquisition` owns requisitions, candidates, interviews, decision evidence and selection decisions. `performance_management` owns performance criteria and observations. `workforce_validation` owns validity studies, subgroup diagnostics, drift, utility and scientific adapters. `integration_hub` owns transport/adapters, not business assessment state. Protected TRD also deliberately keeps assessment results as external immutable snapshot references unless a later ADR transfers instrument lifecycle ownership. + +This leaves two failure modes. Treating assessment as generic integration state loses accountable business purpose, administration identity, cancellation/expiry/correction and decision provenance. Treating an assessment callback as Talent or Validation truth risks silently importing instrument, scoring and psychometric authority into Orgmetra. + +## Constraints + +1. Orgmetra remains authoritative for HR domain truth, but does not acquire assessment instrument, item-bank, response, scoring-algorithm or psychometric-model lifecycle authority through this ADR. +2. External assessment/scoring owners are consumed only through released, versioned contracts. Mutable branches, floating model labels and direct foreign-table access are prohibited. +3. Job/FJA/KSAO job-relatedness remains `job_architecture` truth. Selection authority remains in the governed selection-decision boundary. Validity, fairness, adverse-impact, estimand, transportability and scientific interpretation remain `workforce_validation` responsibilities. +4. Keyverse remains the identity/authentication backend. Subject, accommodation and accessibility data crossing the assessment boundary must be purpose-minimized. +5. Assessment results remain immutable external owner references plus exact contract/version/integrity evidence by default. A digest without a resolvable owner locator is insufficient provenance. +6. Transport completion, HTTP success, provider session completion or receipt delivery cannot authorize a selection, promotion, development, succession or other high-impact employment outcome. +7. Retry/replay must not create a second semantic assignment or administration. A genuinely new administration remains a distinct occurrence even for the same person and procedure. +8. `unavailable`, `interrupted`, `invalidated`, `expired` and `not_verifiable` are explicit non-success states; none may be coerced into a numeric score or normal completion. +9. Rescoring, correction and supersession are append-only. Historical evidence used by a released decision or validation study is not overwritten. +10. This Proposed ADR does not authorize a schema, service, API, event, UI or migration before executable RED→GREEN evidence is reviewed against then-current protected truth. + +## Alternatives considered + +### A. `talent_acquisition` owns assessment orchestration + +This is attractive for recruitment and selection because candidate assessment is adjacent to requisition/application/selection evidence. It is rejected as the general owner because ISO 10667 assessment use spans recruitment, development, appraisal, promotion, succession and reassignment. Expanding `talent_acquisition` across those post-hire lifecycles would distort its Ubiquitous Language and create pressure to copy worker/Talent state into an acquisition context. + +A future implementation may expose a Talent Acquisition ACL that requests an assessment, but it does not own the assessment administration lifecycle. + +### B. dedicated `assessment_delivery` bounded context + +Selected as the product-scope direction. `assessment_delivery` owns the operational lifecycle of an assessment request/assignment and administration occurrence plus immutable external result-handoff evidence. It does not own the instrument, items, responses, scoring model, validity/fairness conclusions or employment decision. + +This isolates a business lifecycle that legitimately spans recruiting and post-hire use while keeping scientific authority external and preserving explicit ACLs to `talent_acquisition`, a future protected `talent_management` context if adopted, `job_architecture`, and `workforce_validation`. + +The cost is another bounded context, schema/API/event lifecycle and release identity. That cost is accepted provisionally because the lifecycle and invariants are domain state, not merely connector state, and because forcing it into acquisition creates a cross-lifecycle semantic mismatch. + +### C. `integration_hub` owns assessment orchestration + +Rejected except for connector/session transport mechanics. Provider authentication, delivery retries, webhook transport and inbox/outbox adapter state may live behind `integration_hub`, but business assignment purpose, administration occurrence, cancellation/expiry, result acceptance and supersession are not generic integration facts. + +### D. assessment assignment remains entirely external + +Rejected as the current product direction. It would require Orgmetra to give up governed assignment/administration provenance while selection and Workforce Validation still depend on exact assessment evidence. If future product scope contracts, this option must be adopted through a new ADR and buyer-facing product/documentation changes rather than by leaving an ambiguous logical object in place. + +## Decision + +Create a dedicated `assessment_delivery` bounded context as the provisional product/domain owner for assessment assignment and immutable result handoff. + +The context owns only these concepts: + +- `AssessmentAssignment`: one tenant/purpose/subject/process request for an exact released assessment procedure contract. +- `AssessmentAdministration`: one actual administration occurrence under an assignment, with explicit occurrence identity and lifecycle. +- `AssessmentResultReference`: an immutable external owner locator bound to exact assessment/scoring contract versions, administration occurrence and integrity evidence. +- `AssessmentResultSupersession`: append-only correction/rescore/supersession lineage. +- `AssessmentDeliveryPolicy`: versioned operational rules for expiry, permitted purposes, retry/idempotency and required evidence; it is not a scoring or psychometric policy. + +The aggregate boundary is `AssessmentAssignment`. It may create or admit administration occurrences only under a versioned assignment policy. Result references attach to one exact administration occurrence and cannot mutate an earlier result in place. + +The Ubiquitous Language deliberately distinguishes: + +- **assignment** — accountable business request to perform a procedure; +- **administration** — one actual occurrence of delivering that procedure; +- **provider session** — transport/provider implementation coordinate, not the domain identity by itself; +- **result reference** — immutable pointer to external result evidence; +- **scoring contract** — released external owner contract defining how a result was produced; +- **selection/HR decision** — downstream human-governed decision outside this context; +- **validity/fairness evidence** — downstream `workforce_validation` scientific authority. + +## Context map + +- `talent_acquisition` → `assessment_delivery`: requests recruitment/selection assignments through a released ACL; receives non-authorizing status/result-reference events. It never treats callback completion as a selection decision. +- future protected `talent_management` → `assessment_delivery`: if that bounded context becomes protected truth, it may request development/promotion/succession assessments through the same released ACL without importing acquisition semantics. +- `job_architecture` → `assessment_delivery`: supplies released Job/FJA/KSAO evidence references required to establish purpose/job-related context. Assessment Delivery stores opaque/versioned references only. +- `assessment_delivery` → external provider/scoring owner: uses released/versioned APIs/events via an integration adapter. Instrument/item/response/scoring internals remain foreign authority. +- `assessment_delivery` → `talent_acquisition` / future Talent consumers: publishes purpose-authorized immutable result-reference evidence; downstream high-impact decisions re-authorize and consume it through their own sealed boundaries. +- `assessment_delivery` → `workforce_validation`: exposes exact procedure/scoring/admin/result coordinates required for reproducible predictor lineage. Validation owns scientific interpretation and may return `not_verifiable` without changing Assessment Delivery history. +- Keyverse → product/auth journey: identity and authorization backend only; Assessment Delivery receives authorized opaque subject/actor evidence rather than owning identity truth. + +No context reads another context's application tables. Physical PostgreSQL co-location does not change this rule. + +## Invariants + +1. An assignment binds tenant, accountable actor, purpose, opaque subject reference, process/Job context where applicable, procedure owner, released procedure version, scoring owner/version where applicable, requested validity window, policy version and immutable provenance. +2. An administration occurrence has stable semantic identity independent of transport retries. Idempotency keys bind retries to that identity; another intentional administration requires a new occurrence. +3. A result reference binds the same tenant, assignment, administration occurrence, procedure/scoring versions, external owner locator and integrity evidence. Wrong-subject, wrong-purpose, wrong-version, wrong-occurrence or floating evidence fails closed. +4. External result content is not copied by default. Any future materialization requires an explicit purpose/data-minimization ADR and does not transfer owner authority. +5. Provider/session status cannot finalize an HR decision or establish validity/fairness. Downstream consumers independently authorize and resolve exact released evidence. +6. Missing, unavailable, interrupted, invalidated, expired or unverifiable evidence remains non-authorizing and non-numeric unless the external owner contract itself defines a legitimate released result. +7. Cancellation and expiry cannot erase a started administration or received historical evidence. Corrections/rescoring create a successor with predecessor linkage and preserve the exact evidence used by prior decisions/studies. +8. Accommodation/accessibility provenance records only what is required to verify correct administration and routing. Sensitive detail remains with the appropriate privacy/assessment owner and is never copied into generic audit narratives. +9. Audit/outbox evidence for material lifecycle transitions is immutable, tenant-scoped and correlated. Long external/LLM/scoring work never runs while an Orgmetra database transaction or explicit lock is held. +10. `assessment_delivery` does not compute psychometric scores, select cut scores, infer protected attributes, decide fairness/validity, or recommend/finalize employment outcomes. + +## Transaction and persistence boundary + +If the Proposed decision proceeds to implementation, the context receives a separate owned schema/role. Aggregate mutations use short transactions: validate/re-resolve local current state, perform one idempotent append/transition, write correlated audit/outbox evidence, commit, then perform external transport outside the transaction. Provider calls and scoring wait states cannot hold database locks. + +Inbox/webhook processing is idempotent and binds external messages to an existing exact assignment/administration coordinate before accepting a state transition. UPSERT may be used only where the domain idempotency key defines one semantic fact; it must not collapse genuinely distinct administrations or overwrite historical results. + +## RED acceptance before implementation can become protected truth + +Executable tests must fail at least for: + +1. mutable/floating procedure or scoring references; +2. a digest without a released external owner locator/version; +3. duplicate administration from transport retry; +4. collapse of two legitimate administrations for one subject; +5. callback/result evidence for the wrong tenant, subject, assignment, occurrence, purpose, Job/process or effective window; +6. provider/scoring contract drift between assignment and result without explicit historical version binding; +7. unavailable/interrupted/invalidated/not-verifiable evidence converted to zero score or normal completion; +8. result correction/rescoring that overwrites predecessor evidence; +9. provider completion directly advancing/rejecting/finalizing a candidate or worker decision; +10. cross-context SQL, source copying or mutable branch dependency; +11. accommodation/accessibility evidence omitted when required for administration provenance, or unrestricted sensitive detail copied into this context; +12. Workforce Validation unable to resolve the exact assessment/scoring/admin/result evidence used as a predictor; +13. an external call or scoring wait performed inside a long-lived Orgmetra database transaction/lock; +14. an AI-based assessment result whose development/scoring/version/use provenance is insufficient for verification/audit being represented as verified. + +Positive acceptance must prove one semantic assignment and administration across retry/replay, append-only result supersession, tenant/purpose isolation, purpose-minimized audit/outbox, released-contract ACLs, right-cleared provider E2E data, and downstream reconstruction by the governed selection and Workforce Validation boundaries. + +## Scientific and standards basis + +ISO 10667-1:2020 remains the published client-side standard and covers work-related assessment use including recruitment, development, appraisal, promotion, succession and reassignment. ISO 10667-2:2020 remains the published service-provider counterpart. Both Edition 3 projects are under development at stage 20.00 as of the protected review date; they are change-watch inputs, not released normative replacements. + +SIOP's *Principles for the Validation and Use of Personnel Selection Procedures* and its recommendations for AI-based assessments reinforce job-relatedness, score consistency, fairness, appropriate use and documented development/scoring decisions. AERA, APA, and NCME's *Standards for Educational and Psychological Testing* remains supporting measurement authority. These sources justify provenance and scientific separation; they do not certify a provider, procedure, score, cut score or deployment. + +Peer-reviewed validity evidence in the reference note is used only to reinforce versioned procedure/use-context and restriction-of-range/criterion interpretation concerns. `workforce_validation` remains the scientific owner. + +## Consequences + +### Positive + +- Recruiting and post-hire assessment can share one operational Ubiquitous Language without turning `talent_acquisition` into a lifecycle-spanning catch-all. +- Provider transport is separated from business assessment state. +- External scientific/scoring ownership stays explicit while Orgmetra retains enough immutable provenance for accountable decisions and later validation. +- Retry, correction and unverifiable states become domain-visible instead of being inferred from callbacks. + +### Costs and risks + +- A new bounded context adds service/schema/API/event/release overhead. +- The exact provider/scoring ACL cannot be finalized until a released external owner contract exists. +- Post-hire consumers remain dependent on their own accepted owner boundaries; this ADR does not make proposed `talent_management` protected truth. +- ISO 10667 Edition 3 work must be monitored and the ADR revisited if published requirements materially change the boundary. + +## Completion boundary + +This ADR remains Proposed until `assessment_delivery` reaches executable protected truth with code-current Context Map/UL/aggregate/invariants, released provider/scoring ACLs, normalized schema and migrations, purpose-bound API/events, idempotency/correction/recovery, immutable audit/outbox, SECURITY/THREAT_MODEL/OPERABILITY/TEST_STRATEGY updates, right-cleared E2E evidence, applicable p95 evidence, 100% owned production docstring/test/edge coverage, and reproducible downstream selection and #425 Workforce Validation consumption. + +`docs/product-technical-gap-baseline.md` remains under PR #100 single-writer ownership. This ADR does not mark Assessment delivery as shipped capability. \ No newline at end of file From a566c5cfdd30eab71badad147fe91c1c2e476f6d Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 21 Sep 2026 03:02:34 +0900 Subject: [PATCH 2/7] docs(assessment): add standards and scientific traceability --- ...nt-assignment-result-handoff-references.md | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 docs/doctoring/assessment-assignment-result-handoff-references.md diff --git a/docs/doctoring/assessment-assignment-result-handoff-references.md b/docs/doctoring/assessment-assignment-result-handoff-references.md new file mode 100644 index 000000000..1182acad8 --- /dev/null +++ b/docs/doctoring/assessment-assignment-result-handoff-references.md @@ -0,0 +1,50 @@ +# Assessment assignment and result-handoff references + +This note supports ADR 0429. It records the authority used to separate Orgmetra assessment-delivery orchestration from external instrument/scoring ownership and from downstream validity/fairness authority. It does not certify an assessment provider, instrument, scoring model, cutoff, legal conclusion, or deployment. + +## Current standards status reviewed 2026-09-21 + +International Organization for Standardization. (2020a). *Assessment service delivery—Procedures and methods to assess people in work and organizational settings—Part 1: Requirements for the client* (ISO 10667-1:2020). https://www.iso.org/standard/74716.html + +ISO 10667-1:2020 is the published client-side edition. Its public scope covers the client's needs/rationale, conditions of use, assessment approach, access/use/storage of results, and organizational decisions. Its examples include recruitment, selection, development, appraisal, promotion, succession planning, and reassignment. This is the strongest standards basis for treating assessment assignment/use as a business lifecycle rather than generic connector state. + +International Organization for Standardization. (2020b). *Assessment service delivery—Procedures and methods to assess people in work and organizational settings—Part 2: Requirements for service providers* (ISO 10667-2:2020). https://www.iso.org/standard/74717.html + +ISO 10667-2:2020 remains the published service-provider counterpart. Its public scope includes selection/implementation/evaluation of assessment procedures, interpretation/reporting, personal and assessment data, competence/professionalism, and organizational decisions. ADR 0429 uses it to require explicit released provider/procedure/scoring provenance; Orgmetra does not infer provider compliance from a result callback. + +International Organization for Standardization. (2026a). *Assessment service delivery—Procedures and methods to assess people in work and organizational settings—Part 1: Requirements for the client* (ISO/AWI 10667-1, Edition 3, work item). https://www.iso.org/standard/94563.html + +International Organization for Standardization. (2026b). *Assessment service delivery—Procedures and methods to assess people in work and organizational settings—Part 2: Requirements for service providers* (ISO/AWI 10667-2, Edition 3, work item). https://www.iso.org/standard/94564.html + +Both Edition 3 work items were registered on 2026-05-27 and remain under development at stage 20.00 at this review. They are change-watch evidence only. ADR 0429 must be rechecked when either replacement reaches publication or exposes a material requirement that changes the client/provider boundary. + +## Professional standards and guidance + +American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). *Standards for educational and psychological testing*. American Educational Research Association. + +Society for Industrial and Organizational Psychology. (2018). *Principles for the validation and use of personnel selection procedures* (5th ed.). Author. + +Society for Industrial and Organizational Psychology. (2023). *Considerations and recommendations for the validation and use of AI-based assessments for employee selection*. https://www.siop.org/post/siop-releases-recommendations-for-ai-based-assessments/ + +The SIOP AI-assessment recommendations explicitly emphasize job-related score meaning, score consistency, fairness, appropriate operational use, and documentation of development/scoring steps for verification and audit. ADR 0429 uses those principles only to justify immutable procedure/scoring/version/use provenance and fail-closed `not_verifiable` behavior. Scientific adequacy remains a Workforce Validation responsibility. + +## Peer-reviewed evidence + +Berry, C. M., Lievens, F., Zhang, C., & Sackett, P. R. (2024). Insights from an updated personnel selection meta-analytic matrix: Revisiting general mental ability tests' role in the validity–diversity trade-off. *Journal of Applied Psychology, 109*(10), 1611–1634. https://doi.org/10.1037/apl0001203 + +Merritt, S. M., & Ryan, A. M. (2024). Gendered competencies and gender composition: A human versus algorithm evaluator comparison. *International Journal of Selection and Assessment, 32*(2), 225–248. https://doi.org/10.1111/ijsa.12459 + +Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. *Journal of Applied Psychology, 107*(11), 2040–2068. https://doi.org/10.1037/apl0000994 + +These papers reinforce that assessment interpretation depends on the procedure, scoring/use context, criterion and sampling/selection conditions. They do not justify a universal assessment score, cutoff, validity coefficient, diversity conclusion, or automated employment decision. + +## Architectural interpretation + +The reference set supports four narrow decisions in ADR 0429: + +1. assessment use is cross-lifecycle business state, so it should not be hidden as generic integration transport; +2. Orgmetra needs exact purpose, procedure/scoring contract version, administration occurrence, result-owner locator and correction provenance to support accountable consumption; +3. operational assessment delivery does not imply ownership of instrument content, item/response data, scoring algorithms, or psychometric conclusions; and +4. downstream selection and Workforce Validation must independently authorize and reconstruct the exact evidence they consume. + +Any future implementation must doctor new jurisdiction-specific legal requirements separately. These professional/measurement sources are not a substitute for legal advice or tenant-specific compliance policy. \ No newline at end of file From 2dac716e367d9844991c41e631da9285a3001fe2 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 21 Sep 2026 03:03:13 +0900 Subject: [PATCH 3/7] docs(assessment): add owner-boundary traceability --- ...ment-assignment-result-handoff-boundary.md | 118 ++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 docs/traceability/assessment-assignment-result-handoff-boundary.md diff --git a/docs/traceability/assessment-assignment-result-handoff-boundary.md b/docs/traceability/assessment-assignment-result-handoff-boundary.md new file mode 100644 index 000000000..8d9eb3a8e --- /dev/null +++ b/docs/traceability/assessment-assignment-result-handoff-boundary.md @@ -0,0 +1,118 @@ +# Assessment assignment and result-handoff boundary traceability + +Status: Proposed design evidence for ADR 0429 / Issue #429. Nothing in this document is shipped assessment-delivery capability until the corresponding executable owner contract reaches protected truth. + +Protected baseline reviewed: `develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f`. + +## Current protected authority + +| Protected authority | Existing truth | Assessment-delivery consequence | +|---|---|---| +| `docs/PRD.md` | Orgmetra spans the employment lifecycle; selection evidence and later validity evidence must remain reconstructable | Assessment evidence must retain exact business purpose and downstream decision/study provenance rather than becoming an opaque provider callback | +| `docs/TRD.md` | Assessment results remain external immutable snapshot references unless a later ADR transfers instrument lifecycle ownership | ADR 0429 does not transfer instrument, item-bank, response, scoring-model or psychometric-result ownership | +| `ARCHITECTURE.md` | `talent_acquisition`, `performance_management`, `workforce_validation`, `integration_hub` and other contexts have explicit separate schema/API authority; no assessment context exists | A new context is only Proposed; no existing service may silently claim assessment lifecycle state in the meantime | +| `scripts/foundation-contract-core.mjs` | `assessment_assignment` is already part of the canonical logical database-object vocabulary | The product concept exists, but its aggregate/schema/API ownership must not be inferred from the inventory token | +| `talent_acquisition` boundary | Owns requisitions, candidates, interviews, decision evidence and selection decisions | It may request assessment delivery through an ACL but does not own cross-lifecycle administration state or scientific scoring truth | +| `workforce_validation` boundary | Owns validity studies, exact evidence/outcome linkage, subgroup diagnostics, drift, utility and scientific adapters | It consumes exact released assessment/scoring/admin/result coordinates and remains authoritative for validity/fairness/scientific interpretation | +| `integration_hub` boundary | Owns adapters, inbox/outbox and transport/migration state | Provider transport may be implemented through adapters, but business assessment assignment/admin/result lifecycle is not connector state | +| Keyverse boundary | Identity/authentication backend | Assessment Delivery receives purpose-authorized opaque subject/actor evidence; it does not become an identity store | + +## Proposed Context Map + +```mermaid +flowchart LR + TA[talent_acquisition] -->|released request ACL| AD[assessment_delivery - Proposed] + TM[future protected talent_management] -.->|released request ACL if adopted| AD + JA[job_architecture] -->|released Job/FJA/KSAO evidence refs| AD + AD -->|versioned provider adapter| EXT[external assessment/scoring owner] + EXT -->|immutable result locator + version + integrity| AD + AD -->|non-authorizing result evidence| TA + AD -->|exact predictor coordinates| WV[workforce_validation] + KV[Keyverse] -. identity/authz backend .-> AD + AD -->|audit/outbox| AP[audit_provenance] +``` + +Dashed/future edges are not current product authority. Physical database co-location never permits cross-context application SQL. + +## Proposed Ubiquitous Language + +| Term | Proposed meaning | Explicit non-meaning | +|---|---|---| +| `AssessmentAssignment` | accountable business request for one purpose/subject/process and exact released assessment procedure contract | provider session, score, selection decision | +| `AssessmentAdministration` | one actual administration occurrence under an assignment | transport retry, assessment instrument definition | +| `provider_session_reference` | external transport/provider coordinate associated with an administration | canonical administration identity by itself | +| `AssessmentResultReference` | immutable external owner locator bound to one administration and exact procedure/scoring contract versions | copied provider payload, locally authoritative score | +| `AssessmentResultSupersession` | append-only predecessor/successor lineage for correction/rescore | in-place result mutation | +| `AssessmentDeliveryPolicy` | operational expiry/idempotency/permitted-purpose/evidence requirements | scoring model, cut score, validity or fairness policy | +| `not_verifiable` | required owner/version/provenance cannot be reconstructed sufficiently for authorized use | zero score, failure-to-pass, neutral scientific result | + +## Proposed aggregate and transaction boundary + +`AssessmentAssignment` is the aggregate root. One command transaction may validate local current state, append one idempotent assignment/admin/result-reference transition, write correlated audit/outbox evidence and commit. External provider/scoring calls happen after commit and never while an Orgmetra transaction or explicit database lock waits for remote/LLM/scoring work. + +A provider callback enters through an idempotent inbox/adapter boundary, resolves one exact tenant/assignment/administration coordinate, validates released contract identity and purpose, and then performs one short domain transition. The callback cannot call a downstream selection/promotion/fairness finalizer in the same authority boundary. + +## Ownership matrix + +| Fact / behavior | Proposed owner | Consumer(s) | Forbidden shortcut | +|---|---|---|---| +| assessment business request/purpose | `assessment_delivery` | Talent/HR workflows, audit | generic connector metadata as sole authority | +| administration occurrence identity/status | `assessment_delivery` | requester, audit, validation | provider session ID as sole semantic identity | +| instrument/item/content lifecycle | external assessment owner | Assessment Delivery reference only | source copy into Orgmetra | +| response data | external/privacy owner by contract | purpose-bound only if explicitly released | unrestricted HRIS replication | +| scoring algorithm/model | external scoring owner | reference by exact released version | local floating model alias | +| result evidence | external result owner; Orgmetra stores immutable reference | selection/Talent/validation through ACL | digest-only or mutable URL as authority | +| Job/FJA/KSAO job-related evidence | `job_architecture` | Assessment Delivery / selection / validation | copied job semantics | +| employment decision | governed selection or later protected Talent owner | audit, validation | assessment callback finalizes decision | +| validity/fairness/adverse impact | `workforce_validation` | decision governance/reporting | provider completion or raw score treated as scientific GREEN | +| adapter retry/webhook transport | `integration_hub` / released adapter implementation | Assessment Delivery | connector owns business lifecycle | +| identity/authentication | Keyverse | all authorized consumers | Assessment Delivery identity shadow table | + +## Contract requirements before implementation may be accepted + +A released Assessment Delivery contract must preserve at minimum: + +- `tenant_record_id` or equivalent tenant-scoped opaque identity; +- accountable actor and purpose; +- opaque subject reference plus process/Job context required for the purpose; +- stable semantic assignment identity and stable administration occurrence identity; +- external procedure owner and exact released procedure version; +- external scoring owner/version when scoring applies; +- requested/effective/expiry window and operational policy version; +- provider session reference only as transport provenance; +- immutable external result owner locator, result/contract version and integrity evidence; +- explicit lifecycle outcomes including cancellation, expiry, unavailable, interrupted, invalidated and not-verifiable; +- append-only correction/rescore/supersession lineage; +- accommodation/accessibility provenance sufficient to verify correct administration without duplicating unrestricted sensitive detail; +- idempotency, causation/correlation and immutable audit/outbox evidence. + +## RED → GREEN verification map + +| RED finding | Minimum GREEN evidence | +|---|---| +| floating/mutable procedure or scoring reference accepted | contract/domain tests reject unversioned, mutable or alias-only authority | +| digest without released owner locator accepted | domain tests require owner/context/version/locator + integrity evidence | +| retry creates duplicate administration | PostgreSQL/service contract proves one semantic occurrence across replay while allowing an intentional second administration | +| wrong tenant/subject/purpose/occurrence result accepted | authorization/domain tests fail closed before mutation | +| provider contract drifts between assignment and result | exact historical version binding is persisted and mismatch requires explicit supersession/repair | +| unavailable/interrupted/invalidated/not-verifiable becomes numeric/normal completion | typed-state tests prove non-authorizing semantics end to end | +| rescore overwrites prior result | append-only predecessor/successor tests preserve both historical evidence identities | +| callback advances/rejects/finalizes worker/candidate | architecture/service tests prove no direct high-impact decision command from provider callback boundary | +| external provider call runs inside DB transaction | integration test/telemetry proves commit precedes remote wait and connection/lock cleanup is bounded | +| cross-context SQL or source copy appears | architecture/static contract rejects foreign application-table/source ownership | +| Workforce Validation cannot reconstruct predictor identity | E2E resolves exact assessment/scoring/admin/result coordinates consumed by a study version | +| AI assessment lacks development/scoring/use provenance | result remains `not_verifiable`/non-authorizing until released owner evidence is resolvable | + +Synthetic fixtures are suitable for unit/contract failure tests only. Buyer/scientific acceptance requires real or right-cleared assessment-provider evidence whose use permits the asserted verification. No synthetic-only acceptance may be reported as production validity/fairness evidence. + +## Standards and scientific traceability + +The APA 7 reference set and current standards-status notes are in `docs/doctoring/assessment-assignment-result-handoff-references.md`. + +The key standards interpretation is deliberately bounded: ISO 10667-1:2020 supports a client-side lifecycle spanning multiple work-related assessment purposes; ISO 10667-2:2020 defines service-provider concerns; both Edition 3 replacements are still work items. SIOP/AERA-APA-NCME material supports provenance, job-relatedness, reliability/validity/fairness and auditability. None of these sources turns a provider callback into Orgmetra decision authority. + +## Documentation/release convergence required for completion + +If implementation proceeds, completion requires code-current updates to the canonical Context Map/Architecture, TRD, data model/ERD/UML, API/event contracts, SECURITY, THREAT_MODEL, TEST_STRATEGY, OPERABILITY/recovery and release evidence through their live single-writer owners. `docs/product-technical-gap-baseline.md` remains owned by PR #100 and must not be edited from this ADR lane. + +ADR 0429 remains Proposed until those executable contracts reach protected truth. A docs-only merge of this proposal is not sufficient to close Issue #429. \ No newline at end of file From 564ad62e2a41ab00aa25d0a4289b5d230374323e Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 21 Sep 2026 04:01:31 +0900 Subject: [PATCH 4/7] docs(assessment): record Berry validity-matrix erratum --- .../assessment-assignment-result-handoff-references.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/doctoring/assessment-assignment-result-handoff-references.md b/docs/doctoring/assessment-assignment-result-handoff-references.md index 1182acad8..31e93a628 100644 --- a/docs/doctoring/assessment-assignment-result-handoff-references.md +++ b/docs/doctoring/assessment-assignment-result-handoff-references.md @@ -32,6 +32,10 @@ The SIOP AI-assessment recommendations explicitly emphasize job-related score me Berry, C. M., Lievens, F., Zhang, C., & Sackett, P. R. (2024). Insights from an updated personnel selection meta-analytic matrix: Revisiting general mental ability tests' role in the validity–diversity trade-off. *Journal of Applied Psychology, 109*(10), 1611–1634. https://doi.org/10.1037/apl0001203 +*Correction to “Insights from an updated personnel selection meta-analytic matrix: Revisiting general mental ability tests' role in the validity-diversity trade-off” by Berry et al. (2024).* (2025). *Journal of Applied Psychology, 110*(9), 1239. https://doi.org/10.1037/apl0001308 + +The 2025 erratum corrects adverse-impact ratios for the dotted lines in Figure 1 of Berry et al. (2024). It states that the error does not change the article's conclusions. Any future Orgmetra use of numerical adverse-impact ratios from that figure must therefore use the corrected figure/erratum rather than the original Figure 1 values. + Merritt, S. M., & Ryan, A. M. (2024). Gendered competencies and gender composition: A human versus algorithm evaluator comparison. *International Journal of Selection and Assessment, 32*(2), 225–248. https://doi.org/10.1111/ijsa.12459 Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. *Journal of Applied Psychology, 107*(11), 2040–2068. https://doi.org/10.1037/apl0000994 From 25bd201ff0838abc5baf39a3f2cafe1267070a09 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 21 Sep 2026 05:07:40 +0900 Subject: [PATCH 5/7] docs(assessment): restore specialist operations ownership --- ...ment-assignment-result-handoff-boundary.md | 277 +++++++++++------- 1 file changed, 173 insertions(+), 104 deletions(-) diff --git a/docs/adr/0429-assessment-assignment-result-handoff-boundary.md b/docs/adr/0429-assessment-assignment-result-handoff-boundary.md index 506ba2c52..e63c4840d 100644 --- a/docs/adr/0429-assessment-assignment-result-handoff-boundary.md +++ b/docs/adr/0429-assessment-assignment-result-handoff-boundary.md @@ -1,157 +1,226 @@ -# ADR 0429: Assessment delivery owns assignment and result handoff, not instrument truth +# ADR 0429: Assessment assignment coordination and external result handoff -- Status: Proposed -- Date: 2026-09-21 -- Issue: #429 -- Protected baseline reviewed: `develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f` +## Status -## Problem +Status: Proposed -Orgmetra already names `assessment_assignment` in its canonical logical database-object vocabulary, and selection/validation flows require assessment evidence. Protected product truth does not, however, identify an executable owner for the business lifecycle that requests an assessment, binds it to one accountable HR purpose and subject, tracks an administration occurrence, and accepts an immutable result reference from an external assessment/scoring owner. +Date: 2026-09-21 -The existing context map cannot safely absorb this by implication. `talent_acquisition` owns requisitions, candidates, interviews, decision evidence and selection decisions. `performance_management` owns performance criteria and observations. `workforce_validation` owns validity studies, subgroup diagnostics, drift, utility and scientific adapters. `integration_hub` owns transport/adapters, not business assessment state. Protected TRD also deliberately keeps assessment results as external immutable snapshot references unless a later ADR transfers instrument lifecycle ownership. +Issue: #429 -This leaves two failure modes. Treating assessment as generic integration state loses accountable business purpose, administration identity, cancellation/expiry/correction and decision provenance. Treating an assessment callback as Talent or Validation truth risks silently importing instrument, scoring and psychometric authority into Orgmetra. +Protected Orgmetra baseline reviewed: `develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f` -## Constraints +## Context -1. Orgmetra remains authoritative for HR domain truth, but does not acquire assessment instrument, item-bank, response, scoring-algorithm or psychometric-model lifecycle authority through this ADR. -2. External assessment/scoring owners are consumed only through released, versioned contracts. Mutable branches, floating model labels and direct foreign-table access are prohibited. -3. Job/FJA/KSAO job-relatedness remains `job_architecture` truth. Selection authority remains in the governed selection-decision boundary. Validity, fairness, adverse-impact, estimand, transportability and scientific interpretation remain `workforce_validation` responsibilities. -4. Keyverse remains the identity/authentication backend. Subject, accommodation and accessibility data crossing the assessment boundary must be purpose-minimized. -5. Assessment results remain immutable external owner references plus exact contract/version/integrity evidence by default. A digest without a resolvable owner locator is insufficient provenance. -6. Transport completion, HTTP success, provider session completion or receipt delivery cannot authorize a selection, promotion, development, succession or other high-impact employment outcome. -7. Retry/replay must not create a second semantic assignment or administration. A genuinely new administration remains a distinct occurrence even for the same person and procedure. -8. `unavailable`, `interrupted`, `invalidated`, `expired` and `not_verifiable` are explicit non-success states; none may be coerced into a numeric score or normal completion. -9. Rescoring, correction and supersession are append-only. Historical evidence used by a released decision or validation study is not overwritten. -10. This Proposed ADR does not authorize a schema, service, API, event, UI or migration before executable RED→GREEN evidence is reviewed against then-current protected truth. +Orgmetra needs durable HR-side truth that an accountable business process requested an assessment for a particular purpose and later consumed an exact assessment result as evidence. The canonical logical-object vocabulary already names `assessment_assignment`, while Workforce Validation needs an exact procedure/scoring/result coordinate when an assessment is used as a predictor. -## Alternatives considered +That need does **not** transfer assessment operations into Orgmetra. Protected Orgmetra authority is explicit: -### A. `talent_acquisition` owns assessment orchestration +- `CLAUDE.md` assigns assessment operations and immutable assessment result snapshots to Psychometrics Commons; +- the protected README assigns assessment lifecycle to Psychometrics Commons and psychometric computation to specialist numerical owners; +- Accepted ADR 0001 keeps assessment operations behind the Psychometrics Commons specialist boundary and has Orgmetra store references to specialist artifacts; and +- the protected TRD keeps assessment results as external immutable snapshot references unless a later accepted ADR explicitly transfers instrument lifecycle ownership. -This is attractive for recruitment and selection because candidate assessment is adjacent to requisition/application/selection evidence. It is rejected as the general owner because ISO 10667 assessment use spans recruitment, development, appraisal, promotion, succession and reassignment. Expanding `talent_acquisition` across those post-hire lifecycles would distort its Ubiquitous Language and create pressure to copy worker/Talent state into an acquisition context. +Psychometrics Commons protected-main authority independently owns product APIs, participant/session lifecycle, response events, scoring dispatch, immutable result snapshots, product persistence and resource authorization. This ADR does not supersede that boundary. -A future implementation may expose a Talent Acquisition ACL that requests an assessment, but it does not own the assessment administration lifecycle. +The predecessor form of this ADR proposed an Orgmetra `assessment_delivery` context with a local `AssessmentAdministration` entity and local result-supersession lifecycle. Fresh authority review proved that design was too broad: it duplicated the specialist's assessment-operations authority even though instrument and scoring ownership were described as external. The repair below narrows Orgmetra to **assessment coordination**: HR business intent, purpose, correlation, released external references and accountable consumption. -### B. dedicated `assessment_delivery` bounded context +A separate integration constraint is currently material. At this review, `ContextualWisdomLab/psychometrics-commons` has no published GitHub Release. Protected-main source or an immutable commit can inform design and review, but it is not a released production dependency under the CWL owner-contract rule. Executable Orgmetra integration therefore remains fail-closed until the owner publishes the required immutable contract and Orgmetra consumes that release through an ACL. -Selected as the product-scope direction. `assessment_delivery` owns the operational lifecycle of an assessment request/assignment and administration occurrence plus immutable external result-handoff evidence. It does not own the instrument, items, responses, scoring model, validity/fairness conclusions or employment decision. +## Decision drivers -This isolates a business lifecycle that legitimately spans recruiting and post-hire use while keeping scientific authority external and preserving explicit ACLs to `talent_acquisition`, a future protected `talent_management` context if adopted, `job_architecture`, and `workforce_validation`. +The boundary must: -The cost is another bounded context, schema/API/event lifecycle and release identity. That cost is accepted provisionally because the lifecycle and invariants are domain state, not merely connector state, and because forcing it into acquisition creates a cross-lifecycle semantic mismatch. +- preserve Orgmetra's HR/employment-process truth without turning a specialist service into the HR system of record; +- preserve Psychometrics Commons ownership of assessment sessions, responses, scoring dispatch, result snapshots and their correction/supersession lifecycle; +- support recruitment, selection, development, appraisal, promotion, succession and reassignment without forcing all assessment use into `talent_acquisition`; +- keep external calls outside database transactions and explicit locks; +- make retries and duplicate callbacks replay-safe without conflating genuinely distinct assessment executions; +- preserve exact procedure, instrument, scoring, calibration/norm and result-snapshot provenance when the released owner contract exposes those coordinates; +- prevent callback success, HTTP success, provider completion or score presence from becoming employment-decision, validity or fairness authority; +- minimize PII and keep accommodation/accessibility evidence purpose-bound; and +- allow Workforce Validation to reconstruct the exact released evidence consumed by a study. -### C. `integration_hub` owns assessment orchestration +## Considered options -Rejected except for connector/session transport mechanics. Provider authentication, delivery retries, webhook transport and inbox/outbox adapter state may live behind `integration_hub`, but business assignment purpose, administration occurrence, cancellation/expiry, result acceptance and supersession are not generic integration facts. +### Option A — Put all assessment assignment state in `talent_acquisition` -### D. assessment assignment remains entirely external +Rejected as the general owner. Recruitment and selection are only some assessment-use cases. Promotion, development, succession, reassignment and other post-hire uses would either leak into a pre-hire context or require duplicate ownership. -Rejected as the current product direction. It would require Orgmetra to give up governed assignment/administration provenance while selection and Workforce Validation still depend on exact assessment evidence. If future product scope contracts, this option must be adopted through a new ADR and buyer-facing product/documentation changes rather than by leaving an ambiguous logical object in place. +`talent_acquisition` may request an assessment and later consume authorized evidence through a released Orgmetra contract. It does not own the cross-lifecycle assessment-assignment truth. -## Decision +### Option B — Dedicated `assessment_coordination` bounded context -Create a dedicated `assessment_delivery` bounded context as the provisional product/domain owner for assessment assignment and immutable result handoff. +**Selected as the Proposed direction.** -The context owns only these concepts: +`assessment_coordination` owns the HR-side business intent and evidence correlation required to say: -- `AssessmentAssignment`: one tenant/purpose/subject/process request for an exact released assessment procedure contract. -- `AssessmentAdministration`: one actual administration occurrence under an assignment, with explicit occurrence identity and lifecycle. -- `AssessmentResultReference`: an immutable external owner locator bound to exact assessment/scoring contract versions, administration occurrence and integrity evidence. -- `AssessmentResultSupersession`: append-only correction/rescore/supersession lineage. -- `AssessmentDeliveryPolicy`: versioned operational rules for expiry, permitted purposes, retry/idempotency and required evidence; it is not a scoring or psychometric policy. +- who/what business process requested an assessment under an authorized purpose; +- which Job/FJA/KSAO, candidate/worker/process or policy coordinates were in scope; +- which exact released Psychometrics Commons contract was requested/consumed; +- which opaque external assessment-execution reference was bound to the assignment; and +- which immutable external result snapshot reference was later admitted as evidence. -The aggregate boundary is `AssessmentAssignment`. It may create or admit administration occurrences only under a versioned assignment policy. Result references attach to one exact administration occurrence and cannot mutate an earlier result in place. +It does **not** own assessment administration/session lifecycle, item or response evidence, instrument publication, scoring dispatch, scoring-model lifecycle, psychometric computation, result-snapshot construction, or result correction/supersession. Those remain Psychometrics Commons / specialist owner truth. -The Ubiquitous Language deliberately distinguishes: +### Option C — Put business assignment state in `integration_hub` -- **assignment** — accountable business request to perform a procedure; -- **administration** — one actual occurrence of delivering that procedure; -- **provider session** — transport/provider implementation coordinate, not the domain identity by itself; -- **result reference** — immutable pointer to external result evidence; -- **scoring contract** — released external owner contract defining how a result was produced; -- **selection/HR decision** — downstream human-governed decision outside this context; -- **validity/fairness evidence** — downstream `workforce_validation` scientific authority. +Rejected. `integration_hub` may own adapter transport, inbox/outbox and delivery mechanics, but a generic connector state cannot be the authoritative HR record that an assessment was requested for a business purpose and later admitted into a governed employment process. -## Context map +### Option D — Make Psychometrics Commons own both assessment operations and the HR business assignment -- `talent_acquisition` → `assessment_delivery`: requests recruitment/selection assignments through a released ACL; receives non-authorizing status/result-reference events. It never treats callback completion as a selection decision. -- future protected `talent_management` → `assessment_delivery`: if that bounded context becomes protected truth, it may request development/promotion/succession assessments through the same released ACL without importing acquisition semantics. -- `job_architecture` → `assessment_delivery`: supplies released Job/FJA/KSAO evidence references required to establish purpose/job-related context. Assessment Delivery stores opaque/versioned references only. -- `assessment_delivery` → external provider/scoring owner: uses released/versioned APIs/events via an integration adapter. Instrument/item/response/scoring internals remain foreign authority. -- `assessment_delivery` → `talent_acquisition` / future Talent consumers: publishes purpose-authorized immutable result-reference evidence; downstream high-impact decisions re-authorize and consume it through their own sealed boundaries. -- `assessment_delivery` → `workforce_validation`: exposes exact procedure/scoring/admin/result coordinates required for reproducible predictor lineage. Validation owns scientific interpretation and may return `not_verifiable` without changing Assessment Delivery history. -- Keyverse → product/auth journey: identity and authorization backend only; Assessment Delivery receives authorized opaque subject/actor evidence rather than owning identity truth. +Rejected as the complete boundary. Psychometrics Commons owns the assessment operation, but it must not become the authoritative employment-process ledger. Orgmetra must retain the business-purpose assignment and downstream employment-decision/evidence linkage while consuming only released specialist contracts. -No context reads another context's application tables. Physical PostgreSQL co-location does not change this rule. +## Ubiquitous language and aggregate boundary + +### `AssessmentAssignment` — Aggregate Root + +An Orgmetra-owned HR business intent to request or consume an external assessment for a governed purpose. + +The aggregate binds, at minimum where applicable: + +- tenant scope; +- a stable assignment identity; +- purpose and reason vocabulary; +- accountable requesting actor/process evidence; +- candidate, Employment, Job, Position, Talent-process or other HR scope through purpose-minimized governed references rather than copied foreign truth; +- requested/effective/expiry window; +- exact assignment-policy version; +- required external owner/contract identity; and +- append-only external execution/result bindings admitted under that assignment. + +The assignment lifecycle is **HR coordination state**, not the external assessment-session state. Orgmetra must not mirror provider session transitions as if they were local business truth. A local assignment may be requested, cancelled or expired independently of whether an external session was started; externally observed execution/result state is carried only in owner-issued references/evidence. + +### `AssessmentExecutionReference` — Value Object + +A purpose-minimized reference to an assessment execution/session owned by Psychometrics Commons. It binds only fields supplied by a **released** owner contract, such as: + +- owner/service and contract identity; +- exact contract/schema version; +- opaque execution/session/administration reference; +- exact assessment-spec/instrument/procedure coordinate when exposed; +- exact owner-issued status/evidence coordinate when exposed; and +- owner-issued integrity/provenance evidence where the contract defines it. + +Orgmetra does not create, mutate or infer the external session lifecycle from this value. + +### `AssessmentResultSnapshotReference` — Value Object + +An immutable reference to a Psychometrics Commons result snapshot. It binds the released owner contract and the exact immutable snapshot coordinates needed for later verification, including the owner locator, schema/version identity, content/artifact digest when defined, and exact instrument/scoring/calibration/norm/output coordinates exposed by the owner. + +A digest alone is not a locator. A provider label or floating model alias is not a versioned result coordinate. + +### `AssessmentResultBinding` — Entity + +An append-only Orgmetra record that one exact external result snapshot reference was admitted under one `AssessmentAssignment` for one governed purpose. A later owner-issued correction or superseding result creates another binding that points to the new immutable owner snapshot and, when supplied by the owner contract, its supersedes relation. Orgmetra never rewrites the old binding and never manufactures the specialist's supersession semantics. + +### `AssessmentAssignmentPolicy` — Domain policy/value contract + +A versioned Orgmetra policy that governs the HR-side allowed purpose, requester/process scope, assignment window, expected evidence class and downstream consumption constraints. It does not define instrument content, scoring rules, assessment-session transitions or psychometric quality. + +## Context Map + +| Context / owner | Authority | Relationship | +|---|---|---| +| `assessment_coordination` (Orgmetra) | HR assessment assignment intent, purpose, correlation and append-only external evidence binding | Proposed owner | +| Psychometrics Commons | assessment APIs, instrument publication, participant/session lifecycle, responses, scoring dispatch, immutable result snapshots, snapshot correction/supersession, persistence and resource authorization | External upstream specialist; consume released contract through ACL | +| `talent_acquisition` | recruiting/selection workflow and governed decision evidence | Requests/consumes assessment coordination through released Orgmetra boundary | +| future protected `talent_management` if adopted | development/succession/internal-mobility process truth | Requests/consumes assessment coordination through released Orgmetra boundary | +| `performance_management` | performance criterion/observation truth | May request/consume evidence for a governed purpose; does not own assessment operations | +| `job_architecture` | Job/FJA/KSAO truth | Referenced through released/versioned Orgmetra truth | +| `workforce_validation` | validity/fairness/scientific study design and interpretation | Consumes exact released assessment/result coordinates and Orgmetra assignment lineage | +| `integration_hub` | transport adapter/inbox/outbox mechanics where used | ACL/transport only, never business-lifecycle owner | +| Keyverse | identity/authentication/authorization backend | Identity/policy dependency; no copied credential truth | +| `fast-mlsirm` | reusable psychometric numerical kernels | Specialist numerical owner, normally reached through its owning released product contract | + +No context may query another service's application tables. Physical database co-location does not change ownership. ## Invariants -1. An assignment binds tenant, accountable actor, purpose, opaque subject reference, process/Job context where applicable, procedure owner, released procedure version, scoring owner/version where applicable, requested validity window, policy version and immutable provenance. -2. An administration occurrence has stable semantic identity independent of transport retries. Idempotency keys bind retries to that identity; another intentional administration requires a new occurrence. -3. A result reference binds the same tenant, assignment, administration occurrence, procedure/scoring versions, external owner locator and integrity evidence. Wrong-subject, wrong-purpose, wrong-version, wrong-occurrence or floating evidence fails closed. -4. External result content is not copied by default. Any future materialization requires an explicit purpose/data-minimization ADR and does not transfer owner authority. -5. Provider/session status cannot finalize an HR decision or establish validity/fairness. Downstream consumers independently authorize and resolve exact released evidence. -6. Missing, unavailable, interrupted, invalidated, expired or unverifiable evidence remains non-authorizing and non-numeric unless the external owner contract itself defines a legitimate released result. -7. Cancellation and expiry cannot erase a started administration or received historical evidence. Corrections/rescoring create a successor with predecessor linkage and preserve the exact evidence used by prior decisions/studies. -8. Accommodation/accessibility provenance records only what is required to verify correct administration and routing. Sensitive detail remains with the appropriate privacy/assessment owner and is never copied into generic audit narratives. -9. Audit/outbox evidence for material lifecycle transitions is immutable, tenant-scoped and correlated. Long external/LLM/scoring work never runs while an Orgmetra database transaction or explicit lock is held. -10. `assessment_delivery` does not compute psychometric scores, select cut scores, infer protected attributes, decide fairness/validity, or recommend/finalize employment outcomes. +1. **Released owner contract only.** Production integration rejects mutable branch/PR/main source dependencies. A commit SHA may be design evidence but is not a substitute for an immutable published owner release under this repository's integration rule. +2. **No local assessment-operation ownership.** Orgmetra does not create a local `AssessmentAdministration`/session aggregate, response ledger, scoring dispatch lifecycle or result-snapshot lifecycle. +3. **Assignment purpose is explicit.** An assessment execution/result cannot be bound without an exact tenant, assignment, purpose/policy and accountable process/actor coordinate. +4. **Replay is not a new assignment.** Replaying the same semantic assignment/request identity must not create another local assignment or outbound intent. +5. **Distinct executions remain distinct.** Two genuinely distinct Psychometrics Commons executions must never be deduplicated merely because instrument, person/process scope or numeric scores match. +6. **External completion is non-authorizing.** Callback delivery, HTTP 2xx, external `completed`, or a present score cannot approve/reject/advance an employment decision and cannot establish validity or fairness. +7. **Non-success stays explicit.** `unavailable`, `interrupted`, `invalidated`, `not_verifiable` or equivalent owner states must not be coerced into zero, missing-at-random, pass, fail or normal completion. +8. **Bindings are append-only.** An owner-issued correction/rescore/superseding snapshot creates a new local binding. Historical bindings remain reconstructable as recorded. +9. **No local scientific reconstruction.** Orgmetra must not rebuild a result from raw response/provider data or infer an absent instrument/scoring/calibration/version coordinate. +10. **Purpose-bound PII.** Person/candidate/accommodation/accessibility information is retained only where required by the authorized HR purpose. Raw responses, item content, credentials and provider payloads do not enter the coordination aggregate by default. +11. **Short local transactions.** No database transaction or explicit lock waits on Psychometrics Commons or another remote provider. Local state/outbox commits complete first; remote work occurs outside the transaction; inbound evidence is admitted in a separate idempotent local transaction. +12. **No cross-context SQL/source copy.** All specialist evidence arrives through released package/API/event/adapter contracts. +13. **Downstream re-verification.** Selection/Talent/Workforce Validation must re-resolve the exact authorized assignment and released external result binding they consume. This ADR grants no downstream decision or scientific authority. + +## Transaction and idempotency model + +The `AssessmentAssignment` aggregate is the only Orgmetra transaction root proposed here. Its transaction may create/update the local coordination state, append one outbound intent/audit/outbox record, or append one verified external reference binding. It does not include remote assessment-session execution. + +Outbound dispatch uses one stable semantic request/idempotency identity bound to tenant + assignment + intended owner contract. Transport retries reuse that identity. The external owner remains responsible for its own session/admin idempotency according to its released contract. -## Transaction and persistence boundary +Inbound execution/result evidence is verified against the exact released owner contract and the intended assignment/purpose before an append-only binding is written. Conflicting replay fails closed and becomes reconciliation evidence; it does not overwrite either local or specialist truth. -If the Proposed decision proceeds to implementation, the context receives a separate owned schema/role. Aggregate mutations use short transactions: validate/re-resolve local current state, perform one idempotent append/transition, write correlated audit/outbox evidence, commit, then perform external transport outside the transaction. Provider calls and scoring wait states cannot hold database locks. +## Failure semantics -Inbox/webhook processing is idempotent and binds external messages to an existing exact assignment/administration coordinate before accepting a state transition. UPSERT may be used only where the domain idempotency key defines one semantic fact; it must not collapse genuinely distinct administrations or overwrite historical results. +- missing released Psychometrics Commons contract → integration unavailable / fail closed; +- external owner unavailable → assessment execution/result unavailable, not an invented local score/status; +- unknown or mismatched execution/result reference → quarantine/reconciliation, not automatic admission; +- external non-success → preserve exact owner semantics and downstream non-authorizing state; +- stale/mismatched assignment purpose or policy → reject the binding; +- result snapshot without owner locator/version/integrity required by contract → `not_verifiable`; +- correction/supersession without a verifiable owner chain → preserve old binding and reject authority of the new claim until reconciled. -## RED acceptance before implementation can become protected truth +## RED acceptance before implementation can be called complete -Executable tests must fail at least for: +Future executable owner work must prove at least: -1. mutable/floating procedure or scoring references; -2. a digest without a released external owner locator/version; -3. duplicate administration from transport retry; -4. collapse of two legitimate administrations for one subject; -5. callback/result evidence for the wrong tenant, subject, assignment, occurrence, purpose, Job/process or effective window; -6. provider/scoring contract drift between assignment and result without explicit historical version binding; -7. unavailable/interrupted/invalidated/not-verifiable evidence converted to zero score or normal completion; -8. result correction/rescoring that overwrites predecessor evidence; -9. provider completion directly advancing/rejecting/finalizing a candidate or worker decision; -10. cross-context SQL, source copying or mutable branch dependency; -11. accommodation/accessibility evidence omitted when required for administration provenance, or unrestricted sensitive detail copied into this context; -12. Workforce Validation unable to resolve the exact assessment/scoring/admin/result evidence used as a predictor; -13. an external call or scoring wait performed inside a long-lived Orgmetra database transaction/lock; -14. an AI-based assessment result whose development/scoring/version/use provenance is insufficient for verification/audit being represented as verified. +- Orgmetra cannot persist a locally owned assessment-session/administration lifecycle or raw response/scoring-result payload as coordination truth; +- a mutable Psychometrics Commons branch/main/PR reference is rejected as production authority; +- identical semantic assignment replay does not create another assignment/outbound intent; +- two distinct owner execution references remain distinct; +- wrong tenant, subject/process, purpose, assignment, procedure/instrument/scoring/version or owner-result coordinate fails closed; +- external `completed`/callback success cannot advance/reject/finalize a candidate or worker action; +- `unavailable`/`interrupted`/`invalidated`/`not_verifiable` cannot become numeric score, pass, fail or normal completion; +- a later owner-issued superseding snapshot appends a new binding without rewriting the earlier binding; +- raw responses, item text, credentials and unnecessary accommodation details are absent from the coordination record/event contract; +- remote owner calls occur outside database transactions/explicit locks; +- no cross-service SQL or source copy exists; and +- Workforce Validation can identify the exact Orgmetra assignment plus exact released Psychometrics Commons result/scoring coordinate used by a predictor, or must report `not_verifiable`. -Positive acceptance must prove one semantic assignment and administration across retry/replay, append-only result supersession, tenant/purpose isolation, purpose-minimized audit/outbox, released-contract ACLs, right-cleared provider E2E data, and downstream reconstruction by the governed selection and Workforce Validation boundaries. +Synthetic fixtures are suitable for unit/contract RED. Buyer/scientific acceptance still requires right-cleared real owner-contract evidence. -## Scientific and standards basis +## Standards and scientific evidence -ISO 10667-1:2020 remains the published client-side standard and covers work-related assessment use including recruitment, development, appraisal, promotion, succession and reassignment. ISO 10667-2:2020 remains the published service-provider counterpart. Both Edition 3 projects are under development at stage 20.00 as of the protected review date; they are change-watch inputs, not released normative replacements. +The reference doctoring note records ISO 10667-1:2020 and ISO 10667-2:2020 as the current published client/provider editions reviewed for this decision, with Edition 3 work items at stage 20.00 as change-watch evidence. The client/provider split supports an Orgmetra HR-use/assignment contract without transferring the specialist service-provider operation into Orgmetra. -SIOP's *Principles for the Validation and Use of Personnel Selection Procedures* and its recommendations for AI-based assessments reinforce job-relatedness, score consistency, fairness, appropriate use and documented development/scoring decisions. AERA, APA, and NCME's *Standards for Educational and Psychological Testing* remains supporting measurement authority. These sources justify provenance and scientific separation; they do not certify a provider, procedure, score, cut score or deployment. +SIOP and AERA/APA/NCME material supports explicit intended use, score/procedure provenance, documentation, validity/fairness review and accountable interpretation. It does not authorize Orgmetra to duplicate the assessment engine or infer validity from a completed assessment. -Peer-reviewed validity evidence in the reference note is used only to reinforce versioned procedure/use-context and restriction-of-range/criterion interpretation concerns. `workforce_validation` remains the scientific owner. +The Berry et al. (2024) personnel-selection evidence is accompanied by the 2025 *Journal of Applied Psychology* correction (DOI `10.1037/apl0001308`). Any future numerical use of the corrected Figure 1 adverse-impact ratios must use the erratum/corrected figure, not the original erroneous ratios. ## Consequences ### Positive -- Recruiting and post-hire assessment can share one operational Ubiquitous Language without turning `talent_acquisition` into a lifecycle-spanning catch-all. -- Provider transport is separated from business assessment state. -- External scientific/scoring ownership stays explicit while Orgmetra retains enough immutable provenance for accountable decisions and later validation. -- Retry, correction and unverifiable states become domain-visible instead of being inferred from callbacks. +- Orgmetra retains reconstructable HR purpose and evidence linkage without becoming an assessment engine. +- Cross-lifecycle assessment use no longer has to live in a recruitment-only context. +- Psychometrics Commons remains the single assessment-operations/result-snapshot authority. +- Retries, corrections and result consumption can be audited without copying raw specialist data. +- Workforce Validation can bind exact HR purpose and exact specialist evidence when a released contract exists. -### Costs and risks +### Costs / risks -- A new bounded context adds service/schema/API/event/release overhead. -- The exact provider/scoring ACL cannot be finalized until a released external owner contract exists. -- Post-hire consumers remain dependent on their own accepted owner boundaries; this ADR does not make proposed `talent_management` protected truth. -- ISO 10667 Edition 3 work must be monitored and the ADR revisited if published requirements materially change the boundary. +- A released Psychometrics Commons integration contract is a hard prerequisite; the current empty release inventory blocks production consumption. +- Coordination and specialist session/result state are intentionally not one transaction; reconciliation and operator-visible degraded states are required. +- Purpose and evidence-binding policy must remain versioned as more post-hire Talent use cases arrive. +- A later explicit Accepted ADR would be required to move any assessment-operation lifecycle into Orgmetra; this Proposed ADR does not do so. -## Completion boundary +## Follow-up order -This ADR remains Proposed until `assessment_delivery` reaches executable protected truth with code-current Context Map/UL/aggregate/invariants, released provider/scoring ACLs, normalized schema and migrations, purpose-bound API/events, idempotency/correction/recovery, immutable audit/outbox, SECURITY/THREAT_MODEL/OPERABILITY/TEST_STRATEGY updates, right-cleared E2E evidence, applicable p95 evidence, 100% owned production docstring/test/edge coverage, and reproducible downstream selection and #425 Workforce Validation consumption. +1. Psychometrics Commons owner publishes an immutable released assessment execution/result-snapshot contract suitable for Orgmetra consumption, with SBOM/provenance/reproducibility/compatibility evidence. +2. This ADR is revalidated against that released contract and then-current protected Orgmetra truth. +3. Implement `assessment_coordination` domain/API/persistence test-first without local session/scoring/result ownership. +4. Add released ACL/adapter consumption, purpose-bound authorization, inbox/outbox/reconciliation and real/right-cleared E2E evidence. +5. Reconcile protected Architecture/TRD/DATA_MODEL/ERD/UML/API/SECURITY/THREAT_MODEL/TEST_STRATEGY/OPERABILITY through their canonical writers. +6. Workforce Validation consumes only protected/released assignment/result-binding truth and preserves `not_verifiable` when the exact specialist coordinate cannot be reconstructed. +7. Update `docs/product-technical-gap-baseline.md` only through PR #100 when protected/released truth actually changes. -`docs/product-technical-gap-baseline.md` remains under PR #100 single-writer ownership. This ADR does not mark Assessment delivery as shipped capability. \ No newline at end of file +A documentation merge alone does not complete #429 or authorize production capability. \ No newline at end of file From 0c35255a483d7a66c81261ab86092135df062540 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 21 Sep 2026 05:08:35 +0900 Subject: [PATCH 6/7] docs(assessment): align coordination traceability --- ...ment-assignment-result-handoff-boundary.md | 270 +++++++++++------- 1 file changed, 162 insertions(+), 108 deletions(-) diff --git a/docs/traceability/assessment-assignment-result-handoff-boundary.md b/docs/traceability/assessment-assignment-result-handoff-boundary.md index 8d9eb3a8e..cb78e46f2 100644 --- a/docs/traceability/assessment-assignment-result-handoff-boundary.md +++ b/docs/traceability/assessment-assignment-result-handoff-boundary.md @@ -1,118 +1,172 @@ -# Assessment assignment and result-handoff boundary traceability - -Status: Proposed design evidence for ADR 0429 / Issue #429. Nothing in this document is shipped assessment-delivery capability until the corresponding executable owner contract reaches protected truth. - -Protected baseline reviewed: `develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f`. - -## Current protected authority - -| Protected authority | Existing truth | Assessment-delivery consequence | -|---|---|---| -| `docs/PRD.md` | Orgmetra spans the employment lifecycle; selection evidence and later validity evidence must remain reconstructable | Assessment evidence must retain exact business purpose and downstream decision/study provenance rather than becoming an opaque provider callback | -| `docs/TRD.md` | Assessment results remain external immutable snapshot references unless a later ADR transfers instrument lifecycle ownership | ADR 0429 does not transfer instrument, item-bank, response, scoring-model or psychometric-result ownership | -| `ARCHITECTURE.md` | `talent_acquisition`, `performance_management`, `workforce_validation`, `integration_hub` and other contexts have explicit separate schema/API authority; no assessment context exists | A new context is only Proposed; no existing service may silently claim assessment lifecycle state in the meantime | -| `scripts/foundation-contract-core.mjs` | `assessment_assignment` is already part of the canonical logical database-object vocabulary | The product concept exists, but its aggregate/schema/API ownership must not be inferred from the inventory token | -| `talent_acquisition` boundary | Owns requisitions, candidates, interviews, decision evidence and selection decisions | It may request assessment delivery through an ACL but does not own cross-lifecycle administration state or scientific scoring truth | -| `workforce_validation` boundary | Owns validity studies, exact evidence/outcome linkage, subgroup diagnostics, drift, utility and scientific adapters | It consumes exact released assessment/scoring/admin/result coordinates and remains authoritative for validity/fairness/scientific interpretation | -| `integration_hub` boundary | Owns adapters, inbox/outbox and transport/migration state | Provider transport may be implemented through adapters, but business assessment assignment/admin/result lifecycle is not connector state | -| Keyverse boundary | Identity/authentication backend | Assessment Delivery receives purpose-authorized opaque subject/actor evidence; it does not become an identity store | - -## Proposed Context Map - -```mermaid -flowchart LR - TA[talent_acquisition] -->|released request ACL| AD[assessment_delivery - Proposed] - TM[future protected talent_management] -.->|released request ACL if adopted| AD - JA[job_architecture] -->|released Job/FJA/KSAO evidence refs| AD - AD -->|versioned provider adapter| EXT[external assessment/scoring owner] - EXT -->|immutable result locator + version + integrity| AD - AD -->|non-authorizing result evidence| TA - AD -->|exact predictor coordinates| WV[workforce_validation] - KV[Keyverse] -. identity/authz backend .-> AD - AD -->|audit/outbox| AP[audit_provenance] -``` +# Assessment assignment coordination and external result-handoff traceability + +Status: Active-PR design evidence for Proposed ADR 0429. This file is not protected product capability and does not make a mutable Psychometrics Commons source revision a production dependency. + +Protected Orgmetra baseline reviewed: `develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f`. -Dashed/future edges are not current product authority. Physical database co-location never permits cross-context application SQL. +## Authority repair recorded on this branch -## Proposed Ubiquitous Language +The predecessor ADR/traceability design proposed a local `assessment_delivery` context containing an `AssessmentAdministration` entity and local result-supersession lifecycle. Fresh protected-authority review found that this contradicted Orgmetra `CLAUDE.md`, README and Accepted ADR 0001, all of which keep assessment operations / lifecycle and immutable assessment result snapshots with Psychometrics Commons. -| Term | Proposed meaning | Explicit non-meaning | -|---|---|---| -| `AssessmentAssignment` | accountable business request for one purpose/subject/process and exact released assessment procedure contract | provider session, score, selection decision | -| `AssessmentAdministration` | one actual administration occurrence under an assignment | transport retry, assessment instrument definition | -| `provider_session_reference` | external transport/provider coordinate associated with an administration | canonical administration identity by itself | -| `AssessmentResultReference` | immutable external owner locator bound to one administration and exact procedure/scoring contract versions | copied provider payload, locally authoritative score | -| `AssessmentResultSupersession` | append-only predecessor/successor lineage for correction/rescore | in-place result mutation | -| `AssessmentDeliveryPolicy` | operational expiry/idempotency/permitted-purpose/evidence requirements | scoring model, cut score, validity or fairness policy | -| `not_verifiable` | required owner/version/provenance cannot be reconstructed sufficiently for authorized use | zero score, failure-to-pass, neutral scientific result | +Psychometrics Commons protected README independently assigns itself participant/session lifecycle, response events, scoring dispatch, immutable result snapshots, product persistence and resource authorization. Its GitHub release inventory was empty at this review, so protected-main source is design evidence only and production integration must fail closed until an immutable released owner contract exists. -## Proposed aggregate and transaction boundary +The current repair therefore narrows the Proposed owner to **`assessment_coordination`**: Orgmetra owns HR-side assessment assignment intent, purpose, correlation and append-only binding to released external execution/result references. It does not own the assessment administration/session, response, scoring or result-snapshot lifecycle. -`AssessmentAssignment` is the aggregate root. One command transaction may validate local current state, append one idempotent assignment/admin/result-reference transition, write correlated audit/outbox evidence and commit. External provider/scoring calls happen after commit and never while an Orgmetra transaction or explicit database lock waits for remote/LLM/scoring work. +## Ubiquitous language + +| Term | Kind | Owner | Meaning | +|---|---|---|---| +| `AssessmentAssignment` | Aggregate Root | Orgmetra `assessment_coordination` | HR business intent/purpose to request or consume an external assessment | +| `AssessmentExecutionReference` | Value Object | Orgmetra projection of released Psychometrics Commons evidence | Opaque released-contract reference to one specialist-owned session/execution | +| `AssessmentResultSnapshotReference` | Value Object | Orgmetra projection of released Psychometrics Commons evidence | Immutable released-contract reference to one specialist-owned result snapshot | +| `AssessmentResultBinding` | Entity | Orgmetra `assessment_coordination` | Append-only admission of one exact external result snapshot under one assignment/purpose | +| `AssessmentAssignmentPolicy` | Domain policy/value contract | Orgmetra | Versioned HR-side purpose/request/window/evidence-consumption policy; not an instrument/scoring/session policy | +| assessment session / administration | External aggregate/lifecycle | Psychometrics Commons | Actual assessment execution/session lifecycle; not copied into Orgmetra | +| response evidence | External evidence | Psychometrics Commons | Raw/structured assessment response evidence; not Orgmetra coordination truth | +| scoring dispatch / result snapshot / result supersession | External lifecycle/evidence | Psychometrics Commons + numerical owner where applicable | Specialist scoring and immutable result authority | +| Job/FJA/KSAO truth | Domain truth | `job_architecture` | Job and work-analysis evidence referenced by purpose/version | +| selection decision | Domain truth | `talent_acquisition` / governed decision boundary | Human-accountable employment decision; assessment completion is non-authorizing | +| validity/fairness interpretation | Scientific truth | `workforce_validation` | Study design, estimand, validity/fairness and interpretation | + +## Context Map + +```text + job_architecture -------- released refs --------> assessment_coordination + talent_acquisition ------ request/consume ------> assessment_coordination + future talent_management - request/consume -----> assessment_coordination + performance_management -- request/consume ------> assessment_coordination + + assessment_coordination -- released ACL -------> Psychometrics Commons + | | + | HR purpose / assignment | session/response/scoring/ + | + external evidence binding | immutable result snapshots + v v + workforce_validation <------ exact released coordinates / lineage + + integration_hub may carry adapter/inbox/outbox transport only. + Keyverse remains identity/authn/authz backend. + fast-mlsirm remains numerical psychometric-kernel owner where its released contract is consumed by the product owner. +``` -A provider callback enters through an idempotent inbox/adapter boundary, resolves one exact tenant/assignment/administration coordinate, validates released contract identity and purpose, and then performs one short domain transition. The callback cannot call a downstream selection/promotion/fairness finalizer in the same authority boundary. +No arrow authorizes direct application-table SQL, source copying or mutable branch consumption. ## Ownership matrix -| Fact / behavior | Proposed owner | Consumer(s) | Forbidden shortcut | +| Concern | Orgmetra `assessment_coordination` | Psychometrics Commons / specialist owner | Downstream owner | +|---|---:|---:|---:| +| HR assessment assignment identity | owns | references/correlation only if contract supports it | consumes released Orgmetra truth | +| HR purpose/reason/process scope | owns | consumes only what released request contract requires | re-authorizes for downstream use | +| assignment request/cancel/expiry coordination | owns | external service receives request/correlation | downstream does not infer session state | +| assessment session/administration lifecycle | **does not own** | owns | reference only | +| item/instrument publication | does not own | owns | reference only | +| raw responses / response events | does not own | owns | no copy by default | +| scoring dispatch | does not own | owns | reference only | +| psychometric numerical kernel | does not own | `fast-mlsirm` / specialist numerical owner | reference/evidence only | +| immutable result snapshot | does not construct/mutate | owns | consumes released reference | +| result correction/supersession | does not create | owns | local append-only binding follows verifiable owner chain | +| assignment-to-result binding | owns append-only local evidence binding | publishes authoritative result reference | consumes exact binding | +| selection/promotion/etc. decision | does not authorize | does not authorize | governed HR decision owner | +| validity/fairness/scientific inference | does not authorize | result owner does not imply validity | `workforce_validation` | +| connector delivery/retry transport | optional adapter consumer | published service endpoint/event owner | `integration_hub` may implement transport mechanics | + +## Requirement traceability + +| ID | Requirement / RED condition | Authority | Future executable evidence | |---|---|---|---| -| assessment business request/purpose | `assessment_delivery` | Talent/HR workflows, audit | generic connector metadata as sole authority | -| administration occurrence identity/status | `assessment_delivery` | requester, audit, validation | provider session ID as sole semantic identity | -| instrument/item/content lifecycle | external assessment owner | Assessment Delivery reference only | source copy into Orgmetra | -| response data | external/privacy owner by contract | purpose-bound only if explicitly released | unrestricted HRIS replication | -| scoring algorithm/model | external scoring owner | reference by exact released version | local floating model alias | -| result evidence | external result owner; Orgmetra stores immutable reference | selection/Talent/validation through ACL | digest-only or mutable URL as authority | -| Job/FJA/KSAO job-related evidence | `job_architecture` | Assessment Delivery / selection / validation | copied job semantics | -| employment decision | governed selection or later protected Talent owner | audit, validation | assessment callback finalizes decision | -| validity/fairness/adverse impact | `workforce_validation` | decision governance/reporting | provider completion or raw score treated as scientific GREEN | -| adapter retry/webhook transport | `integration_hub` / released adapter implementation | Assessment Delivery | connector owns business lifecycle | -| identity/authentication | Keyverse | all authorized consumers | Assessment Delivery identity shadow table | - -## Contract requirements before implementation may be accepted - -A released Assessment Delivery contract must preserve at minimum: - -- `tenant_record_id` or equivalent tenant-scoped opaque identity; -- accountable actor and purpose; -- opaque subject reference plus process/Job context required for the purpose; -- stable semantic assignment identity and stable administration occurrence identity; -- external procedure owner and exact released procedure version; -- external scoring owner/version when scoring applies; -- requested/effective/expiry window and operational policy version; -- provider session reference only as transport provenance; -- immutable external result owner locator, result/contract version and integrity evidence; -- explicit lifecycle outcomes including cancellation, expiry, unavailable, interrupted, invalidated and not-verifiable; -- append-only correction/rescore/supersession lineage; -- accommodation/accessibility provenance sufficient to verify correct administration without duplicating unrestricted sensitive detail; -- idempotency, causation/correlation and immutable audit/outbox evidence. - -## RED → GREEN verification map - -| RED finding | Minimum GREEN evidence | -|---|---| -| floating/mutable procedure or scoring reference accepted | contract/domain tests reject unversioned, mutable or alias-only authority | -| digest without released owner locator accepted | domain tests require owner/context/version/locator + integrity evidence | -| retry creates duplicate administration | PostgreSQL/service contract proves one semantic occurrence across replay while allowing an intentional second administration | -| wrong tenant/subject/purpose/occurrence result accepted | authorization/domain tests fail closed before mutation | -| provider contract drifts between assignment and result | exact historical version binding is persisted and mismatch requires explicit supersession/repair | -| unavailable/interrupted/invalidated/not-verifiable becomes numeric/normal completion | typed-state tests prove non-authorizing semantics end to end | -| rescore overwrites prior result | append-only predecessor/successor tests preserve both historical evidence identities | -| callback advances/rejects/finalizes worker/candidate | architecture/service tests prove no direct high-impact decision command from provider callback boundary | -| external provider call runs inside DB transaction | integration test/telemetry proves commit precedes remote wait and connection/lock cleanup is bounded | -| cross-context SQL or source copy appears | architecture/static contract rejects foreign application-table/source ownership | -| Workforce Validation cannot reconstruct predictor identity | E2E resolves exact assessment/scoring/admin/result coordinates consumed by a study version | -| AI assessment lacks development/scoring/use provenance | result remains `not_verifiable`/non-authorizing until released owner evidence is resolvable | - -Synthetic fixtures are suitable for unit/contract failure tests only. Buyer/scientific acceptance requires real or right-cleared assessment-provider evidence whose use permits the asserted verification. No synthetic-only acceptance may be reported as production validity/fairness evidence. - -## Standards and scientific traceability - -The APA 7 reference set and current standards-status notes are in `docs/doctoring/assessment-assignment-result-handoff-references.md`. - -The key standards interpretation is deliberately bounded: ISO 10667-1:2020 supports a client-side lifecycle spanning multiple work-related assessment purposes; ISO 10667-2:2020 defines service-provider concerns; both Edition 3 replacements are still work items. SIOP/AERA-APA-NCME material supports provenance, job-relatedness, reliability/validity/fairness and auditability. None of these sources turns a provider callback into Orgmetra decision authority. - -## Documentation/release convergence required for completion - -If implementation proceeds, completion requires code-current updates to the canonical Context Map/Architecture, TRD, data model/ERD/UML, API/event contracts, SECURITY, THREAT_MODEL, TEST_STRATEGY, OPERABILITY/recovery and release evidence through their live single-writer owners. `docs/product-technical-gap-baseline.md` remains owned by PR #100 and must not be edited from this ADR lane. - -ADR 0429 remains Proposed until those executable contracts reach protected truth. A docs-only merge of this proposal is not sufficient to close Issue #429. \ No newline at end of file +| ASSMT-001 | Orgmetra owns HR-side `assessment_assignment` intent/purpose/correlation but not assessment operations | Protected CLAUDE/README, ADR 0001, ADR 0429 | architecture/domain ownership tests and schema/API inventory | +| ASSMT-002 | Production consumption requires an immutable **released** Psychometrics Commons contract | CWL integration rule; ADR 0002; ADR 0429 | release/version/package-digest/SBOM/provenance fixture and consumer conformance | +| ASSMT-003 | A mutable branch/PR/main or digest-only source snapshot cannot be production authority | ADR 0002 / automation contract | negative ACL/config contract | +| ASSMT-004 | Orgmetra must not own a local `AssessmentAdministration`/session aggregate, response ledger, scoring lifecycle or result-snapshot lifecycle | Protected specialist boundary | architecture fitness / schema-event negative tests | +| ASSMT-005 | One `AssessmentAssignment` binds exact tenant, purpose/policy, accountable process/actor and required owner contract before dispatch | ADR 0429 | aggregate RED/GREEN | +| ASSMT-006 | Identical semantic assignment/request replay is idempotent; genuinely distinct external executions remain distinct | ADR 0429 | unit + PostgreSQL concurrency/idempotency tests | +| ASSMT-007 | External execution/result refs must bind exact released contract/schema and owner locator; digest-only or floating aliases fail closed | ADR 0429 | consumer contract tests | +| ASSMT-008 | Callback/HTTP success/external `completed` cannot approve, reject or advance employment decisions | ADR 0001 / ADR 0429 | adversarial application/API tests | +| ASSMT-009 | `unavailable`, `interrupted`, `invalidated`, `not_verifiable` and equivalent owner states cannot coerce to score/pass/fail/normal completion | ADR 0429 / Operability boundary | state-machine/API tests | +| ASSMT-010 | Result correction/supersession creates a new local binding only when an owner-issued chain is verifiable; old binding remains reconstructable | ADR 0429 | append-only persistence + as-recorded tests | +| ASSMT-011 | Raw responses, item text, provider payload, credentials and unnecessary accommodation detail are excluded from coordination truth by default | AGENTS/SECURITY privacy rules / ADR 0429 | schema/event/privacy negative tests | +| ASSMT-012 | Remote Psychometrics Commons/provider work never waits inside Orgmetra DB transaction/explicit lock | DDD/minimal-transaction rule / ADR 0429 | integration fault/timeout + lock-duration tests | +| ASSMT-013 | Selection/Talent re-resolves exact assignment + released result binding before consequential use | ADR 0001 / ADR 0429 | decision-boundary integration tests | +| ASSMT-014 | Workforce Validation can recover exact assignment plus exact released procedure/instrument/scoring/calibration/norm/result coordinates or reports `not_verifiable` | Workforce Validation scientific contract | #425 consumer/reproducibility tests | +| ASSMT-015 | No cross-service SQL or source copy | ADR 0002 | dependency/static architecture tests | +| ASSMT-016 | Synthetic fixtures are mechanism evidence only; buyer/scientific acceptance uses real/right-cleared released owner evidence | test/scientific policy | acceptance evidence bundle | + +## Aggregate and transaction boundary + +The proposed Orgmetra aggregate root is only `AssessmentAssignment`. + +A local transaction may: + +1. create/update the HR coordination state; +2. append one audited outbound intent/outbox record; or +3. verify and append one external execution/result binding. + +It must not execute an assessment, wait for a remote service, score a response or construct a result snapshot while the transaction/lock is open. + +`AssessmentExecutionReference` and `AssessmentResultSnapshotReference` are inert released-contract value objects. `AssessmentResultBinding` is append-only Orgmetra evidence that points to an immutable specialist artifact. The specialist owns the actual session/result aggregate and its supersession history. + +## RED → GREEN plan + +### DDD/ownership RED + +Reject any proposed implementation that: + +- adds an Orgmetra `assessment_administration` / `assessment_session` business table or aggregate that mirrors specialist session state; +- stores raw response/item/scoring payloads as `assessment_coordination` truth; +- makes Orgmetra the constructor/mutator of immutable result snapshots or specialist supersession events; +- treats `integration_hub` transport state as HR assessment-assignment truth; or +- treats a recruiting-only context as the universal cross-lifecycle owner. + +GREEN requires one clear HR coordination aggregate and released external references only. + +### Dependency RED + +With Psychometrics Commons release inventory empty, consumer admission must remain unavailable. A branch SHA, PR SHA, protected-main SHA or hand-copied schema must fail production dependency admission. + +GREEN requires an immutable owner release with exact contract/schema identity, artifact/package digest, SBOM/provenance/reproducibility/compatibility evidence and consumer conformance fixtures. + +### Replay/concurrency RED + +- same assignment/request replay creates a duplicate local assignment or duplicate semantic outbound intent; +- two distinct owner execution refs collapse because person/instrument/result values happen to match; +- conflicting replay overwrites the previous binding; or +- a remote call is performed while a local transaction/explicit lock is held. + +GREEN requires idempotent local identity, explicit external occurrence identity, append-only conflict evidence and short local transactions. + +### Authority RED + +- callback success or external `completed` advances/rejects/finalizes a candidate/worker action; +- result presence becomes validity/fairness GREEN; +- missing owner version/locator/digest requirements are accepted; +- non-success becomes zero/pass/fail; or +- a local rescore overwrites specialist truth. + +GREEN preserves non-authorizing evidence semantics and forces downstream independent authorization/scientific interpretation. + +### Privacy/security RED + +- raw response/item content, credentials, provider payload or unnecessary accommodation detail enters shared coordination evidence; +- foreign tenant/process evidence can bind to an assignment; +- an unauthorized actor can learn existence through a reference lookup; or +- a consumer queries Psychometrics Commons application tables. + +GREEN requires purpose-bound authorization, minimal released references, tenant isolation, no existence oracle and ACL/API/event-only composition. + +### Workforce Validation RED + +A study that uses an assessment predictor but cannot identify the exact Orgmetra assignment and exact released Psychometrics Commons result/scoring coordinate must be `not_verifiable`. Equal human-readable labels or equal numeric scores do not prove coordinate identity. + +GREEN requires exact released/versioned predictor evidence and reproducible linkage into #425's study-design authority. + +## Documentation / owner reconciliation + +This branch may propose the boundary only. It does not edit protected `CLAUDE.md`, README, Architecture/TRD/Data Model/ERD/UML/API or `docs/product-technical-gap-baseline.md`. + +After executable owner proof and an immutable Psychometrics Commons release exist: + +- protected product/technical docs are reconciled through their canonical writer; +- #100 alone updates the durable product-technical gap baseline; +- downstream Workforce Validation consumes only protected/released truth; and +- release evidence must remain exact-head/current, not transferred from this docs proposal. + +A docs-only merge is not #429 completion. \ No newline at end of file From 376e12f6cad870eec38842e27a2c3b4ea19debed Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 21 Sep 2026 05:08:55 +0900 Subject: [PATCH 7/7] docs(assessment): correct client specialist boundary --- ...nt-assignment-result-handoff-references.md | 35 +++++++++++++------ 1 file changed, 25 insertions(+), 10 deletions(-) diff --git a/docs/doctoring/assessment-assignment-result-handoff-references.md b/docs/doctoring/assessment-assignment-result-handoff-references.md index 31e93a628..ed3ed58db 100644 --- a/docs/doctoring/assessment-assignment-result-handoff-references.md +++ b/docs/doctoring/assessment-assignment-result-handoff-references.md @@ -1,16 +1,30 @@ -# Assessment assignment and result-handoff references +# Assessment assignment coordination and result-handoff references -This note supports ADR 0429. It records the authority used to separate Orgmetra assessment-delivery orchestration from external instrument/scoring ownership and from downstream validity/fairness authority. It does not certify an assessment provider, instrument, scoring model, cutoff, legal conclusion, or deployment. +This note supports Proposed ADR 0429. It records the authority used to separate **Orgmetra HR-side assessment coordination** from **Psychometrics Commons assessment operations / immutable result ownership** and from downstream validity/fairness authority. It does not certify an assessment provider, instrument, scoring model, cutoff, legal conclusion, or deployment. + +## Repository ownership authority reviewed 2026-09-21 + +Protected Orgmetra authority is internally consistent on the specialist boundary: + +- `CLAUDE.md`: Psychometrics Commons owns assessment operations and immutable assessment result snapshots. +- Repository README: Psychometrics Commons + fast-mlsirm own the assessment lifecycle and psychometric computation boundary. +- Accepted ADR 0001: Psychometrics Commons, fast-mlsirm and TEPP remain specialist boundaries for assessment operations, numerical kernels and temporal analysis artifacts; Orgmetra stores references to those artifacts. +- Accepted ADR 0002: sibling CWL products remain behind explicit package/API/event/adapter contracts and product ownership does not transfer merely because a protocol is consumed. +- Protected TRD: assessment results stay external immutable snapshot references unless a later ADR transfers instrument lifecycle ownership. + +Psychometrics Commons protected README independently states that it owns product APIs, instrument publication, participant/session lifecycle, response events, scoring dispatch, immutable result snapshots, product persistence and resource authorization. Therefore an Orgmetra-local `AssessmentAdministration`/session or result-snapshot lifecycle would duplicate existing protected owner truth. + +At this review the Psychometrics Commons GitHub release inventory is empty. Protected-main or an immutable commit is useful design evidence but, under the CWL integration/release rule applied to this work, is not a production contract for Orgmetra. ADR 0429 therefore treats an immutable released owner contract as a hard prerequisite for executable integration. ## Current standards status reviewed 2026-09-21 International Organization for Standardization. (2020a). *Assessment service delivery—Procedures and methods to assess people in work and organizational settings—Part 1: Requirements for the client* (ISO 10667-1:2020). https://www.iso.org/standard/74716.html -ISO 10667-1:2020 is the published client-side edition. Its public scope covers the client's needs/rationale, conditions of use, assessment approach, access/use/storage of results, and organizational decisions. Its examples include recruitment, selection, development, appraisal, promotion, succession planning, and reassignment. This is the strongest standards basis for treating assessment assignment/use as a business lifecycle rather than generic connector state. +ISO 10667-1:2020 is the published client-side edition. Its public scope covers the client's needs/rationale, conditions of use, assessment approach, access/use/storage of results, and organizational decisions. Its examples include recruitment, selection, development, appraisal, promotion, succession planning, and reassignment. This supports Orgmetra owning the HR-side reason/purpose/assignment and accountable use of assessment evidence across more than recruitment. It does **not** require the client HRIS to own the assessment service provider's session or scoring runtime. International Organization for Standardization. (2020b). *Assessment service delivery—Procedures and methods to assess people in work and organizational settings—Part 2: Requirements for service providers* (ISO 10667-2:2020). https://www.iso.org/standard/74717.html -ISO 10667-2:2020 remains the published service-provider counterpart. Its public scope includes selection/implementation/evaluation of assessment procedures, interpretation/reporting, personal and assessment data, competence/professionalism, and organizational decisions. ADR 0429 uses it to require explicit released provider/procedure/scoring provenance; Orgmetra does not infer provider compliance from a result callback. +ISO 10667-2:2020 remains the published service-provider counterpart. Its public scope includes selection/implementation/evaluation of assessment procedures, interpretation/reporting, personal and assessment data, competence/professionalism, and organizational decisions. ADR 0429 uses the Part 1 / Part 2 split to preserve a client-side HR coordination contract while leaving the service-provider assessment operation behind the specialist boundary. A result callback is never treated as proof of provider compliance or scientific adequacy. International Organization for Standardization. (2026a). *Assessment service delivery—Procedures and methods to assess people in work and organizational settings—Part 1: Requirements for the client* (ISO/AWI 10667-1, Edition 3, work item). https://www.iso.org/standard/94563.html @@ -26,7 +40,7 @@ Society for Industrial and Organizational Psychology. (2018). *Principles for th Society for Industrial and Organizational Psychology. (2023). *Considerations and recommendations for the validation and use of AI-based assessments for employee selection*. https://www.siop.org/post/siop-releases-recommendations-for-ai-based-assessments/ -The SIOP AI-assessment recommendations explicitly emphasize job-related score meaning, score consistency, fairness, appropriate operational use, and documentation of development/scoring steps for verification and audit. ADR 0429 uses those principles only to justify immutable procedure/scoring/version/use provenance and fail-closed `not_verifiable` behavior. Scientific adequacy remains a Workforce Validation responsibility. +The SIOP AI-assessment recommendations emphasize job-related score meaning, score consistency, fairness, appropriate operational use, and documentation of development/scoring steps for verification and audit. ADR 0429 uses those principles to require an exact released procedure/instrument/scoring/result coordinate and explicit intended use. They do not transfer scoring or assessment-session ownership into Orgmetra. Scientific adequacy remains a Workforce Validation responsibility. ## Peer-reviewed evidence @@ -44,11 +58,12 @@ These papers reinforce that assessment interpretation depends on the procedure, ## Architectural interpretation -The reference set supports four narrow decisions in ADR 0429: +The combined repository, standards and research evidence supports five narrow decisions in ADR 0429: -1. assessment use is cross-lifecycle business state, so it should not be hidden as generic integration transport; -2. Orgmetra needs exact purpose, procedure/scoring contract version, administration occurrence, result-owner locator and correction provenance to support accountable consumption; -3. operational assessment delivery does not imply ownership of instrument content, item/response data, scoring algorithms, or psychometric conclusions; and -4. downstream selection and Workforce Validation must independently authorize and reconstruct the exact evidence they consume. +1. Orgmetra needs cross-lifecycle **HR assignment/purpose/correlation truth**, so that business fact should not be hidden as generic connector transport or forced into a recruitment-only context. +2. Psychometrics Commons remains the owner of assessment session/administration, response, scoring-dispatch, immutable result-snapshot and specialist supersession lifecycles; Orgmetra stores only released references/bindings. +3. A production result handoff requires an immutable released owner contract. A protected-main SHA, PR SHA, source copy or digest-only reference is not the cross-repository production contract. +4. Orgmetra needs exact purpose plus exact released execution/result/scoring provenance to support accountable downstream consumption, but callback success or score presence grants no selection, validity or fairness authority. +5. Selection/Talent and Workforce Validation must independently authorize and reconstruct the exact Orgmetra assignment and exact specialist evidence they consume. Any future implementation must doctor new jurisdiction-specific legal requirements separately. These professional/measurement sources are not a substitute for legal advice or tenant-specific compliance policy. \ No newline at end of file