Skip to content

Add ModelRejection enum to categorize pipeline rejection reasons - #2585

Open
jfallows wants to merge 5 commits into
developfrom
claude/cool-wozniak-ebva0y
Open

jfallows wants to merge 5 commits into
developfrom
claude/cool-wozniak-ebva0y

Conversation

@jfallows

Copy link
Copy Markdown
Contributor

Description

This change introduces a new ModelRejection enum to categorize the reasons why a ModelPipeline rejects data, distinguishing between invalid data (malformed or failing validation) and withheld data (acceptable but deliberately not delivered).

Changes

  • New ModelRejection enum: Defines two rejection categories:

    • INVALID: The value is unacceptable (malformed or fails validation)
    • WITHHELD: The value is acceptable but the pipeline chose not to deliver it
  • Enhanced ModelPipelineResult:

    • Added rejection() getter to retrieve the rejection category
    • Added overloaded set() method accepting an optional ModelRejection parameter
    • Existing 3-argument set() method now delegates to the 4-argument version with null rejection
    • Rejection is cleared when status is set to non-REJECTED values
  • Updated pipeline implementations:

    • CoreModelPipeline: Reports INVALID rejection for structural validation failures
    • CoreExtModelPipeline: Reports WITHHELD for stage-level withholding and INVALID for validation failures or envelope errors
  • Comprehensive test coverage:

    • New ModelPipelineResultTest validates result behavior with and without rejections
    • Updated CoreExtModelPipelineTest and CoreModelLenientTest to verify rejection categorization

Motivation

This change provides better observability and diagnostics by explicitly categorizing why data was rejected, enabling downstream handlers and monitoring systems to distinguish between data quality issues and intentional filtering decisions.

https://claude.ai/code/session_01KhKM5SjiYaBR7nTx5RGoq3

Adds the ModelRejection enum (INVALID/WITHHELD) and a failing unit test for
ModelPipelineResult's not-yet-added rejection() accessor and 4-arg set()
overload. ModelPipelineResult itself is not yet updated -- next commit wires
it up and updates CoreModelPipeline/CoreExtModelPipeline to report it.

Towards #2583
Adds a rejection() accessor and a 4-arg set(...) overload so an
implementation can optionally report why a REJECTED outcome occurred. The
existing 3-arg set(...) is unchanged (rejection defaults to null), so every
existing ModelPipeline implementation and consumer keeps compiling and
behaving exactly as before.

Verified via ModelPipelineResultTest (all 5 cases green).

Towards #2583
…WIP)

CoreModelPipeline always reports INVALID (its only rejection cause is a
validity fault). CoreExtModelPipeline reports WITHHELD when a stage called
withhold() and INVALID otherwise (a stage's own reject(diagnostic) call or
the base validator's own structural failure).

Maven build verification still in progress (multi-module reactor
bootstrap in a fresh checkout); standalone javac compile checked.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KhKM5SjiYaBR7nTx5RGoq3
…ion test

shouldRejectValueFailingBaseValidation drives a real validation-failure
report, which needs a working clock; reuse the existing engine(...) test
helper instead of a bare mock.

Verified via a real Maven surefire run (not just standalone javac):
ModelPipelineResultTest (5/5), CoreModelLenientTest (5/5),
CoreExtModelPipelineTest (13/13) all green. checkstyle:check clean for
runtime/engine and runtime/model-core.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KhKM5SjiYaBR7nTx5RGoq3
…ELD coverage

TestModel could only ever report an unclassified rejection via its reject
list, so no test implementation could exercise ModelRejection.WITHHELD
without depending on a production model. Add an additive withhold config
list, parallel to reject, that reports WITHHELD on a match.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KhKM5SjiYaBR7nTx5RGoq3
@jfallows
jfallows force-pushed the claude/cool-wozniak-ebva0y branch from 254c2e7 to 02efc1c Compare September 24, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants