[lipikaramaswamy]

Related Issue

Related to #262.

Plan Document

plans/262/detection-semantics.md

Summary

Establishes a shared, deterministic lifecycle for detection candidates before
the regex work in #265 is rebuilt on top of it.

The implementation will provide:

• a private detector-backend contract that publishes one ordered
  DetectionResult per invocation and distinguishes no findings from failure;
• a structured SourceTextRef for each canonical record field and a
  SourceView mapping backend-local spans into canonical source offsets;
• immutable observation references composed from detection-result and
  observation IDs, plus short validation aliases scoped to one dispatched
  request inventory;
• source-aware normalized semantic identity for post-admission coalescing;
• single-use validation decisions and explicit admission states;
• compact conflict groups with explicit source geometry so overlapping and
  repeated-value candidates survive validation without duplicating context;
• accepted provenance and propagation authority;
• eligibility before destructive overlap resolution;
• prompt-independent augmentation admission so fix: switch augment prompt to plain text input, drop GLiNER seed entities #267 can move to plain text
  without adding another merge path;
• bounded name splitting and occurrence propagation with origin-specific
  authority;
• one final deterministic reconciliation policy across detector, augmenter,
  and derived candidates;
• trace diagnostics and focused behavioral tests.

The first commit contains the reviewed implementation plan. Production code and
tests will be added to this same draft PR.

Review focus

1. Does the private detector-backend boundary preserve the existing
   DataDesigner execution path while publishing reusable detection results?
2. Do SourceTextRef and SourceView cleanly distinguish records and fields
   while mapping overlapping or chunked views into canonical offsets?
3. Are immutable observation references, request-local validation aliases, and
   source-aware normalized semantic identity separated precisely enough?
4. Does each validator response have authority only over its dispatched
   request inventory, including for chunked validation and retries?
5. Are accepted, rejected, and fallback_protected the correct terminal
   admission states?
6. Is the conflict-group geometry sufficient for repeated values, clipped
   excerpts, and groups crossing validation chunks?
7. Are provenance and occurrence-propagation permissions preserved without
   granting authority to rejected or undecided origins?
8. Does the bounded split-then-propagate derivation preserve name-part coverage
   without transferring authority between coalesced origins?
9. Does the final policy correctly rank different overlaps by protected
   coverage before source and reserve source priority for equal geometry?
10. Does the implementation keep augmentation prompt construction independent
    from output admission so fix: switch augment prompt to plain text input, drop GLiNER seed entities #267 can switch to plain text without adding another
    merge path?
11. Is the deterministic comparator and internal source priority suitable for
    the sources currently on main and the later regex integration?
12. Are the trace diagnostics and test matrix sufficient to make failures
    observable and prevent decision replay?

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Refactoring
• CI, release, or contributor workflow update

Contributor Checklist

• PR title follows Conventional Commits
• Related issue is identified
• Plan document is linked above
• Public API impact checked; no public API change is planned
• No real PII added
• No credentials or secrets added

Validation

Plan validation completed:

• git diff --cached --check
• make copyright-check
• repository make check through the pre-commit hook

Implementation tests and results will be recorded here as the code lands.

Documentation and Artifacts

• Plan document added under the issue-scoped plans/262/ path
• No tutorial sources or runtime artifacts changed

[binaryaaron]

agent (review-pr): Plan feedback — shared detection semantics and composable backends.

The shared lifecycle is the right foundation for #265 and the eventual #302 integration. Following a council review, I suggest adding a narrow internal entity-detection backend contract to this plan, with GLiNER first and regex added when #265 is rebuilt. The four inline comments identify contracts to clarify before implementation; this is a review of the plan at fd1fea9b, not an approval of production code.

The intended sequential composition is:

source -> regex -> regex observations --------------------+
                      |                                  |
                      +-> optional GLiNER context        |
source -----------------> GLiNER -> new observations -----+
                                                         |
                                                      collect
                                                         |
                               shared admission / derivation / reconciliation

Without the regex-to-GLiNER data binding, the two backends can execute independently. Ordering and data consumption should be explicit: running GLiNER after regex does not automatically mean it consumes regex output.

Ownership

• Backends produce source-bound observations and distinguish successful empty output from failure.
• The shared detection lifecycle owns candidate admission, label eligibility, accepted provenance, propagation, and overlap reconciliation.
• Graph bindings own execution dependencies, request associations, retries, resource limits, and artifact provenance. Qualification of the final transformed output remains separate from detection acceptance.

Identity and reuse

Assign immutable, source-scoped observation references before publishing a batch for downstream reuse. Collection preserves them; short validator tokens such as c1 can be request-local aliases. Do not allocate identity from completion order or regenerate it during merging.

GLiNER can produce new claims while using regex context, and those claims follow its own admission route. Forwarded regex observations retain their original identity and restrictions; passing them through GLiNER cannot create another supporting origin. Derived claims retain explicit parent references. Graph-level dependency records influence, while entity-level provenance determines authority. The simplest initial collection contract retains the original regex batch and adds GLiNER's newly produced observations.

Scope

A private _EntityDetectionBackend ABC is reasonable. Introduce the observation contract and a narrow GLiNER boundary while preserving the existing DataDesigner execution/export path. Add the regex implementation with #265 and graph execution bindings during #302 integration. An async detect() method alone does not express the effect, batching, cancellation, and resource contracts the SDK needs. Keep graph imports out of the domain types.

Define a concrete supported reuse behavior before standardizing a generic hints interface. Supplying context, selecting labels, masking text, and skipping detector work have different coverage consequences. Unsupported context should not silently be ignored. Keep augmentation's invocation contract separate initially while allowing it to produce compatible observations and use the shared lifecycle.

Validation

Use shared semantic fixtures for the pure helpers, serialized DataDesigner execution, and eventual graph adapter. Include the four inline scenarios, sequential reuse without duplicated authority, preservation of both source batches, and serialization round trips. Pin benchmark inputs, configuration, baseline, and acceptance criteria, including coverage changes from fallback and seed-over-augmentation precedence.

The excluded-rule export finding remains a separate #265 item. Also align this plan's older allow_model_detection / validate_with_llm references with #265's current detect_additional_matches / validate_matches_with_llm names.

[lipikaramaswamy]

Addressed the plan review in 1bce7b7, with terminology clarified in 5f38e04.

The revised plan now defines:

• a private detector-backend contract with one ordered DetectionResult per invocation and distinct no-findings and failure results;
• immutable observation references composed from detection-result and observation IDs and allocated before publication;
• short validator aliases scoped to one dispatched request inventory;
• explicit backend ownership versus shared lifecycle ownership;
• GLiNER as the first adapter while retaining the existing DataDesigner execution/export path;
• no implicit data dependency from execution order; later backend reuse requires a typed binding, preserves the original result, and publishes only new claims;
• request-scoped decision authority, explicit conflict-group geometry, bounded derivation, and serialization tests;
• the current regex names detect_additional_matches and validate_matches_with_llm.

I also updated #304 so unusable label proposals receive exact-span fallback, while usable but excluded or disallowed proposals are rejected and traced separately.

[asteier2026]

When we have overlapping spans where one is contained in another, I definitely see that taking the longer span makes sense. It's just not clear to me what the right thing to do is when the spans overlap. Ideally, we would use sensitivity but we won't have that information until after the sensitivity disposition. I'm thinking this situation won't happen often and it's probably not worth worrying about. Taking the longest span is a good default.

[asteier2026]

It's not clear to me what happens when we have "May called yesterday. The appointment is in May". Could we end up tagging both "May"s as names?

[binaryaaron]

The source/view identity and execution-scoped admission changes address the main concerns Andrea raised. A few implementation details will matter when these components are used in #302:

• Collect all observations before resolving overlaps. With A=[0,4), B=[3,9), and C=[8,15), longest-first selection across all three retains A and C. Resolving A/B first and then adding C retains only C. Add a test showing that collecting results from graph branches does not discard candidates before the final reconciliation step.
• Preserve each origin’s permissions after merging equivalent candidates. If a regex observation prohibits propagation and an accepted augmentation observation permits it, only the augmentation observation supports protecting a new occurrence. A combined propagation flag and a list of origin IDs would lose that distinction.
• Convert offsets to the original source coordinates exactly once. The source section says backends do this before publishing observations, while Step 2 also describes converting offsets. Specify which step performs the conversion and which checks the result.
• Keep detector behavior separate from workflow execution. The private backend interface should describe detector inputs and results. NDD and future graph adapters should manage transport, retries, scheduling, and result publication. Validation and merging should remain shared operations outside the backend classes.

#302 can track which inputs and artifact versions an operation used. The detection adapter will still need to check serialized observations and associate each validation result with the source text, context, and policy used to produce it. Reusing an observation should preserve its detector identity, but should not automatically reuse an earlier validation result. Accepting an entity also does not establish that the graph’s final output qualifies for Protected release.

These details can be specified and tested in #305 without implementing graph integration or validation under multiple policies yet.

[asteier2026]

Say we had:
[
{"id": "c1", "decision": "reclass", "proposed_label": " "},
{"id": "c1", "decision": "reclass", "proposed_label": "city"}
]
The first is non-empty. Would normalization occur before we decide which to use?

[lipikaramaswamy]

It's not clear to me what happens when we have "May called yesterday. The appointment is in May". Could we end up tagging both "May"s as names?

Yes, this can happen today. This behavior has existed since the original replace-mode implementation: GLiNER finds one occurrence → validator confirms the value is sensitive → protect every identical occurrence. The motivation was reasonable at the time

• GLiNER may detect one mention but miss another because of context, confidence threshold, or chunking.
• Replacing every repeated occurrence avoids leaking the same identifier elsewhere.
• It gives consistent replacements for repeated values.

So we have been making the semantic assumption that if two spans have the same text, they have the same label and meaning. However, this assumes that equal text has equal meaning. That is not true for your example, or something even more complex like May will handle the situation in July, or at worst it may be in May. The three May/may occurrences refer to a person, a modal verb, and a month.

I no longer think global blind propagation should be the standard final stage. A better target workflow would be:

GLiNER
→ produce occurrence-specific observations
→ validate each observed occurrence independently

Regex
→ produce every matched occurrence
→ accept or validate each match according to its configured rule

Known repeated value (maybe? this implies low trust in gliner / regex / any other additional engine) 
→ engine enumerates its additional occurrences
→ assign each occurrence a request-local ID and context
→ validate each occurrence independently

Augmenter
→ return an occurrence-specific source quote
→ engine grounds that quote to one canonical source span

The augmenter currently returns:

{
  "value": "May",
  "label": "first_name",
  "reason": "person name"
}

Because that result has no occurrence anchor, the engine materializes it at every matching occurrence. The proposed response would be:

{
  "value": "May",
  "label": "first_name",
  "source_quote": "May will handle the situation in July",
  "reason": "May is the person performing the action"
}

source_quote would be defined as a verbatim source substring that contains the entity value exactly once and identifies one occurrence within the request’s SourceView. The engine—not the model—would locate that quote, calculate canonical offsets, and verify that the resulting source slice equals value.
If the quote is missing or ambiguous, the engine must not silently choose the first match. It can retry with a longer quote or enumerate the possible occurrences for a targeted validation decision.

We should benchmark the recall effect of removing blind propagation. For highly stable identifiers, exact expansion may still be useful as an explicit policy, but perhaps relying on regex / gliner is sufficient. For contextual entities such as names, locations, and organizations, additional occurrences should receive independent decisions.

I think #305 should establish occurrence-specific authority now; otherwise the new lifecycle risks formalizing the existing assumption that accepting one occurrence authorizes every equal string.

[asteier2026]

Regarding the "May" example. Your suggestions may work. My gut feeling is it will happen way more often that gliner/augment will miss repeated instances of the same value/label than we will have ambiguous cases like "May".

[lipikaramaswamy]

Say we had: [ {"id": "c1", "decision": "reclass", "proposed_label": " "}, {"id": "c1", "decision": "reclass", "proposed_label": "city"} ] The first is non-empty. Would normalization occur before we decide which to use?

Good catch @asteier2026. We probably should not use normalization or response ordering to choose between duplicate entries.
Each validation request has an immutable inventory, and a successful response may contain at most one entry for each assigned alias. If an assigned alias appears more than once—even when the entries are identical—the entire response is semantically invalid.

How about this: we discard the invalid response entirely, and retry the same validation request. With a validator pool, the retry proceeds to the next alias in pool order. With a single validator alias, we will allow one bounded same-alias semantic retry. If all attempts return invalid responses, the request follows the existing terminal FailedRecord path.
Therefore, this response:
[
{"id": "c1", "decision": "reclass", "proposed_label": " "},
{"id": "c1", "decision": "reclass", "proposed_label": "city"}
]
is rejected and retried; we do not select either entry. Normalization and usability checks occur only after the response has passed the one-entry-per-alias check. I also suspect that this issue is not going to crop up that frequently if we reduce the maximum number of entities per validation call from 100 to 25, giving each LLM call less work to do.

This is distinct from an omitted alias or a single unusable entry in an otherwise valid response. Those cases retain exact-span fallback protection without propagation rather than retrying the request.

[lipikaramaswamy]

Follow-up on @asteier2026's concern that GLiNER or augmentation may miss repeated occurrences more often than equal-text ambiguity occurs:

I audited 12 completed trace artifacts covering 986 full-run records across Gemma, GPT-OSS, and Qwen on RAT-Bench, Nemotron-PII, and OpenPII.

• Blind global value propagation affected 74.5% of RAT-Bench records.
• The propagated spans were dominated by broad open-class categories: occupation, field_of_study, employment_status, and education_level accounted for 64.4% of propagated spans, while first_name accounted for 5.4%.
• Manual inspection measured only 38%-44% precision for propagated spans. Common failures were case-insensitive homograph collisions such as OR -> or, US -> us, and IT -> It.
• Separately, 15.6% of unique augmenter suggestions matched multiple non-seed occurrences, and those expansions generated 42.1% of all non-seed augmenter materializations.

This does not prove that repeated-mention recall is unimportant. It does show that unconditional propagation is doing much more than recovering missed repeated names and identifiers, and that the assumption "equal text means equal entity meaning" creates substantial noise.

The revised plan therefore removes blind expand_entity_occurrences() propagation and _split_full_names(). GLiNER observations remain occurrence-specific; regex will emit each pattern match; and augmentation will return exact value plus adjacent context that the engine grounds to canonical occurrences. We will run RAT-Bench, Nemotron-PII, and OpenPII before and after the change, reporting repeated-sensitive-value recall, ambiguous-string false positives, per-label precision/recall, failed-record rate, latency, and token usage separately for each dataset. The change advances only if that benchmark makes the recall/precision tradeoff acceptable.

[lipikaramaswamy]

The implementation plan has been substantially revised in 9854bff, and the settled review threads have been updated and resolved.

One experiment remains explicit in the plan: compare seed-tagged versus canonical plain-text augmenter input on the same RAT-Bench, Nemotron-PII, and OpenPII samples, especially duplicate rediscovery versus genuinely new recovery.

If there are no additional comments by end of day Wednesday, October 21, I will start implementation.