[fas89]

Follow-up to #799. That PR made FLUID contract, fragment and bundle validation read a schema pattern's $ as ECMA-262 does: the end of the value. Six other JSON Schema checks still ran jsonschema's own pattern, which is Python's re.search, and its $ also matches before a final \n:

site	reached from
forge/core/artifact_validators.py::_validate_against_schema	fluid validate-artifacts (ODCS and Bitol ODPS files)
providers/odcs/validation.py::collect_errors	fluid odcs validate, ODCS export and import
providers/odps_standard/validation.py::validate	Bitol ODPS provider
providers/opds/validator.py::validate_against_opds_schema	OPDS provider, validate-artifacts (pinned Draft202012Validator)
forge_datamodel/emit/osi_sidecar.py::validate_osi_document	forge copilot's Ossie conformance check
extension_schemas.py::assemble_proposed_extensions	forge copilot, LLM-proposed extensions.<key> blocks (pinned Draft7Validator)

So ODCS refused schema[].id: "t 1" but accepted "t1\n", which an id: | block produces. Against the real data-product-forge-custom-scaffold schema, the copilot also kept a customScaffold block with use: "ci:basic\n", ref: "main\n" or id: "ci\n". All six checks now go through schema_manager.schema_validator and carry #799's line-break hint.

Decisions

• Default dialect. schema_validator(schema, default=None) passes default to jsonschema's validator_for for a schema with no $schema. OpenStack Rally uses validator_for(..., default=Draft7Validator) the same way.
  • The extension site passes Draft7Validator, the documented plugin contract. An undeclared plugin schema stays draft-07, so dependencies still works.
  • A plugin schema that declares 2020-12 is now read as 2020-12, so its unevaluatedProperties: false is enforced instead of ignored.
  • OPDS passes Draft202012Validator, which is what it pinned before.
• ODPS keeps jsonschema.validate (the schema is checked first, then the best error is raised), now with cls=type(schema_validator(schema)).
• patternProperties keys keep re.search.
  • jsonschema matches key patterns in three keywords: patternProperties, additionalProperties and unevaluatedProperties. check-jsonschema overrides only patternProperties, which lets a key that one keyword matches and another does not through additionalProperties: false unchecked. So I did not copy that approach.
  • Only OPDS has key patterns: ^[a-z]{2}$, in three places. Two of them accept any key. $defs/Languages does not, so a hand-written OPDS file can carry {"en\n": ...} there. fluid's own emitter writes en.
• Tripwires (tests/test_schema_pattern_end_anchor.py) now cover all 13 shipped schemas, not only the FLUID ones.
  • They pin the patternProperties set above.
  • They also fail on $recursiveRef and $dynamicRef. I checked that both re-pick jsonschema's stock class, the same as a nested $schema or $ref: "#".
• One way to build a validator. A test walks fluid_build/ and fails on any validator built outside schema_manager. It catches validator_for(), Draft*Validator(), jsonschema.validate() / js.validate() / jsonschema.validators.validate|extend|create() without a cls= from schema_validator, and from jsonschema import validate. It finds the six sites on the security: schema validation let a value with a trailing newline through a pattern #799 base and seven on the old main.
• jsonschema missing.
  • schema_validator raises ImportError when jsonschema could not be imported. Without that, the Ossie check, its one unguarded caller, raised NameError: jsonschema.
  • validate_opds now reads the same JSONSCHEMA_AVAILABLE flag as the other checks. The find_spec probe, which found a jsonschema that then failed to import, is gone.

Found while reviewing, and fixed here:

• Malformed plugin schema (this branch). assemble_proposed_extensions raised on a plugin schema whose $schema is not a string. Now a schema that cannot be applied drops its block with a redacted warning, and the run continues. That also covers {"type": 5}, which raised on main.
• fluid odcs validate (pre-existing).
  • It cut each message to 120 characters, so for an id like customer_orders_v1 the hint ended ... block keeps and |, which reads like advice to use |. The hint is now kept whole (schema_manager.LINE_BREAK_HINT).
  • It printed the message as Rich markup, so a failing value containing [/] ended the command with Unexpected error: closing tag '[/]' ... has nothing to close (exit 2, also on main). Now it prints with markup=False, and there is a CHANGELOG Fixed entry.

Borrow receipts

• security: schema validation let a value with a trailing newline through a pattern #799's schema_validator: reused. Its own receipts cover check-jsonschema, fastjsonschema/js-regex and networknt#1201.
• jsonschema 4.26 validator_for(schema, default=…): documented, and honoured for an unknown $schema since 4.10.3. Prior art: OpenStack Rally 9f93c810.
• check-jsonschema regex_variants.py: overrides pattern, patternProperties and format: regex, but not additionalProperties or unevaluatedProperties. That gap is why I made the key decision above.

Validation

• New tests in tests/test_standard_schema_pattern_end_anchor.py:
  • The 8 per-site tests are red on the security: schema validation let a value with a trailing newline through a pattern #799 base and on origin/main, and green here.
  • Each follow-up test is red on the commit before its fix: the hint, jsonschema missing, the malformed plugin schema, [/], and OPDS's undeclared 2020-12. Mutations are caught: OPDS default= changed to Draft 7, and a disabled collect_errors guard.
  • Two tests pin behaviour that must not change: ODPS still checks the schema first, and an undeclared extension schema is still read as draft-07.
• Full suite on the branch: 32487 passed, 17 failed, 2 errors. Every failure is the local-venv baseline that security: schema validation let a value with a trailing newline through a pattern #799 also listed (startup and timing budgets, dlt version, DuckDB sandbox, test_ci_* imports, live coding agent), or a timing flake that passes in isolation on the branch (machine load average was about 130).
• Related suites (64 files): 2471 passed. The only failures are the startup-budget baselines.
• Live CLI:
  • fluid generate artifacts examples/02-csv-to-data-product/contract.fluid.yaml, then the ODCS schema[0].id turned into an id: | block and the MANIFEST re-hashed. fluid validate-artifacts exits 1 here (at schema/0/id: 't1\n' does not match '^[A-Za-z0-9_-]+$' (it ends in a line break, …)) and 0 on main. Plain and |- ids exit 0 on both.
  • fluid odcs validate on the same file exits 1 here and 0 on main.
• Gates:
  • Pass: ruff check fluid_build/ tests/, the dead-code sweep, black==24.10.0 --check fluid_build/ tests/, scripts/check_license_headers.py, the product guardrail, detect-secrets-hook (1.5.0) and both lint-imports configs.
  • tests/perf/test_startup_budget.py::test_heavy_sdks_not_loaded_on_help passes: jsonschema stays off fluid --help, and every new import is function-local. The module-count budgets read the same as on main locally.

Reviews

• /security-review: no findings on the main change. A focused re-review of the last fix commit is noted in a comment.
• /world-class-inspection: architecture and tests PASS. Docs and code health raised P3 concerns, now fixed and re-verified: the ODPS docstring, the CHANGELOG hint wording, the Ossie NameError and the dual availability probe. UX: the cut-off hint, fixed.
• /defect-scan: four hunters (security, logic, runtime, cross-file contract) plus CLI journeys. Two real low defects came up, the malformed plugin $schema and the [/] markup crash, and both are fixed above. Everything else was refuted or pre-existing.

Follow-ups, pre-existing and out of scope

• data-product-forge-custom-scaffold: its own validator builds a stock Draft7Validator, so fluid validate (via the plugin's extension validator) still accepts use: "ci:basic\n" or ref: "main\n". It needs a fix in that repo.
• OPDS emitter: any 2-character metadata.language becomes the product.details key. A code that does not match ^[a-z]{2}$ (EN) leaves that entry unvalidated, because the upstream schema allows any key there.
• fluid odcs validate cascade: a failing schema[] item also prints a cascading Unevaluated properties are not allowed ('id', 'name', … were unexpected) error.
• fluid odcs validate markup: the parse-error, vowl and roundtrip lines in the same command still print exception text as markup.
• vowl: the optional third-party second pass reads $ as Python does. The first pass now catches these values.

[github-actions]

📄 Documentation Reminder

This PR appears to be missing a documentation reference. Our docs live in a separate repo.

Please update the PR description with one of:

• Link a docs PR — check the "Docs PR linked" box and paste the URL
• Mark as no docs needed — check "No docs needed" with a justification
• Acknowledge docs TODO — check "Docs TODO" and create the docs PR before merge

See the Contributing Guide for details.