[fas89]

What and why

Python's $ matches at the end of the string and also just before a final \n, so re.match(r"^[a-z]+$", "orders\n") succeeds and the caller goes on to use "orders\n". #788 and #797 fixed the known whole-string validators with \Z or fullmatch, and #799 fixed the JSON-schema pattern keyword. Nothing kept the class from coming back, and it already had: #793 and #801, merged after #797, each added a $ + .match validator.

This adds tests/security/test_no_dollar_anchored_validators.py, a static gate over fluid_build/. No production file changes.

$ pytest tests/security/test_no_dollar_anchored_validators.py   # with a new validator added to _ids.py
E   These calls run a pattern that ends in `$` with .match or .search, so a value with a final
E   newline passes. End the pattern in \Z or call fullmatch. If the `$` is deliberate, add the
E   call to _ACCEPTED with the reason:
E       fluid_build/build_runners/_ids.py:77 in canary: _CANARY_RE.match(value)
E           pattern '^[a-z]+$', from fluid_build/build_runners/_ids.py:73

How it works

• It reads, it does not import. The scan walks the AST of every module under fluid_build/, so the gate has no dependency on extras or on import side effects.

• It judges the call, not the literal. For every re.match / re.search, every .match / .search on a compiled pattern, and every method reference (filter(RX.match, names)), it follows the pattern back to where it was written:

  • a literal or constant, an f-string, + or str.join;
  • a dict, tuple or comprehension, .get / .items() / .values(), tuple / dict / MappingProxyType / sorted / enumerate;
  • an import, including from . import sub in a package __init__;
  • a helper's return value, a lambda, a property / cached_property;
  • self.X, an inherited or generic base, type(self), RX.pattern.

  A call with fullmatch, sub, findall or split is not reported, and neither is a $ pattern that only such calls use.

• The anchor rule reads the pattern with re's own parser (re._parser, or sre_parse on 3.10). A call is reported when an alternative is anchored at the start (.match anchors it anyway) and can end at a $. It is not reported under MULTILINE, or when the $ follows an unbounded repeat that consumes the newline anyway (\s*$, [^/]*$, (.+)$ under DOTALL). So a half-fixed ^a$|^b\Z, a $ inside a group, VERBOSE and escapes are read the way re reads them.

• A pattern the scan cannot read is reported too: one compiled at run time from data (a schema pattern) and a re.Pattern parameter the caller fills.

• Deliberate $ is listed, with the reason. _ACCEPTED is keyed by (file, function, pattern), so a new function that reuses a listed pattern is reported again. 59 entries:

  • 30: the value is stripped, split or rewritten just before;
  • 4: the value is another function's output;
  • 6: refusals and detectors, where \Z would loosen them;
  • 11: parsers that only read captured groups;
  • 2: judgment calls;
  • 6: patterns read at run time (incl. security: schema validation let a value with a trailing newline through a pattern #799's _pattern_keyword).
    An entry that no longer matches a call fails the test, so the list only shrinks. An entry also pins how many calls it covers (one, unless _ACCEPTED_CALLS says otherwise), so a second call with the same pattern in the same function is reported instead of passing on the first one's reason.

• It cannot pass vacuously. A module it cannot read (syntax, encoding, nesting depth) fails the gate by name, and it must follow more than 150 calls. Name lookups are memoised and each receiver is read once, so long chains stay linear; a test pins this.

Borrow-before-build receipts

• No tool covers this for Python. ruff, pylint, bandit and dlint have no rule; astral-sh/ruff#15017 proposes one (open, needs-decision). CodeQL's py/regex/unmatchable-dollar is a different bug. The Semgrep registry's check-validation-regex is Ruby only.
• Semgrep was the alternative. .semgrep/fluid-rules.yml is already gated in CI, but Semgrep CE propagates values within one file only, so it cannot follow a pattern through an import or a helper's return, and every deliberate site would need a # nosemgrep in production code.
• Borrowed: Brakeman's CheckValidationRegex rule (a validator anchors at both ends), the OpenSSF regex guide's advice (\Z or fullmatch), and the shape of this repo's own AST gates (test_no_unencoded_text_io.py, test_no_reserved_tld_defaults.py, fragments/test_no_raw_readers.py).

Not covered

An unanchored .search(r"\.ya?ml$") is a suffix check, not a whole-string validator. The scan does not follow a pattern into a registry filled by .append or item assignment, into a rule object, into an attribute only a subclass sets, or through +=, getattr, functools.partial or a star import. fluid_build/ has no validator of these shapes today. A test pins that the package imports re as a module, since from re import match would hide calls from the scan. A pattern built with % or str.format cannot be read, so it is reported as built at run time.

JSON-schema pattern keywords run inside jsonschema, out of the scan's sight. FLUID contracts go through schema_manager.schema_validator (#799); the ODCS, ODPS, OPDS, OSI and extension-schema checks still run stock jsonschema, which #811 addresses.

Follow-ups (listed in _ACCEPTED, not changed here)

• build_runners/hooks/quality_gate.py::_check runs a contract's regex gate with re.match, so ^[A-Z]{3}$ passes "ABC\n". It is not reachable today: nothing reads the hook REGISTRY, and the DuckDB runner checks the gate in SQL with regexp_matches (RE2, where $ is the very end). Use fullmatch (or _end_anchored) before the hook is wired.
• cli/import_workflow/dbt.py::_recency_window returns fluid_window verbatim; strip it and return the stripped value, as sla._as_iso_duration does.

Validation

• Green on main (635ea30e): 73 passed on Python 3.10, 3.11, 3.12, 3.13 and 3.14.
• Red where it should be:
  • main before security: refuse a trailing newline in whole-string validators #797 (e59e325c): 44 calls, including every validator security: refuse a trailing newline in whole-string validators #797 fixed and the shapes its literal-only sweep missed (bigquery._validate_ident, a dict of patterns; ODCS _root_identifier and ODPS _is_semver, schema patterns compiled at run time; _patterns.validate_and_quote_identifier, a pattern the caller passes);
  • a new ^[a-z]+$ + .match validator planted in _ids.py: exactly that one report.
• New sites since security: refuse a trailing newline in whole-string validators #797, each read and listed: iac/providers/aws.py::_hive_decimal and gcp.py::_bq_outcome (fix(iac): column types reach Glue and BigQuery faithfully, or validate refuses them #793; column_types.split_type() removes all whitespace), _databricks_load.staging_volume and two databricks_retention parsers (feat(databricks): bronze landing, retention, scoped grants and enforcement follow-ups #801; stripped or whitespace-split first), schema_manager._pattern_keyword (security: schema validation let a value with a trailing newline through a pattern #799; _end_anchored() already turns $ into \Z).
• Gates: whole-tree ruff, the dead-code gate, black 24.10.0 (2038 files unchanged), license headers, lint-imports (6 kept, 0 broken), product guardrail (0 failing), detect-secrets-hook 1.5.0 on the changed file, scripts/mypy_strict.py. tests/security/ with -n 4: 417 passed, 5 skipped.
• ripwire --quality-delta: the 20 gating rows are pre-existing dead code; none is on this file. The new scanner's largest functions are at complexity 16–21 (bar 15).

Reviews

• Security review (Opus judge, own worktree): no blocking findings.

  • It read every security-relevant allowlist entry and its callers (the DuckDB sandbox's _REMOTE_RE, the federation secret-ref allowlist, Snowflake quote_identifier and _validate_policy_signature, SodaCL, forge_db_tools, the Databricks staging volume and jobs, the DLP detectors, the IP-shape refusals, the overlay rules, schema_manager) and found each reason true. For the sandbox it fuzzed 19,608 s3:// strings: the groups are identical under $ and \Z.
  • The test reads only fluid_build/**/*.py, writes only under tmp_path, and imports or runs nothing from the package.
  • It followed every .match / .search call (208 with a pattern; the other 7 are memory-store and argparse .search) and every $-ending literal in the package: each is reported, used with fullmatch or .sub, absorbs the newline, or is not a regex.
  • Its LOW findings: an entry could silently cover a second same-pattern call, fixed with the call counts above; the quality_gate gap, listed above; the third-party schema checks, which security: the ODCS and plugin-extension checks let a trailing newline through a pattern #811 covers.

• Verifier (Sonnet, own worktree, before the rewrite): read the cited code for about 40 entries and found every reason true; no difference across Python 3.10–3.14. Its blind spots were fixed (from . import sub, an exponential call chain, modules dropping out silently, alternation / VERBOSE / escapes, Pattern annotations matched as a substring, containers and wrappers) or listed above under Not covered.

[github-actions]

📄 Documentation Reminder

This PR appears to be missing a documentation reference. Our docs live in a separate repo.

Please update the PR description with one of:

• Link a docs PR — check the "Docs PR linked" box and paste the URL
• Mark as no docs needed — check "No docs needed" with a justification
• Acknowledge docs TODO — check "Docs TODO" and create the docs PR before merge

See the Contributing Guide for details.