Evaluate match() and search() as I-Regexp, compare content types by media type (dpp-criteria 203202e)

[fabianekc]

Adapts dpplint to dpp-criteria 203202e (PR #5): content type matching under "Order of evaluation within a request", regular expressions inside JSONPath under "Regular expressions", and unusable patterns under "Results" in CRITERIA-FORMAT.md.

Changes

• Regular expressions inside JSONPath: match() and search() in applies_if now evaluate their pattern as I-Regexp (RFC 9485), as RFC 9535 requires. They use the new class IRegexp, which parses the pattern with the ABNF of RFC 9485 and translates it into a Ruby Regexp.
  • match() needs the entire value, search() a substring.
  • . matches anything except LF and CR; matching is case-sensitive and counts code points.
  • Only the RFC 9485 escapes and \p{..}/\P{..} exist.
  • Following RFC 9535, the result is false for values that are not strings and for patterns that do not conform.
• Paths in applies_if:
  • The path parser reads JSONPath string literals in single or double quotes, with the escapes of RFC 9535 2.3.1.1.
  • The filter selects array elements or member values, and nothing from a string.
  • Fixed: a number such as 59040 no longer matches '59040' by being treated as a string, and exists now holds for null and false values.
• matches in applies_if: uses ECMA-262 (EcmaRegexp), like all JSON assertions.
• Unusable applies_if condition: an invalid matches pattern, an unsupported path or an invalid string literal skips the criterion with the reason. Previously an unsupported path raised an error.
• content_type:
  • The expected value is compared as a media type without parameters, case-insensitively, like the response already was.
  • +json types such as application/ld+json count as JSON media types.
  • A body that does not parse gives "response is not valid JSON". In resolve the body must be a single JSON object; this belongs to the content type check, so headers are not evaluated if it fails.

Tests

• 159 runs, 522 assertions, 0 failures, 0 errors, 0 skips in the image built with ./build.sh (dpp-criteria 203202e).
• I-Regexp:
  • match() versus search().
  • ^ and $, following the JSONPath Compliance Test Suite cases "explicit caret" and "explicit dollar".
  • . on U+2028, LF and CR; the categories \p{Lu} and \P{L}; values that are not strings.
  • Non-conforming patterns.
  • The patterns of DPP-BAT-002 ([Bb]atter) and DPP-PCDS-008 (59040|PCDS|pcds).
• Cross-checked against jsonpath-rfc9535 (Python): 2296 cases (41 patterns × 28 values × match/search), 0 differences.
• applies_if: both criteria, filter on array, object and string, invalid I-Regexp with exists: true and exists: false, string literals and escapes, exists on null and false, the skip reasons.
• content_type: parameters, upper case, application/ld+json, invalid JSON, a body that is not an object, a type that is not JSON.

Reference passport

https://dpp.oydapp.eu/01/09520123456788/21/000001:

• Overall: 13 of 13 automated checks passed, 0 failed, 1 warning (DPP-ID-016), 9 skipped.
• DPP-DAT-016 now passes because the reference service changed. With Accept text/html it now returns text/html with Vary: Accept, Origin; with */* it still returns JSON.
• DPP-BAT-002, DPP-PCDS-008 and DPP-ID-010 are skipped (condition not met): the passport has neither contentSpecificationIds nor facilityId.

Results are from automated checks only and establish no presumption of conformity.

Decisions

• ^ and $ in I-Regexp anchor the whole value. The RFC 9485 ABNF lists them as ordinary characters, but the mappings in RFC 9485 section 5 and the Compliance Test Suite treat them as anchors, and so does jsonpath-rfc9535.
• [z-a] and {n,m} with m < n count as non-conforming (XML Schema, on which I-Regexp is based, rejects them).
• A non-conforming I-Regexp in JSONPath gives false, as RFC 9535 says. This is provisional, see open question 1.

Open questions for dpp-criteria

1. RFC 9535 makes match()/search() false for a non-conforming pattern, while "Results" asks for skipped before any request, never failed. With exists: false, or equals/in on the filter result, the RFC behaviour lets the criterion run and possibly fail. Which one applies to patterns inside JSONPath? The current criteria are not affected.
2. Should the format settle whether ^ and $ in JSONPath patterns are anchors or ordinary characters, or rule them out?
3. How does matches of a JSON assertion, including in applies_if, apply to values that are not strings?