Evaluate match() and search() as I-Regexp, compare content types by media type (dpp-criteria 203202e)

README.md

@@ -26,7 +26,9 @@ and do not establish a presumption of conformity.
 - Criteria with `check.type: resolve` request the product identifier
   themselves (with the Accept header the criterion names) and need
   `GET /api/v1/validate/<product identifier>`; with `POST` they are skipped.
-  `expect` checks status and content type first and the header fields
+  `expect` checks status and content type first (media type without
+  parameters, case-insensitive; for `application/json` and `+json` types the
+  body must be a single JSON object) and the header fields
   (`exists`, `equals`, `contains`, `matches`; field names case-insensitive,
   several fields of one name combined with commas) only if both hold.
   `matches` is an ECMA-262 regular expression without flags, searched
@@ -37,6 +39,13 @@ and do not establish a presumption of conformity.
   evaluated independently. A
   criterion fails if a check with severity error fails; if only checks with
   `severity: warning` fail, the result is `warning`.
+- `applies_if` supports the paths `$.<member>` and
+  `$.<member>[?search(@, '<regex>')]` or `[?match(@, '<regex>')]`. The
+  regular expression of `search()` and `match()` is an I-Regexp (RFC 9485) as
+  RFC 9535 requires: `match()` needs the entire value, `search()` a substring,
+  and a pattern that does not conform selects nothing. `matches` of the
+  condition itself is ECMA-262 as above. A condition dpplint cannot evaluate
+  skips the criterion with the reason.
 - Criteria with `check.type: did` send the DIDs of the passport to
   [didlint](https://didlint.ownyourdata.eu) (DID Core, DID Resolution) and
   check linked verifiable presentations for the VC Data Model 2.0 context. The

app/services/applies_if.rb

@@ -1,38 +1,136 @@
 # Evaluates the applies_if conditions of a criterion on the passport JSON.
-# Supported paths: "$.attribute" and "$.attribute[?search(@, '<regex>')]".
+# Each condition is a JSON assertion (CRITERIA-FORMAT.md): a JSONPath `path`
+# and one of `exists`, `equals`, `in`, `matches`.
+#
+# Supported paths (RFC 9535):
+#   $.<member>
+#   $.<member>[?search(@, '<I-Regexp>')]
+#   $.<member>[?match(@, '<I-Regexp>')]
+# The string literal may be single- or double-quoted with the escapes of
+# RFC 9535, 2.3.1.1. The filter selects the elements of an array (or the
+# member values of an object) for which the function holds; search() and
+# match() evaluate their pattern as I-Regexp (RFC 9485, see IRegexp), are false
+# for values that are not strings and, following RFC 9535, false for a pattern
+# that does not conform to RFC 9485.
+#
+# `matches` of the assertion itself is an ECMA-262 regular expression
+# (see EcmaRegexp), as for all JSON assertions.
 class AppliesIf
-  SIMPLE = /\A\$\.([A-Za-z_][A-Za-z0-9_]*)\z/
-  SEARCH = /\A\$\.([A-Za-z_][A-Za-z0-9_]*)\[\?search\(@,\s*'(.*)'\)\]\z/
+  MEMBER = /[A-Za-z_\u0080-\u{10FFFF}][A-Za-z0-9_\u0080-\u{10FFFF}]*/
+  SIMPLE = /\A\$\.(#{MEMBER})\z/
+  FILTER = /\A\$\.(#{MEMBER})\[\s*\?\s*(match|search)\(\s*@\s*,\s*(.*)\s*\)\s*\]\z/m
+  ESCAPES = { "b" => "\b", "f" => "\f", "n" => "\n", "r" => "\r", "t" => "\t", "/" => "/", "\\" => "\\" }.freeze
+
+  class Unsupported < StandardError; end
 
   def self.holds?(conditions, passport)
     Array(conditions).all? { |c| new(c, passport).holds? }
   end
 
+  # nil if every condition can be evaluated, otherwise the reason. A criterion
+  # with such a condition is skipped.
+  def self.problem(conditions)
+    Array(conditions).each do |c|
+      parse_path(c["path"].to_s)
+      next unless c.key?("matches")
+
+      reason = EcmaRegexp.problem(c["matches"])
+      return "regular expression #{c['matches'].to_s.inspect} in applies_if #{reason}" if reason
+    end
+    nil
+  rescue Unsupported => e
+    e.message
+  end
+
+  # [member, nil, nil] or [member, "match" | "search", pattern]
+  def self.parse_path(path)
+    if (m = path.match(SIMPLE))
+      [m[1], nil, nil]
+    elsif (m = path.match(FILTER))
+      [m[1], m[2], string_literal(m[3].rstrip)]
+    else
+      raise Unsupported, "applies_if path #{path} is not supported in this version"
+    end
+  end
+
+  # A JSONPath string literal (RFC 9535, 2.3.1.1).
+  def self.string_literal(literal)
+    quote = literal[0]
+    unless literal.size >= 2 && %w[' "].include?(quote) && literal[-1] == quote
+      raise Unsupported, "applies_if: #{literal} is not a JSONPath string literal"
+    end
+
+    chars = literal[1...-1].chars
+    out = +""
+    until chars.empty?
+      c = chars.shift
+      if c == "\\"
+        out << unescape(chars, quote, literal)
+      elsif c == quote || c.ord < 0x20
+        raise Unsupported, "applies_if: #{literal} is not a valid JSONPath string literal"
+      else
+        out << c
+      end
+    end
+    out
+  end
+
+  def self.unescape(chars, quote, literal)
+    e = chars.shift
+    return e if e == quote
+    return ESCAPES[e] if ESCAPES.key?(e)
+    raise Unsupported, "applies_if: invalid escape in JSONPath string literal #{literal}" unless e == "u"
+
+    code = hex4(chars, literal)
+    if (0xD800..0xDBFF).cover?(code)
+      raise Unsupported, "applies_if: lone surrogate in #{literal}" unless chars.shift(2) == ["\\", "u"]
+
+      low = hex4(chars, literal)
+      raise Unsupported, "applies_if: lone surrogate in #{literal}" unless (0xDC00..0xDFFF).cover?(low)
+
+      code = 0x10000 + ((code - 0xD800) << 10) + (low - 0xDC00)
+    elsif (0xDC00..0xDFFF).cover?(code)
+      raise Unsupported, "applies_if: lone surrogate in #{literal}"
+    end
+    code.chr(Encoding::UTF_8)
+  end
+
+  def self.hex4(chars, literal)
+    hex = chars.shift(4).join
+    raise Unsupported, "applies_if: invalid \\u escape in #{literal}" unless hex.match?(/\A\h{4}\z/)
+
+    hex.to_i(16)
+  end
+
   def initialize(condition, passport)
     @c = condition
     @passport = passport
   end
 
   def holds?
     values = select
-    return values.any? == @c["exists"] if @c.key?("exists")
+    return !values.empty? == @c["exists"] if @c.key?("exists")
     return values.any? { |v| v == @c["equals"] } if @c.key?("equals")
     return values.any? { |v| @c["in"].include?(v) } if @c.key?("in")
-    return values.any? { |v| v.to_s.match?(Regexp.new(@c["matches"])) } if @c.key?("matches")
+    return values.any? { |v| EcmaRegexp.search?(@c["matches"], v.to_s) } if @c.key?("matches")
 
-    values.any?
+    !values.empty?
   end
 
   private
 
   def select
-    path = @c["path"].to_s
-    if (m = path.match(SIMPLE))
-      @passport.key?(m[1]) ? [@passport[m[1]]] : []
-    elsif (m = path.match(SEARCH))
-      Array(@passport[m[1]]).select { |v| v.to_s.match?(Regexp.new(m[2])) }
-    else
-      raise ArgumentError, "unsupported applies_if path #{path}"
-    end
+    member, function, pattern = self.class.parse_path(@c["path"].to_s)
+    return [] unless @passport.is_a?(Hash) && @passport.key?(member)
+
+    value = @passport[member]
+    return [value] unless function
+
+    candidates = case value
+                 when Array then value
+                 when Hash then value.values
+                 else []
+                 end
+    candidates.select { |v| IRegexp.public_send("#{function}?", pattern, v) }
   end
 end

app/services/ecma_regexp.rb

@@ -1,5 +1,6 @@
 # Regular expressions as CRITERIA-FORMAT.md of dpp-criteria defines them for
-# `matches` ("Regular expressions"): ECMA-262 syntax without flags, searched
+# `matches` in JSON and header assertions, including applies_if ("Regular
+# expressions"): ECMA-262 syntax without flags, searched
 # anywhere in the value (no implicit anchoring), case-sensitive.
 #
 # Ruby's own Regexp differs from ECMA-262 in ways that change results, above
@@ -14,10 +15,11 @@
 # Patterns that are not valid ECMA-262 raise Invalid. Valid patterns that use
 # features outside the portable subset of CRITERIA-FORMAT.md, which dpplint
 # does not evaluate (lookaround, named groups, backreferences, legacy octal
-# escapes, \k, lone surrogates), raise Unsupported. Known remaining deviation:
-# characters outside the Basic Multilingual Plane are one character here, two
-# UTF-16 code units in ECMA-262 (relevant only for `.`, classes and
-# quantifiers applied to such a character).
+# escapes, \k, lone surrogates), raise Unsupported. Characters outside the
+# Basic Multilingual Plane count as one character here, as two UTF-16 code
+# units in ECMA-262; CRITERIA-FORMAT.md leaves results that depend on this
+# undefined. Regular expressions inside JSONPath (match(), search()) are
+# I-Regexp, see IRegexp.
 class EcmaRegexp
   class Error < StandardError; end
   class Invalid < Error; end