Repository navigation
Rule sort refactor - #1989
Rule sort refactor#1989mmatera wants to merge 6 commits into
Conversation
|
|
||
| rules = { | ||
| "CubeRoot[n_?NumberQ]": "If[n > 0, Power[n, Divide[1, 3]], Times[-1, Power[Times[-1, n], Divide[1, 3]]]]", | ||
| "CubeRoot[n_Integer|n_Real|n_Rational]": "If[n > 0, Power[n, Divide[1, 3]], Times[-1, Power[Times[-1, n], Divide[1, 3]]]]", |
There was a problem hiding this comment.
This rule must change, because otherwise would come before CubeRoot[n_Complex], just because WL rules come before FunctionRules.
There was a problem hiding this comment.
Is there a particular example or test that demonstrates why this is right?
| ) | ||
| ) | ||
|
|
||
| for pattern, function in self.get_functions( |
There was a problem hiding this comment.
Now FunctionApplyRules come after RewriteRules.
There was a problem hiding this comment.
In what way do we see the effects of this change?
|
|
||
| def analyze_rule(rule) -> Optional[RuleShape]: | ||
| """Return the RuleShape of `rule`, or None if it is not understood.""" | ||
| cached: Optional[RuleShape] | int = getattr(rule, "_specificity_shape", 0) |
There was a problem hiding this comment.
If the idea is right, this function would play the same role that now play the attribute pattern_precedence. By now it is implemented from "outside" to be able to compare behaviors.
There was a problem hiding this comment.
I also would like to see if these "shapes" can be useful to build a "hash" to classify rules candidates to match against different expressions. This would be the final goal of the "dispatch" refactor.
|
I am glad to see this long-standing problem or black hole being understood. Thanks for putting us on that path. I will be looking at the code in more detail later. One thing that bothers me, because I thought or hoped we went over it, is the terminology. At minimum, it is misleading and may hide various algorithmic choices. (A reason for defining concepts precisely is that if one is looking for pre-existing solutions and technology, as one should try to do, it is easier to find a fit if more precise and commonplace nomenclature is used.) "Sorting" is something that is done on an entire set of elements. A sort key is used in a sort as a comparison function. Here, there is no requirement to "sort" anything. The requirement is that there be an ordering between patterns which determines which pattern to try first. The ordering here is a total ordering because for any pair of arbitrary patterns there is always one "right" answer. The terms "order" and "precedence" are, I think, a better fit for what we have here: whenever there is a possibility of several pattern matches, one of these takes precedence over the others, or precedes the other possibilities. In the (hopefully rare) situation we have exhausted all possibilities, then yes, we will have done the work to sort the patterns. But it is an algorithmic choice whether we want to sort elements eagerly or do this lazily. Yes, ordering that is based solely on properties of a pattern and that does not consider the time or order at which the pattern was defined is a partial order. But once the time at which a pattern was defined is included, we have a total ordering. It is also an algorithmic choice whether we want to break up the total ordering into components, some of which are partial orderings. |
I was working on this for a while, and this is just a basic prototype to discuss the general idea. What maybe is the most valueble thing here are the tests that I added and that I will add later.
I agree with this. However, since I found some problems fitting this behaviour on the concepts I have in mind, I found it better to make my findings explicit, and then try to narrow them.
This difference is something that confuses me a little bit: in Spanish, we have a single verb for both words "ordenar", and I do not understand the nuance between "sort" and "order".
From mathematics, a total order is something that decides which of two elements comes "first", let's say, real numbers have a total order because if you take two different real numbers x and y, x<y or y<x. In matrix analysis, the order between two matrices is a "partial order", defined by the condition X-Y>0 if X-Y has positive eigenvalues. So it could happen that X>Y, X<Y or the order is indecidible. In both cases, the order (or not) is an intrinsic property of the elements, not something external like the history of how these elements are presented.
Precedence is a closer concept: a b^c^d is interpreted as a*(b^(c^d)) because ot the operator precedence, and a grouping rule, which defines the "associativity". However, I am not sure if it is the same that the concept of "rule precedence": in this other case, the precedence order depends both on the partial order of the rules (r<s if r is "more specific than" s) and the time at which both rules are introduced in the definition. So, I was doing some research on how to reproduce what WL does using different algorithms (like bisect left, bisect right and the one I put here) and the one that matches better was this one.
Indeed, we always have that trick: add a timestamp. But I am not sure if this is the most sensible way to get it.
So, in resume, yes, it should be better implementations, and probably there is an specific name for this particular way to assign precedence to rule transformations. But at some point, I would like to find something that works and can be described in clear terms. What this does is fix one issue, and avoid breaking almost anything of the existing tests (up to small adjustments on some particular rules) |
Absolutely! Very useful and thanks! ...
I asked Gemini what Donald Knuth's Volume 3 book "Sorting and Searching" is called in Spanish. Let me know if what it reports makes sense: me: gemini:
me
mmatera
In OO languages, there is fluidity between what is stored in the object and what is outside of it. A pattern can have a time as part of the parameter. "Time:" is intended in the Lamport Timestamp sense (which was derived from the relativity time event sense, or a simulation time counter). It does not have to be, and probably would not be, a calendar date and time.
I see it as a direct extension of pattern operator precedence, which is definitely a thing. When I write in Python: m = r"(?P<tag1>a)|(?P<tag2>.)|(?P<tag3>[a-z])")
print(m.group("tag1")
print(m.group("tag2")
print(m.group("tag3")
yy = r"(?P<tag3>[a-z])|(?P<tag1>a)|(?P<tag2>.)"
m2 = re.match(yy, "a")
print(m.group("tag3")This feels very much the same thing that is happening in WMA. A search for the word Precedence used this way turns up from 22.3.5.2 Operator Precedence from Oracle's SQL documentation describing pattern matching.
Please give me some time to double check this.
Again, to be clear, a timestamp in the Lamport sense. And I agree; I don't know if this is the best way to implement it. It could be done as the position that a pattern appears in a list. But while that's an implementation choice which we may or may not use, the concept still is that there is a total order and the ordering function depends on the pattern as well as its timestamp, however it is implemented.
I wasn't suggesting anything about the implementation. The first step is to understand the concepts clearly and cleanly and in a way that it is likely to be found elsewhere or generally used and understood.
I appreciate that it solves a specific problem while not breaking other things. I just want to make sure we don't get into the situation where we are trying to solve a Rubik's Cube by looking at one piece at a time when the problem is better thought of in terms of the subgroups of the cube. |
|
I have been thinking more about the "sort" vs. "order" usage. In English, outside of computer and mathematical use, they mean about the same thing. However, in mathematics, ordering with the "ing" refers to an existential property, while in computer science, "sort" is a materialization into a list of the existential property. |
| # use insort_left to guarantee that if equal rules exist, newer rules will | ||
| # get higher precedence by being inserted before them. see DownValues[]. | ||
| bisect.insort_left(values, rule, key=lambda x: x.pattern_precedence) | ||
| # Mathematica orders rules with a partial order: a rule goes before |
There was a problem hiding this comment.
Here we seem to do eager sorting (as before) instead of, say, lazy-on-demand order improvement.
That said, we probably could still use bisect.insort by storing the information needed to make the sort key a total ordering.
| @@ -0,0 +1,323 @@ | |||
| """ | |||
There was a problem hiding this comment.
This module seems a bit complex, and I wonder if there is a way to make it seem more natural or simpler.
There was a problem hiding this comment.
The module has two parts, the one that computes the "RuleShape" and the part that compares two RuleShapes to decide if one is more general than the other one, or if the order is indeterminate. The first part should be similar to the "pattern_precedence", and should be delegated to the Rule class an Pattern subclasses. This module is just a quick and dirty way to test that this behavior improves the current behavior in master.
|
@mmatera: I have been trying (unsuccessfully) to get this working. I have been doing this in rule_sort_refactor-update. It looks to me from recent CProfile data in https://github.com/orgs/Mathics3/discussions/2065 this area may be one of the reasons Mathics3 is so slow. A question: suppose I have rules: And now, in my Mathics3 program, I refer to |
I guess this does not work because of a merge issue. In any case, maybe the most valuable thing here are the new tests. I have some work in progress for a better implementation that would also support the log(n) lookup (this does not do that). |
First and foremost, thanks for being such a good sport about the breakage and having to rework this. I have some thoughts now that I understand what's happening better. However, some of this has to be a little hand-wavy. The whole thrust of pattern-matching technology is to avoid linear searches over every pattern. The problem we or any Mathematica implementation has, though, is trying to map existing work into WMA's situation. While the power of extended regular expressions and WMA's patterns may be theoretically equivalent, figuring out how to use and map existing well-studied and proven technology to the WMA situation, I think, will require a bit of thought, discussion, and benchmarking. Finally, I'd like to add that using hash tables or dictionaries for literal patterns like F[1], F[2], ... is an O(1) operation, which is better than BTW, to me, this again emphasizes the stupidity in thinking that rewriting this in C or using Cython is somehow going to fix such fundamental algorithmic flaws. |
I totally agree. The new tests are much needed and invaluable. I had, however, to ask Claude Code to give me simple examples. While the two examples are not 100% the same and in different frameworks, showing the difference I hope may be instructive. Instead of: Consider instead: # Simple overwrite of the same literal
ClearAll[f];
f[3] = 9;
f[3] = new;
f[3]
# Overwrite while another literal rule exists
ClearAll[f];
f[1] = a;
f[2] = b;
f[1] = c;
DownValues[f]
# Overwrite after Unset
ClearAll[f];
f[1] = a;
f[2] = b;
f[1] =.;
f[1] = c;
f[1] = d;
DownValues[f]Notice that each test tests one thing, and one thing only. I am guessing that in @tombouctou comments on running tests, and some were expected to fail due to the way he was running the test, stuff like this was getting in the way of running tests in parallel and independently. |
|
Here we have two different problems that could be solved at a time. One is the ordering of the rules: as the order in which available rules are applied to an expression in the evaluation process is different between WMA and Mathics3, the results are different, and that causes some evaluations to crash. The other is a performance problem: we try to apply a lot of rules that we could avoid. In the example of F[1]=1; F[2]=2;... F[10000]=10000; we pass through all the rules before getting the right one. The trick that WMA uses to avoid this linear cost is to use hash tables, in a way that expressions with certain symbols and a certain n-arity look first for rules associated with these symbols and n-arities. Moreover, patterns that are literal expressions (like the one in the example) can be directly dispatched using a dictionary. Regarding the first problem, the problem is that the ordering criteria is not intrinsic to the rule, but involves the time in which the rule is added to a definition. Intrinsic properties of the rules do not provide a total order but a partial order. Still, this partial order has precedence over the timestamp ordering. The other problem requires classifying rules, and for that, we can use the information collected to build the sort key used for the partial order. This is what I am trying to code now. |
Since pytest 9.2.0 came out (2026-10-27), the "Run performance benchmarks" job fails on every pull request, including #1989, #2032, #2069 and #2070. All 73 benchmarks error at setup: ``` AttributeError: 'Function' object has no attribute '_nodeid'. Did you mean: 'nodeid'? ``` pytest-benchmark 5.3.0, its latest release, uses a private attribute that pytest 9.2 removed: ionelmc/pytest-benchmark#321, still open. The last passing run (master, 2026-10-27 10:20) had pytest 9.1.1; the failing ones have 9.2.0. pytest-benchmark is 5.3.0 in both. This keeps pytest below 9.2 in the benchmark workflow, with a comment pointing at the upstream issue, so the pin can go once pytest-benchmark is fixed. `pyproject.toml` does not pin pytest, so the later `pip install -e .[dev,full]` keeps 9.1.x. Checked with a one-test benchmark: pytest 9.2.0 + pytest-benchmark 5.3.0 gives the same error, and pytest 9.1.1 + pytest-benchmark 5.3.0 passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
@mmatera, since you're coding the classification now, I won't start a parallel version. Here is what I can take on, if it helps:
Tell me which of these you want, or none, and I'll keep out of your way otherwise. |
Following the suggestion in #2067 ("the same thing for matching rule and element order precedence"). This adds a new tutorial section to the Manual, "Order of Rules and of Elements", right after "Functions and Patterns". - **Which Rule Is Tried First**: the more specific rule is used. A value such as `f[1]` comes before `f[x_]`, `x_Integer` before `x_`, `_` before `__` before `___`, a conditional rule before the same rule without a condition, and `{x_}` before `x_List` before `x_`. A rule with the same left-hand side replaces the old one. - **Canonical Order**: what `Sort`, `Order` and `Orderless` use. Numbers come first, then strings, then symbols, then other expressions. Strings and symbols are ordered without regard to case, with lower case first. Monomials are ordered as the terms of a polynomial, and general expressions by head, then by length, then element by element. Every example is a doctest, and every result was checked against WMA 15. I left out the cases where Mathics3 and WMA differ: - Rules that cannot be compared, such as `f[x_?EvenQ]` and `f[x_?PrimeQ]`: WMA tries them in the order of definition. That is #1978, and the draft #1989 addresses it. - `Optional` and repeated pattern names. - The place of sums in the canonical order. These are listed, with WMA 15 values, in the companion developer's guide page, Mathics3/Mathics3-development-guide#51. `docpipeline.py -c "Language Tutorial"`: all pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: R. Bernstein <rocky@users.noreply.github.com>
This PR implements an alternative way to sort rules, which seems to be closer to the one in WMA. The rule order is not a total order but a partial order, where one rule comes before other rule if the first is more specific than the second. Two rules like
F[x_Integer]andF[x_Real]are not comparable, so they are added to a set of rules (let's say, downvalues) in the order they are introduced.This would solve #1978