Add a page on the order of rules and the canonical order by tombouctou · Pull Request #51 · Mathics3/Mathics3-development-guide

tombouctou · 2026-10-28T16:07:26Z

Following the suggestion in Mathics3/mathics-core#2067, for rule matching and element order. The user side is in the Manual, Mathics3/mathics-core#2070.

This adds a new page, code-overview/rule-and-element-order, after "Rules":

Rule order.
- The WMA model: more specific rules come first, by a partial order; rules that cannot be compared are tried in the order of definition; the same left-hand side replaces the old rule; user rules for a builtin come before its own evaluation.
- In Mathics3: where rules are kept (insert_rule and its fingerprint index), and how they are tried (upvalues, then downvalues or subvalues, the literal index, compiled rules in Replace).
- Where Mathics3 differs from WMA 15. A table gives the cases where the result of an evaluation differs (e.g. f[x_?EvenQ] := even; f[x_?PrimeQ] := prime; f[2] is even in WMA and prime in Mathics3), with a reference to Wrong order for matching in Rule mathics-core#1978 and the draft #1989. The fine structure of pattern_precedence is left out until #1989 lands, since it replaces it.
Canonical order. element_order and KeyComparable, the *_ELT_ORDER classes, and how numbers, strings, symbols, monomials (Monomial) and general expressions are ordered. Also the one difference found: sums, which WMA orders with the polynomials.

Every WMA value on the page was checked with WMA 15. sphinx-build -b html docs gives no warning for the new page.

🤖 Generated with Claude Code

The rule order: the WMA model (a partial order by specificity, the definition order for rules that cannot be compared), where Mathics3 keeps rules (insert_rule and its index), how they are tried (upvalues, then downvalues or subvalues, the literal index, compiled rules in Replace), and where Mathics3 differs from WMA 15 (Mathics3/mathics-core#1978, #1989). The canonical order: element_order, the *_ELT_ORDER classes, strings, symbols, monomials and general expressions, and the place of sums, which differs from WMA. The user-level description is in the Mathics3 Manual. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Following the suggestion in #2067 ("the same thing for matching rule and element order precedence"). This adds a new tutorial section to the Manual, "Order of Rules and of Elements", right after "Functions and Patterns". - **Which Rule Is Tried First**: the more specific rule is used. A value such as `f[1]` comes before `f[x_]`, `x_Integer` before `x_`, `_` before `__` before `___`, a conditional rule before the same rule without a condition, and `{x_}` before `x_List` before `x_`. A rule with the same left-hand side replaces the old one. - **Canonical Order**: what `Sort`, `Order` and `Orderless` use. Numbers come first, then strings, then symbols, then other expressions. Strings and symbols are ordered without regard to case, with lower case first. Monomials are ordered as the terms of a polynomial, and general expressions by head, then by length, then element by element. Every example is a doctest, and every result was checked against WMA 15. I left out the cases where Mathics3 and WMA differ: - Rules that cannot be compared, such as `f[x_?EvenQ]` and `f[x_?PrimeQ]`: WMA tries them in the order of definition. That is #1978, and the draft #1989 addresses it. - `Optional` and repeated pattern names. - The place of sums in the canonical order. These are listed, with WMA 15 values, in the companion developer's guide page, Mathics3/Mathics3-development-guide#51. `docpipeline.py -c "Language Tutorial"`: all pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: R. Bernstein <rocky@users.noreply.github.com>

…as in WMA (#2073) Fixes most of #2071. The canonical order (`Sort`, `Order`, `Orderless` heads) now follows WMA 15 in three more ways. **1. Sums.** A sum whose largest term is a monomial sorts with the monomials of its largest variable, after all of them. Its key is that variable to an infinite power (`Expression._sum_element_order`). Two such sums are compared term by term from the largest term down, and if one runs out first, it comes first. ``` Sort[{a, a + b, b, c, x}] {a, b, a + b, c, x} Sort[{x, 1 + x, y, x^2, 1 + x^2, x y}] {x, x^2, 1 + x, 1 + x^2, y, x y} Sort[{f[x], 1 + x, g[x], a + b + c, h[a, b, c]}] {a + b + c, 1 + x, f[x], g[x], h[a, b, c]} Collect[a x + b y + c x + d y, y] a x + c x + (b + d) y ``` `Monomial` now handles two equal infinite exponents; before, `inf - inf` gave `nan`. **2. Symbols from other contexts.** Symbols are ordered by their names without the context, and the full name only breaks ties (`wma_symbol_sort_key`). Before, `Global`x` came before `System`Pi`: ``` x + Pi Pi + x (was x + Pi) Sort[{x, Pi, E, a, z}] {a, E, Pi, x, z} ``` **3. Coefficients.** Monomials that differ only in a numeric coefficient are ordered by it: `Sort[{x, -x, 2 x}]` is `{-x, x, 2 x}`. ## How much closer to WMA I took 78 expressions (numbers, strings, symbols including `Pi` and `E`, monomials with coefficients, sums, products with general factors, general expressions) and computed `Order` for all 3003 pairs in WMA 15 and in Mathics3: | | pairs that differ from WMA | |---|---| | master | 1152 | | this PR | 425 | 767 pairs are fixed. **40 get worse**, all of one kind: products with a factor that is not a symbol, such as `Pi g[x]` or `Sqrt[Pi] Erf[1/x]`. `element_order` skips such factors in a product. Before this PR, the context in `System`Pi` happened to hide that. WMA treats `f[x]`, `Erf[1/x]` and `E^x` as variables of the monomial. That is the next step, in a separate PR. Because of it, `Integrate[Exp[-1/x^2], x]` in `test_calculus.py` is marked `xfail`, and one `Hypergeometric1F1` doctest now shows the other order of its two terms. Still left as before (listed in #2071): - a sum whose largest term has a negative coefficient, such as `1 - x`, or a sum with a negative constant: WMA puts these *before* their largest variable; - powers and sums of general expressions, such as `f[x]^2` and `1 + f[x]`. ## Tests - New `test_canonical_order` in `test/builtin/test_testing_expressions.py`: 16 cases with WMA 15 values; 13 of them fail on master. - Doctests and tests whose expected output was the old order are updated (15 doctests, 8 test cases). I checked each new output against WMA 15. Where WMA simplifies differently (e.g. `Sqrt[2/Pi]` in `BesselJ[1/2, x]`), the order of the factors still matches WMA. - Full run on kimmy: **doctests 4622 passed, 0 failed**. pytest with `-n 4 --dist loadfile`: 15 failed. They are in `test_tensor.py`, `test_eol.py`, `test_latex.py` and `test_diffeqns.py`, and those files pass on their own (`test_diffeqns.py` fails on master in the same run too). These are the order-dependent failures that master has as well. The developer's guide page on the canonical order (Mathics3/Mathics3-development-guide#51) lists the place of sums as a difference from WMA; I'll update it there once this lands. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: R. Bernstein <rocky@users.noreply.github.com>

Navigation Menu

Repository navigation

Add a page on the order of rules and the canonical order - #51

Add a page on the order of rules and the canonical order#51
tombouctou wants to merge 1 commit into
Mathics3:masterfrom
tombouctou:rule-and-element-order

tombouctou commented Oct 28, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

tombouctou commented Oct 28, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant