Repository navigation
Conversation
tamnd
added a commit
that referenced
this pull request
Nov 21, 2026
Refers to #2. This PR adds the lean4export NDJSON 3.x parser of spec 07.1 to `rulean-format`. It is stacked on #24, because it writes into the term store. ## What it does - `json.rs` is the JSON layer. It reads one object into a table of known keys, and it reads integers, booleans, strings and arrays. It checks every other value for well-formed JSON and then skips it. The skip keeps the nesting on a stack, so a deep `mdata` value uses no call stack. - `ndjson.rs` is the record parser. A name, level or expression record goes into the store, and its handle goes into the index table of its kind. A declaration record becomes a `Decl`. - `decl.rs` is the declaration stream of spec 04.7. It keeps every field of the input, also the fields that the checker computes again. The checker compares them and does not trust them (06.1). - `error.rs` has the error kinds. Each kind is a reject or a decline. An unknown version, an index above the table limit and a full store are declines. ## Input rules of 07.1.2 - An index must be defined before it is used, and it must not be defined two times. Gaps and any order are permitted. Name 0 and level 0 are implicit. - Duplicate keys are rejects, also when one key has an escape. - An integer with a sign, a fraction or an exponent is a reject. A value above 2^64 - 1 is a reject. A count field above 2^32 - 1 is a reject. - Each line must be valid UTF-8. A `\u` surrogate without its pair is a reject. - `natVal` has no leading zeros except for "0". The parser checks the bound of 05.8 from the string length before the conversion, and from the bit count after it. - `mdata` is stripped. The record index gets the handle of the inner expression. - An empty line, a truncated last line, a missing meta line and a second meta line are rejects. Major version 3 with minor version 0 or 1 is accepted. ## Tests - `tests/rules.rs` has one test for each rule above. - `tests/rules.rs` also changes, deletes and adds bytes in a valid input 20,000 times. No run panics. - `tests/arena.rs` parses all Arena tests when `RULEAN_ARENA` or `_tmp/arena/tests` exists. Locally, all 225 tests parse, with 16,809 declarations. CI does not fetch the Arena in the test job yet. The next PR adds `rulean check --parse-only` on the Arena in CI. ## Notes - The code is in the Aeneas subset: no `?`, no closures and no early return in a loop. The first error of a line goes into the scanner state, and the rest of the line runs on dummy values. The parser writes into the store only when there is no error. - Performance is not tuned yet. In release mode the 50 MB of the Arena parse in about 1.8 s. The target of 07.1.4 is for M4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the term store of spec 04 to
rulean-store. It is on top of #23, because the store keeps natural number literals asrulean_nat::Nat.What the store has:
ExprIdis au32with the tier in bit 31, the kind in bits 27 to 30 and the index in bits 0 to 26 (spec 04.2.1). The kind needs no memory load.Table, the open-addressing table of 04.2.4. It maps a hash tou32values, and the caller compares its own key on each value with the same hash. So the table needs no closure and no trait, and a lookup never trusts a hash alone. The table can carry epochs, so thatclearis O(1) for the scratch tier.NameId(0)is the anonymous name andLevelId(0)is zero, as in the export.Store::memorygives the bytes of each part, for the measure of 04.9.The store does not normalize levels. That is the job of the checker in M2. The scratch tier comes with the checker too. This PR has only the frozen tier, but the handle already has the tier bit.
Capacity. When the array of one kind has 2^27 nodes, the store sets a flag and returns a handle to an old node. The terms are then wrong, so the driver must check
Store::is_fulland decline before it gives a verdict. The crate documentation says this.Subset. The code stays in the subset of spec 10.6. The table method is
putand notinsert, because the subset lint reports.insert()on any receiver.rotate_left,wrapping_mulandis_emptyare outside the modelled functions, so the code has its own forms of them.This PR covers the two store items of the checklist. The twin and the R2 lemmas for the store are a separate item.
Refers to #2