Skip to content

Grain: index the constant pool by hash - #1202

Merged
schungx merged 4 commits into
rhaiscript:mainfrom
luispedro:grain-const-pool-hash
Oct 31, 2026
Merged

schungx merged 4 commits into
rhaiscript:mainfrom
luispedro:grain-const-pool-hash

Conversation

@luispedro

Copy link
Copy Markdown
Contributor

Fixes #1196, as discussed there: the constant pool is indexed by each Dynamic's hash, so compile time is linear in the number of constants rather than quadratic.

Changes

  • push_const hashes each constant with calc_switch_value_hash. A StraightHashMap maps each hash to the indices of the pooled constants with that hash.
  • On a hash match, same_constant checks that the two values really are equal: type first, strings directly, and other values by {:?} rendering. A hash collision cannot merge two different constants.
  • Only poolable constants (constant_caps is Some) are deduplicated; anything else is appended. Dynamic::hash panics on custom types and timestamps, so this is required. It also fixes an existing bug: custom-type values all render as their type name, so the old scan merged distinct ones. For example, with two custom-syntax states of a custom type, [tag 1, tag 2] returned [1, 1] under Grain. The new test custom_type_states_stay_apart fails on main and passes here.
  • The pool keeps emission order, so disassembly is unchanged.
  • The comment above Expr::Map in the compiler now says what the parser's template actually holds (every key with () as its value).

The parser-side change (putting constant values in the map template so they can be skipped during evaluation) is left for a separate PR, as suggested in the issue.

Timing

The repro from the issue: Grain compile only, release build, median of 11 compiles.

entries main (from the issue) rendering index (BTreeMap<String, u32>) this PR
2000 ~440 ms 1.06 ms 0.41 ms
16000 — 10.1 ms 3.5 ms

Checks

  • cargo test --features internals,grain --tests and --lib pass.
  • cargo fmt --all -- --check passes. Clippy reports nothing on the changed code.
  • cargo check is warning-free with grain plus each of no_object, no_index,no_float, no_closure, sync, no_custom_syntax, no_function, and only_i32,f32_float,decimal. The no_std,grain build has the same 3 warnings as main.

🤖 Generated with Claude Code

@schungx

schungx commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Suggestion: A hash is considered absolute. It already hashes the type of the data, so an int will not match a char ot a bool or a string.

Custom types will never arrive into constants since they are not poolable. The recognized ones are handled by the hashing.

Rhai assumes that such hashing is unique so there is no need to consider collisions - it is assumed to be practically impossible in Rhai.

So the index can simply map to one constant offset and there is no need for same_constant.

@schungx schungx added enhancement vm Issues related to the Rhai Grain bytecodes compiler and VM. labels Oct 27, 2026
@schungx
schungx requested a review from ImTheSquid October 27, 2026 08:22
Comment thread src/grain/compile/mod.rs Resolved
@schungx

schungx commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

@luispedro Would you have time to address this? I can make a PR with the revisions otherwise. :-D

luispedro and others added 3 commits October 30, 2026 15:45
push_const found a duplicate by scanning the pool and rendering every
pooled constant with {:?} again for each new one, which made compiling
quadratic in the number of constants. A map literal with any
non-constant value made it much worse: its template, holding every key,
is pooled first, then each entry's key and value, and each of those
lookups rendered the whole template again.

Index the pool by each constant's hash, and on a hash match compare the
two constants (type first, strings directly, other values by
rendering). Only poolable constants are shared: anything else need not
be hashable, and two custom-type values render the same (as the type's
name) yet differ, so the old scan merged them. The pool keeps emission
order.

Compiling a 2000-entry map literal with one computed value goes from
~440 ms to ~0.4 ms.

Fixes rhaiscript#1196.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A constant's hash covers its type and Rhai treats it as unique, so a
matching hash is the same constant; same_constant is not needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@luispedro
luispedro force-pushed the grain-const-pool-hash branch from 4c2b332 to 15f1636 Compare October 30, 2026 05:48
@luispedro

Copy link
Copy Markdown
Contributor Author

Updated, but always feel free to just take my stuff and edit/merge/discard....

@schungx

schungx commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Rhai Grain no longer merges distinct constants of a custom type (such as two custom syntax states) in the constant pool because they print the same.

I wonder what error this is... since the Grain constants pool should not even pool custom syntax...

EDIT: I have fixed the merge conflicts.

@schungx
schungx requested a review from ImTheSquid October 30, 2026 06:27
@schungx
schungx merged commit 4aad29c into rhaiscript:main Oct 31, 2026
56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement vm Issues related to the Rhai Grain bytecodes compiler and VM.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Grain: compiling a large map literal with any non-constant value is quadratic (constant pool dedup)

3 participants