fix(semantic): keep the trailing trivia of an expansion's placeholders and blocks

[orizi]

A macro rule's expansion is turned into text node by node, and the handlers for the two nodes whose
text is replaced - a placeholder and a $() block - returned before the terminal case that emits a
node's text, so the trivia those nodes carry was dropped. The space a rule writes after one of its
tokens is the only thing keeping the expanded tokens apart, so dropping it welded them together:
macro glue { ($a:ident, $b:ident) => { { $a $b } }; } expanded glue!(_x, _y) to { _x_y }, an
identifier the call site then failed to find (E0006), and macro keep_plus { ($a:ident + $b:ident) => { $a + $b }; } expanded to x+ y. rustc never welds them: its expansion is a token stream, so
the equivalent macro_rules! rule keeps _x and _y two tokens and reports "macro expansion
ignores identifier _y and any tokens following".

expand_node now emits the trailing trivia of a placeholder and of a $() block after the text
that replaces it.

The leading trivia of those nodes stays dropped, deliberately: a placeholder's value has to start
exactly where its CodeMapping does, because Resolution::new maps a path of the expanded code
back to the call by AsSegments::offset, which is the offset of the path's syntax node including
its leading trivia. Emitting the indentation a rule writes before a placeholder moves the value past
the start of its mapping, the mapping is missed, and the value resolves at the definition site
instead - corelib's sha256::dec_and_append_or_return!, whose rule indents $remaining and $arr
on lines of their own, then fails to find them. Nothing is glued by dropping it either: whitespace
before a node is the trailing trivia of the token before it unless a newline separates them, and the
lexer puts that newline in the same trailing trivia, so only indentation is lost.

An expr capture is now recorded with get_text_without_trivia rather than get_text. The trivia
around the expression belongs to the call, not to the captured expression: with the rule's own
trivia now emitted too, keeping it would space the expansion by how the call happened to be written

• pair!(1 , 2) expanded to (1 , 2) - and it would double the separator wherever both are
  present.

New goldens in crates/cairo-lang-semantic/src/expr/expansion_test_data/inline_macros, values
cross-checked by compiling and running the equivalent macro_rules! with rustc 1.96.0:

• "Test two placeholders adjacent in the expansion." - the shape above; { _x _y } and E2133
  instead of { _x_y } and E0006. rustc rejects it too, with the two tokens separate.
• "Test a placeholder followed by a keyword in the expansion." - for $i in $arr, two captures.
  The glue used to produce for vin array![..], which no longer parsed; it now compiles clean.
  rustc prints 3 for the equivalent call.
• "Test an expansion block whose group ends every placeholder against a keyword." - the same glue
  inside a $() block, two groups of two captures each, so it has to be avoided per group. rustc
  prints 6 for the equivalent call.
• "Test that the spacing of the call does not reach the expansion." - pair!(1 , 2) expands to
  (1, 2), pinning the capture-side change. rustc prints (1, 2).

Each of the four fails with only this commit's source changes reverted:

Test "Test two placeholders adjacent in the expansion." failed.
Output tag 'expanded_code' does not match:
<{ _x_y}
>{ _x _y }
Output tag 'diagnostics' does not match:
<error[E0006]: Identifier not found.
>error[E2133]: Missing semicolon

Test "Test a placeholder followed by a keyword in the expansion." failed.
`expect_diagnostics` is false, but diagnostics were generated:

Test "Test an expansion block whose group ends every placeholder against a keyword." failed.
`expect_diagnostics` is false, but diagnostics were generated:

Test "Test that the spacing of the call does not reach the expansion." failed.
Output tag 'expanded_code' does not match:
<(1 , 2)
>(1, 2)

The two that fail on the expectation rather than on an output tag report E2117 for the glued vin /
ain / bin, which is the defect; their expanded_code differs too, but the runner stops at the
expectation.

Seven existing goldens in that file are re-blessed, all of them the same whitespace being restored:

• "Test call-site + matched as a literal token between two captures." x+ y -> x + y. This is
  the bug, not incidental churn: the rule writes $a + $b and the space after $a was dropped.
• "Test a non-repeating placeholder broadcast to every group of a repeating one."
  (10+ 1,10+ 2) -> (10 + 1,10 + 2), same rule shape inside a block.
• "Test a repeating placeholder broadcast into a deeper expansion block."
  ((10* 1),..) -> ((10 * 1),..), same, one nesting level deeper.
• "Test a deeper expansion block whose non-final broadcast group has no captures."
  ((20* 3),..) -> ((20 * 3),..), same rule, a group that matched nothing.
• "Test a doubly nested expansion block flattened into one sum." 1+2+3+4+0 -> 1 +2 +3 +4 + 0:
  the rule is $($($x +)*)* 0, so each group contributes $x and the space before its +, and
  the space after the outer block is its trailing trivia.
• "Test a doubly nested expansion block whose non-final group matched nothing." 3+4+0 ->
  3 +4 + 0, the same rule with an empty group.
• "Test a line comment after an expression capture." (1 // c<NL>, 2) -> (1, 2): the comment is
  the trailing trivia of the call's 1, so it is no longer part of what $a captured. This is the
  rustc value, and the golden's own note said so.

One golden in crates/cairo-lang-semantic/src/expr/test_data/inline_macros changes its rule rather
than its output: "Test expr placeholders in nested repetitions with separators (valid)." asserts
that a pattern is accepted, and its expansion $($x)* $($($y)*)* only parsed because the dropped
trivia glued the digits of 1 2 3 4 5 6 into the single literal 123456. Unglued it is six
juxtaposed literals, so the expansion is now 0 $(+ $x)* $($(+ $y)*)*, which keeps the rule's
subject - the pattern - untouched.

[reviewable-StarkWare]

This change is 

[orizi]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• fix(semantic): reject a + expansion repetition over zero groups #10315
• doc(semantic): document the macro expansion error arms as defensive #10314
• fix(semantic): reject a trailing macro repetition separator like rustc #10313
• feat(semantic): match and emit non-comma macro repetition separators #10312
• feat(parser): accept any single-token macro repetition separator #10311
• refactor(syntax): generalize the macro repetition separator node #10310
• fix(semantic): parenthesize an expression capture spliced into an expansion #10309
• fix(semantic): keep the trailing trivia of an expansion's placeholders and blocks #10308 👈 (View in Graphite)
• fix(semantic): capture an expr placeholder by what the parser consumed #10307
• test(semantic): pin the macro matcher behaviors that already agree with rustc #10306
• fix(semantic): reject a pattern that follows an expr placeholder with a token it would swallow #10305
• fix(semantic): reject the malformed macro rule declarations that silently do nothing #10304
• fix(semantic): match a pattern subtree only against a call subtree with the same delimiters #10303
• fix(semantic): reject a macro rule pattern that reuses a placeholder name #10302
• fix(semantic): expand a macro repetition over its own group of captures #10301
• test(semantic): pin the macro repetition operator and trailing separator behaviors #10300
• refactor(semantic): build a nested capture tree alongside the flat macro captures #10299
• fix(semantic): reject a macro expansion block with nothing repeating at its depth #10298
• fix(formatter): stop adding trailing separators in macro call token trees #10354
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[cursor]

PR Summary

Medium Risk
Changes macro expansion text for all user-defined inline macros; behavior is well-tested but affects core semantic expansion and name resolution via code mappings.

Overview
Fixes user-defined inline macro expansion dropping whitespace from rule templates, which welded adjacent expanded tokens (e.g. _x + _y → _x_y, or for $i in → for vin).

Expansion now appends trailing trivia after placeholders and $() repetition blocks via push_trailing_trivia; leading trivia on those nodes stays omitted so CodeMapping offsets still resolve paths to the call site.

:expr captures store text from span_without_trivia instead of full get_text, so call-site spacing/comments are not spliced into the expansion—spacing comes from the rule’s own trivia.

Golden and new expansion tests document rustc divergences (separator-less $() groups can still glue literals) and refresh expected whitespace (x + y, pair!(1 , 2) → (1, 2)). One diagnostic test’s grid! expansion is updated to use explicit + now that digit-gluing no longer masks invalid syntax.

^{Reviewed by Cursor Bugbot for commit f93a19e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[orizi]

@orizi+AGNT made 5 comments and resolved 5 discussions.
Reviewable status: 0 of 3 files reviewed, all discussions resolved (waiting on TomerStarkware).

---

crates/cairo-lang-semantic/src/expr/expansion_test_data/inline_macros line 685 at r3 (raw file):

Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…

this redeclares the exact pair macro from Test expression captures consuming exactly what the parser consumed. above. drop the block and add one line to that test's expr_code:

let call_spacing = pair!(1 , 2);

it already covers the same rule with comment/whitespace variations, and it is already re-blessed here for the capture-side change.

Done - the block is gone and the consuming-exactly test gained the call_spacing line (its comment now covers the spacing claim too).

---

crates/cairo-lang-semantic/src/expr/test_data/inline_macros line 3717 at r3 (raw file):

Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…

the rule was rewritten to dodge glue this PR does not fix. a $() block has no trivia between groups, so $($x)* still fuses them: grid!(1, 2; ...) would emit the literal 12 - same silent value corruption as the _x_y case the PR is about, one level up.

leaving a comment on the rule hides it. add a golden pinning the remaining behavior (e.g. ($($x:expr),*) => { $($x)* } over two captures), or say in the comment that it is a known divergence from rustc, which keeps the groups as separate tokens.

Both taken: added a golden - "Test the groups of a separator-less expansion repetition gluing together." - pinning cat!(1, 2) gluing to the literal 12 as a known divergence from rustc, and the grid comment now states the divergence and points at that pin instead of quietly dodging it.

---

crates/cairo-lang-semantic/src/items/macro_declaration.rs line 907 at r3 (raw file):

Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…

get_text_without_trivia interns the text into the db and you immediately to_string it back - every captured expression in the crate leaves an interned string behind for nothing. take the span off the file content instead:

let expr_node = as_expr_macro_token_tree(input_iter, file_id, db)?.as_syntax_node();
// The trivia around the expression belongs to the call, not to the expression - the
// expansion spaces the value by its own trivia.
text: expr_node.get_text_of_span(db, expr_node.span_without_trivia(db)).to_string(),

Done - the capture takes the span text off the file content directly; nothing is interned.

---

crates/cairo-lang-semantic/src/items/macro_declaration.rs line 1109 at r3 (raw file):

Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…

used once - inline it, like the MacroParam arm does not.

                self.expand_repetition(&ast::MacroRepetition::from_syntax_node(db, node))?;

Done.

---

crates/cairo-lang-semantic/src/items/macro_declaration.rs line 1143 at r3 (raw file):

Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…

span_without_trivia walks for the leading width too, which you discard. there is an accessor for exactly this end.

        let span = TextSpan::new(node.span_end_without_trivia(db), node.span(db).end);

Done.