fix(sierra-to-casm): prevent wide frame offset overflow

[DicksonWu654]

Summary

Prevent valid wide Sierra function parameter and return layouts from panicking during Sierra-to-CASM reference construction. Frame layout now uses wider signed arithmetic, converts each final CASM offset to i16 with a checked conversion, and returns existing structured compilation errors when an offset is not encodable.

---

Type of change

• Bug fix (fixes incorrect behavior)
• New feature
• Performance improvement
• Documentation change with concrete technical impact
• Style, wording, formatting, or typo-only change

---

Why is this change needed?

A valid 32,766-cell function parameter frame reaches offset fp - 32768, but the previous i16 expression computed an out-of-range intermediate before adding one. Function-call return construction and stack-contiguity validation had symmetric intermediate or sentinel underflows. With overflow checks enabled these layouts panic even though every emitted reference offset is representable.

---

What was the behavior or documentation before?

Wide but representable parameter and return layouts could panic while constructing or validating reference expressions.

---

What is the behavior or documentation after?

Frame offsets are accumulated in i64; final dereference offsets are checked when converted to i16. Representable boundary layouts compile normally, while layouts outside the CASM offset range produce structured errors. Regression coverage exercises the exact parameter and function-call return boundaries, including output ordering and full-width stack validation.

---

Related issue or discussion (if any)

None.

---

Additional context

Validation performed on sierra-minor-update:

• ./scripts/rust_fmt.sh
• ./scripts/rust_fmt.sh --check
• RUSTUP_TOOLCHAIN=nightly-2024-08-22 cargo test -p cairo-lang-sierra-to-casm wide_ (2 passed)
• RUSTUP_TOOLCHAIN=nightly-2024-08-22 cargo test -p cairo-lang-sierra-to-casm invocations::function_call::tests (2 passed)
• git diff --check origin/sierra-minor-update...HEAD

[cursor]

PR Summary

Medium Risk
Changes Sierra-to-CASM reference construction for parameters, call returns, and stack validation; incorrect logic could break compilation of wide types, though behavior is covered by new boundary tests.

Overview
Fixes Sierra-to-CASM panics when building reference expressions for wide but encodable function parameters and call return values, where intermediate offset math used i16 and overflowed before the final offsets were written.

Frame walking now accumulates offsets in i64, builds contiguous Deref cell lists through a shared build_deref_reference helper (used for FP parameters and AP call outputs), and only converts each offset to i16 with a checked conversion. check_references_on_stack compares offsets in i64 so large contiguous return layouts still validate. Layouts that cannot be encoded return structured errors (ParameterOffsetOutOfRange, InvocationError::IntegerOverflow) instead of panicking.

Adds boundary tests for wide parameters and wide multi-output returns, including overflow and stack-contiguity cases.

^{Reviewed by Cursor Bugbot for commit dfac89b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[reviewable-StarkWare]

This change is 

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-12T21:25:52.745442Z	817f617	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[orizi]

@orizi made 1 comment.
Reviewable status: 0 of 3 files reviewed, 1 unresolved discussion (waiting on DicksonWu654).

---

a discussion (no related file):
Rather not extend the supported sierra programs without specific reason.
the i16 overflow is a valid failure for using the program.