Skip to content

fix(translate): restrict float literal syntax - #129

Merged
tetsuo-cpp merged 2 commits into
canonfrom
fix/strict-float-lexing
Oct 24, 2026
Merged

tetsuo-cpp merged 2 commits into
canonfrom
fix/strict-float-lexing

Conversation

@tetsuo-cpp

Copy link
Copy Markdown
Owner

ForthLexer::isFloat called llvm::to_float, which wraps strtod. strtod also accepts hex (0x10, 0x1p3), nan, inf and infinity. Those tokens became f64 literals. So 0x10 silently stored the bit pattern of 16.0, and : nan 7 ; could not define a word.

Change

isFloat now checks an explicit grammar: [+-]? (D+ '.' D* | '.' D+ | D+) ([eE] [+-]? D+)?. A token must have a . or an exponent to count as a float. Everything AGENTS.md documents still lexes as a float (3.14, -2.0, 1.0e-5, 1e3). .5, 5. and +1.5 also still lex as floats, because strtod accepted them before. Anything else is an ordinary word. The diff stays inside the lexer.

This PR does not add hex integer literals. 0x10 is now an unknown word unless the program defines it.

Repro

\! kernel k
\! param a i64[1]
0x10 a !

Before (warpforthc):

	st.global.b32 	[%rd2+4], 1076887552;
	st.global.b32 	[%rd2], 0;

After:

hex.forth:3:1: error: unknown word: 0X10
error: failed to parse Forth source
\! kernel k
: nan 7 ;
nan

Before:

nan.forth:2:3: error: expected word name after ':'
error: failed to parse Forth source

After, the kernel compiles and the PTX has .visible .func ... NAN(.

Tests

  • float-literal-strtod-spellings.forth defines and calls nan, inf, infinity and 0x1p3 as words. It fails before the fix.
  • hex-literal-error.forth checks that 0x10 reports unknown word: 0X10. It fails before the fix.
  • float-literals.forth now also covers 1E3, .5, -.5, 5., +1.5 and 2.5e+2.

The tests are committed before the fix. check-warpforth passes all 119 tests locally.

Fixes #112

@tetsuo-cpp
tetsuo-cpp merged commit 52d4378 into canon Oct 24, 2026
1 check passed
@tetsuo-cpp
tetsuo-cpp deleted the fix/strict-float-lexing branch October 24, 2026 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hex literals, nan and inf are silently lexed as float literals

1 participant