Annotate jawohl itself: no surface crate, and Stream crosses as a handle - #12
Open
zmaril wants to merge 5 commits into
Open
Annotate jawohl itself: no surface crate, and Stream crosses as a handle#12zmaril wants to merge 5 commits into
zmaril wants to merge 5 commits into
Conversation
What jedem looks like applied to jawohl. bindings/surface/src/lib.rs is the only
hand-written file; the Python extension module and the Node addon are both
generated from it, and both are exercised by a real host process.
python: jawohl.is_irrecoverable(schema, '{"role":"sup') -> True
node: jawohl.isIrrecoverable(schema, '{"role":"sup') -> true
Each language gets what it would have written: snake_case in Python and
camelCase in JS from the same Rust names, Option<T> arriving as None and null,
a Result raising a ValueError and throwing an Error, and a synchronous function
staying synchronous rather than returning a promise.
It is a separate workspace on purpose. The published jawohl crate gains no
dependency from any of this -- still regex-automata and nothing else -- and
bindings/** is excluded from the package.
The honest limit, stated in the README rather than glossed: this is the
BATCH-SHAPED SUBSET. jedem v1 exposes functions over plain values, and jawohl's
real API is a stateful Stream, which is a handle. So every function here takes
everything received so far and answers one question about it, which means every
call re-parses from the beginning. For a tool call of a few hundred bytes that
costs nothing, and the most valuable question -- can this generation still
succeed? -- answers fine that way. For a long document it is quadratic. The
incremental Stream API reaches these languages when jedem grows handles, which
is exactly the argument for jedem steps 3 through 5.
A drift guard in the surface crate regenerates in memory and diffs against the
committed bindings, so a surface change nobody regenerated fails the build; it
also asserts jedem::Target::ALL is fully covered, so a new backend cannot be
added without a guard. Verified by mutation: renaming a function fails the test
with the regeneration command.
The README's "Not yet" section is replaced. It promised JavaScript and Python
wrappers in 2023 and never delivered, so this says plainly what exists, where,
and what does not.
One test caught me: in Python r'...\\' is TWO backslashes -- a complete escape,
not a dangling one -- so the completion correctly kept it. The test now asserts
both cases.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HsDxLrGdx6nPaXkVEWkNvS
Rewrites jawohl's bindings on the five jedem changes that landed, which existed because of what this PR cost the first time. The binding crates are generated in full now, not just their source: manifests, module shims and node's build.rs all come from jedem. The two run scripts are gone in favour of `cargo jedem run --python test.py`. The sixteen-line generator main() is a ten-line file that is mostly the macro invocation. Hand-written scaffolding drops from 323 lines to 26 across three files, and the surface -- the only thing anyone actually writes -- is the rest. Three things got better rather than merely shorter. Statuses and validations cross as REAL ENUMS. Python receives Syntax.Complete and Validation.IrrecoverablyInvalid as members of a class; TypeScript receives "Complete" and "IrrecoverablyInvalid" as a string-literal union. Both reject a value that is not a variant. Previously a describe() helper flattened them to strings and Python got 'irrecoverably_invalid' with nothing checking it. The enums are mirrors of jawohl's own, because #[derive(jedem::Enum)] has to be applied where a type is defined and jawohl takes no dependency on jedem; the cost is one From impl each. The six .map_err(|e| e.to_string()) calls are gone. Functions that can fail two ways return Box<dyn Error> directly, which always worked -- jedem never inspected the error type. The marker `pub struct Jawohl;` is gone; the surface is a module. Two rough edges found by using it, both recorded rather than papered over. jedem keys a generated manifest's dependency on the RUST CRATE name, so a package named jawohl-surface produced a manifest cargo could not resolve. The package is now jawohl_surface so the two spellings match. jedem's own demo never hit this because `hello` has no hyphen; worth fixing there. Generated code must not be formatted. rustfmt rewraps method chains exceeding its chain_width, which jedem cannot predict, so `cargo fmt --all` made the committed bindings differ from a fresh generation and the drift guard failed. Formatting generated code is the wrong operation on it -- you regenerate it. A rustfmt.toml records the rule, and only the hand-written crate is formatted. Also fixed: test.mjs still required ./.nodeimport/jawohl.node, a stale addon from before `cargo jedem run` existed, which silently shadowed the real one and produced confusing failures against an old surface. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HsDxLrGdx6nPaXkVEWkNvS
zmaril
force-pushed
the
demo/jedem-bindings
branch
2 times, most recently
from
August 19, 2026 19:15
8a4aec8 to
87345a0
Compare
The bindings were described by a separate crate that restated jawohl's API: two enums mirrored variant for variant with `From` impls both ways, and eight functions wrapping `Stream`. Every one of them was a place the description could drift from the thing described, and the compiler could not catch it because nothing connected them. Now the annotations live on the definitions. `Syntax` and `Validation` derive `jedem::Enum` where they are declared, `complete_json` and `get_closing_string_for_partial_json` carry `#[jedem::export]`, and the `Stream` impl carries it once for the whole block. `bindings/surface` is gone -- there is no hand-written code under `bindings/` at all. `Stream` crosses as a handle, so Python and JavaScript now get the incremental parser rather than the batch shim. That shim re-parsed from byte zero on every call, which is quadratic over a document; the point of the library is that it does not have to be. The host tests are rewritten around it: push chunk by chunk, watch a value settle, and cancel on the chunk that decided the schema was unsatisfiable rather than at end of document. Six methods are marked `#[jedem(skip)]`, each with its reason in place. `with_number_profile` and `with_max_depth` consume `self`, which cannot mean anything once another language owns the handle. `snapshot`, `changes`, `error` and `lowering_report` return unions or records, which jedem cannot lower yet -- `changes` is the one to want back, since it is how events would be consumed instead of polling `status`. Two consequences worth being plain about: jawohl now depends on jedem. Nothing of it survives to runtime -- the descriptor is `&'static` data and the generator is reached only from the `#[cfg(test)]` drift guard -- but it is a real dependency on a crate that is not published, so `cargo package` cannot resolve it and jawohl 0.2.0 cannot ship until jedem 0.1 is on crates.io. The packaging job is marked `continue-on-error` with that written next to it, so the failure stays visible without turning unrelated pull requests red. The drift guard moved into jawohl's own test suite, because the surface is now jawohl's annotations. `cargo test` checks the committed bindings; `JEDEM_WRITE=1 cargo test` rewrites them.
zmaril
force-pushed
the
demo/jedem-bindings
branch
from
August 19, 2026 19:17
87345a0 to
f82452a
Compare
jedem uses `#[diagnostic::on_unimplemented]`, stabilised in 1.78, and its proc-macro chain (proc-macro2, quote, syn, unicode-ident) floors at 1.71. `rust-version` is a promise to consumers, so it moves rather than the dependency being pinned or patched around; 1.78 is over two years old, which is a reasonable floor to ask for. Verified by building against a real 1.78.0 toolchain, not inferred. jedem now declares the same MSRV in its own manifest, so the next crate to depend on it reads the floor instead of discovering it here. The MSRV job's comment no longer singles out serde_json: dev-dependencies are excluded on principle, not because of one crate.
zmaril
force-pushed
the
demo/jedem-bindings
branch
from
August 19, 2026 19:24
bb504db to
8bdb7ed
Compare
jedem lowers Rust's move-builder now, so `with_number_profile` and `with_max_depth` needed nothing done to them: the `#[jedem(skip)]` markers come off and the methods stay exactly as they were. Python and JavaScript get real chaining -- `Stream().with_max_depth(64)` -- and the returned object is the same handle, not a copy. `NumberProfile` derives `jedem::Enum`, which is what the builder taking it needs. The compile error that asked for it named the missing derive outright, which is what `#[diagnostic::on_unimplemented]` is there for. Four skips remain, all of them return types with no lowering yet: `snapshot` and `changes` need unions, `error` and `lowering_report` need records. Both host tests now drive the builders, including the case that only makes sense with one: `with_number_profile(Exact)` defers a numeric bound that `PlainDecimal` decides immediately.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the hand-written surface crate with annotations on jawohl's own
definitions, and upgrades the bindings from a batch shim to the real
incremental parser.
What changed
bindings/surfaceis deleted. It was a crate whose whole job was to restatejawohl's API: two enums mirrored variant for variant with
Fromimpls in bothdirections, plus eight functions wrapping
Stream. Nothing connected it to thething it described, so nothing could catch it drifting.
The annotations now sit on the definitions:
SyntaxandValidationderivejedem::Enumwhere they are declaredcomplete_jsonandget_closing_string_for_partial_jsoncarry#[jedem::export]Streamimpl carries it once, for the whole blockThere is no hand-written code under
bindings/any more — only this README andthe two host tests.
Stream is a handle now
Python and JavaScript get the incremental parser itself. The old shim re-parsed
from byte zero on every call, which is quadratic over a document — the exact
thing the library exists to avoid.
The host tests are rewritten around it, and now assert something the batch API
could not express: cancellation lands on the chunk that decided the schema was
unsatisfiable, not at end of document.
What is still not across
The two Rust move-builders cross unchanged too —
with_number_profileandwith_max_depthtakeselfand returnSelf, which names the same object, sojedem mutates in place and hands the same handle back.
Stream().with_max_depth(64)chains in Python and JavaScript exactly as in Rust, with no annotation on either
method.
Four methods carry
#[jedem(skip)], each with its reason in place:snapshot,changesValueis recursive,Eventhas struct variantserror,lowering_reportchangesis the one to want back: it is how events would be consumed instead ofpolling
status.Two consequences worth stating plainly
jawohl now depends on jedem. Nothing survives to runtime — the descriptor is
&'staticdata, and the generator is reached only from the#[cfg(test)]driftguard — but it is a real dependency on an unpublished crate.
cargo packagecannot resolve it, so jawohl 0.2.0 cannot ship until jedem 0.1 is on
crates.io. The packaging job is marked
continue-on-errorwith that writtenbeside it, so the failure stays visible without turning unrelated PRs red.
The MSRV rises from 1.70 to 1.78. jedem uses
#[diagnostic::on_unimplemented], stabilised in 1.78, and its proc-macro chainfloors at 1.71. Declared rather than patched around, and verified against a real
1.78.0 toolchain. jedem now declares the same floor in its own manifest, so the
next crate to depend on it does not have to discover this the way jawohl did.
The dependency currently points at a jedem branch (
feat/handles), becausethe handle and
bindings:support it needs is in jedem PRs #11 and #12. Thosemerge first; then the
branchkey comes off.The drift guard moved into jawohl's own test suite, since the surface is now
jawohl's annotations.
cargo testchecks the committed bindings,JEDEM_WRITE=1 cargo testrewrites them.Verified
cargo test --all-targets,cargo test --doc, clippy and fmt all clean