Repository navigation
feat(rust): codeupipe-compile, a pipeline written as plain Rust - #26
Merged
Merged
Conversation
packages/codeupipe-compile-rs: codeupipe-compile reads a Rust pipeline and its steps with syn, and writes them as a module of plain Rust: a struct with one Option field for each key that the steps use, one function for each step, and a run function. The body of each step is its own code, with each key of the Payload as a field and each setting as its literal. It does not run the code. The rule: code that compiles gives the same output as the pipeline. A pipeline for which the compiler cannot keep that rule is refused, with the place and the fix: a Hook, a nested pipeline, a setting that is not a literal, a key that is not text in the code, a key with two types, a read with no type, another method of the Payload, and others. The compiler is a codeupipe pipeline of five Filters. tests/same_output.rs compiles the expected modules into the test and compares them with the pipelines on many inputs and errors. On an M1, pricing (3 steps of arithmetic) runs 150x faster than Pipeline::run, signup (4 steps of text) 3.1x. A Cargo workspace of its own, as the linter. A new CI job runs fmt, clippy, the tests, the bench and Rust 1.75; the linter job also checks its code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a key is text, the compiled steps work on a second struct that borrows the caller's text (SignupWork, a Cow for each text field). A step that writes text owns it, and run copies into the output only the caller's text that no step wrote. A String setting is made once, by a function of its own with a OnceLock, and not at each run. The signature of run does not change. A pipeline with no text key is compiled as before. The compiler gets a sixth Filter for this, BorrowTextFilter, after the types of the keys are known. Signup: 0.164 to 0.086 µs, 5.9x faster than Pipeline::run (it was 3.1x). A new greet fixture gives back the caller's note when its Valve does not run, and tests/allocations.rs checks that a run allocates only for the text that its steps make and the caller's text that it gives back. --keep, to compute only the keys that the caller keeps, waits in #25. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
packages/codeupipe-compile-rs:codeupipe-compilereads a Rust pipeline and its steps withsyn, and writes them as one module of plain Rust with no codeupipe:Optionfield for each key that the steps use;pub fn run(input: &XData) -> Result<XData, StepError>.The same rule as the other compilers: code that compiles gives the same output as the pipeline. A pipeline for which the compiler cannot keep that rule is refused, with the place and the fix. The refusals cover a Hook, a nested pipeline, a setting that is not a literal, a key that is not text in the code, a key with two types, a read with no type, and another method of the Payload. The compiler is itself a codeupipe pipeline of six Filters.
Two commits
docs/rust-plan.md, "Phase 3").Cowfor each text field). A step that writes text owns it, andruncopies into the output only the text that no step wrote. AStringsetting is made once (aOnceLock), not at each run. The signature ofrundoes not change. This was written by hand and measured before it was built.How fast
cargo run --release --example bench, Apple M1, best of 9 × 20,000:Pipeline::run--keep, which computes only the keys that the caller keeps, waits for a real pipeline that needs it: #25 has the measurements and the rules.How it is checked
tests/same_output.rs: the expected modules are compiled into the test and give the output and the errors of their pipelines on many inputs, including the caller's own value of a key that a Valve may write.tests/allocations.rs: a run allocates only for the text that its steps make, and for the caller's text that it gives back.tests/compile.rs: the compiler writes those modules, and refuses each shape with its place.rust-compilejob runs fmt, clippy, the tests, the bench, and Rust 1.75. Therust-lintjob also lints the compiler's code.Local: 19 tests pass on stable and on 1.75, clippy and fmt are clean, and the linter with
--stricthas no findings. The runtime's own pricing example is refused ataudit_tap.rs:9:33, because its Tap printspayload.to_dict().Not yet
A
rust:staticconformance target, nested pipelines,from_map/to_map, and--keep(#25).🤖 Generated with Claude Code