Repository navigation
A Rust runtime, written to the contract - #18
Merged
Merged
Conversation
docs/rust-plan.md maps the Rust runtime against the 2.0 contract: the packages/ layout, decisions D1-D10, and four phases. Phase 1 is packages/codeupipe-rs (crate codeupipe, 0.1.0, no dependencies), built test-first from conformance/API.md with codeupipe-ts as the reference. A Filter takes &mut Payload (D2) because TS changes the Payload in place and on_error must see what the failing Filter left. Taps, Predicates and Hooks get &Payload, so "only a Filter writes" is a compile error. 46 tests, clippy -D warnings and fmt clean. The pricing example prints the same result as the TS one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pricing example's AuditTap failed on every run, in Python, TS and Rust, to show that a Tap failure is recorded and the run goes on. Now it prints what it reads and State has no errors. The rule is unchanged and the conformance cases and core tests still check it. The compile-ts strict type check had no console, which is why the TS Tap threw instead of logging. The check now declares console, which every JavaScript host has, and nothing else of a browser or of Node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conformance/runners/rust runs the cases against packages/codeupipe-rs, with its own small JSON reader and writer (no download, so it builds offline) and one file per step type, named as in the TS runner. run.py gets the rust:run target and leaves it out when there is no cargo. add_filter and add_valve take the step's type, which gives its label, so the runner hands each Filter over by its own type through a small generic trait and not as a Box<dyn Filter>. Checked that the runner catches a broken runtime: with Tap failures not recorded, 3 cases fail. API.md gets a Rust column, the README a Rust package row, MATRIX.md all eight targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The speed job compared each number with a baseline. A shared CI machine is too noisy for that, so the benchmark job now only measures every target, Rust included, and keeps the numbers in the job summary and as an artifact. It never fails the run. The perf pytest test is removed; bench.py --check stays for a machine with fixed resources. bench.py measures rust:run. Its baseline is kept under its own key, "rust", since it does not depend on the Python version, and --record replaces only the targets it measured. The new rust job runs fmt, clippy, the tests, the example and rust:run conformance, and builds with Rust 1.75, the crate's rust-version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A run spent its time in allocation, not in dispatch. Three changes, with no change to behaviour or API: - Map::insert looks the key up first, so a key that is there makes no new String. - State keeps its storage between runs and writes the labels over. - A key is an Arc<str>, made once and shared by the index and by every copy. A map of 8 keys or fewer is searched in order and has no index; a larger one gets an index with room to grow. The hash stays SipHash, so crafted keys from untrusted JSON cannot slow a map down. 10 filters: 1.10 -> 0.25 us. 50 new keys: 7.26 -> 4.24 us. The dynamic run is now faster than compiled Python in 4 of 5 scenarios, so the plan puts the Rust compiler (phase 3) on hold. Guard tests first: a shorter run after a longer one, maps on both sides of the index size, Send + Sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Payload stays the default: a map that any developer can read, which
shows that anything can go there. A type of your own can take its place:
each primitive takes the data type as a parameter that defaults to
Payload (Filter<P = Payload>, Pipeline<P = Payload>), and the type opts
in with impl PipelineData for Order {}, which asks for Clone and an
optional stamp for lineage. Code written for Payload does not change.
A step for one type in a pipeline of another is a compile error, checked
by a doc test. examples/typed_pricing is the pricing example with an
Order struct and gives the same result. Typed, the pricing run takes
0.024 us against 0.18 us, and 50 writes 0.46 us against 4.3 us.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The contract gets "A type of your own": Payload stays the default, the form that the contract, the conformance cases and the compilers use. A pipeline can carry another type instead; the Pipeline asks only for copy(). Lineage is kept only by a type that has _stamp(label) (stamp in Rust); for any other type it does nothing and the run goes on. Python took such an object already, by duck typing, but crashed with lineage on. It now calls _stamp only when the object has it. TypeScript gets the type parameter that Rust has: Filter, Predicate, Tap, Hook and Pipeline default to Payload, and PipelineData<P> asks for copy() and an optional _stamp. Mixing types does not type-check (@ts-expect-error). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Filter or Tap whose parameter was not called payload (call(self, order)) compiled to a function of data whose body still used order, so the compiled step raised NameError: code that compiled and did not give the output of the dynamic run. The rewrite knew only the name payload. The rewrite now takes the parameter's own name. A Valve's predicate keeps its name too, where before it was renamed to payload, which merged it with any unrelated local called payload. A refusal names the parameter as the step wrote it (order.price, not payload.price), which also covers a step over a type of your own. Tests first: the three output modes against the dynamic run, with parameters called order, p and data and a local called payload, and the refusal of order.price. 538 tests, conformance and 2,000 random pipelines pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The contract on beta now says that a State belongs to one run: run(p, state), a run inside a run, and state on the Pipeline as the State of the run that started last. rust:run reports the three state/ cases as unsupported, not as passed. API.md says so in its notes on Rust, and the plan lists what the change needs in Rust: run(&self), a run with the caller's State, state() behind a RefCell, shared errors, and the cost to the State storage that is kept between runs today. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
conformance/run.py is a codeupipe pipeline since #15, so it imports codeupipe. The TypeScript and Rust jobs did not install the package, and their conformance step failed with ModuleNotFoundError (on #15 as well; CI does not run on pushes to beta). Checked locally with nothing but packages/codeupipe on the path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The contract says that a State belongs to one run, not to the Pipeline. In Rust: - run(&self) in place of run(&mut self), so a step can hold the pipeline it is in (an Rc) and run it again while it runs. - run_with_state(&input, &mut state) for the caller's State, and State::new(). - state() gives an Rc<State>: the State of the run that started last, once it has ended. Each run takes a number when it starts; its State is kept only if no later run has started. A snapshot never changes under a reader, and holding one never blocks a run. - A State's errors are Arc<dyn Error> (SharedError), so a State can be cloned: with the caller's State, the Pipeline keeps a copy. Two States take turns, so a run still makes no new label String; the Rc of a run costs about 0.03 us (10 filters: 0.25 -> 0.28 us). The runner plays run_again as the TypeScript runner does and reports the caller's State, pipeline_state and inner_states. Tests first (state_per_run.rs); breaking the "started last" rule fails one case and one unit test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JoshuaWink
added a commit
that referenced
this pull request
Oct 29, 2026
* feat(go): codeupipe for Go, the eight primitives packages/codeupipe-go is the Go runtime, written to conformance/API.md with no dependencies. A Filter, Predicate, Tap and Hook take a context.Context first and the data second. Payload is the default; each primitive also has a generic form (FilterOf[P]) for a type of your own, and the short name is the same type. What the contract refuses is a compile error: a function as a step, a Tap that is a Filter, AddValve with its arguments in the wrong order, a step for another type. refused_test.go builds each shape and checks the error. Tests are ported from packages/codeupipe-ts, with the ones that only Go can express: ctx reaches each step, 50 runs in goroutines keep their own State, and a nil step is refused when it is added. go test -race, go vet, gofmt and staticcheck are clean on Go 1.22.12 and 1.25.0. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> * feat(go): the Go conformance runner, bench and CI go:run passes all 30 conformance cases, with nothing unsupported, and every other target still passes in the same run of conformance/run.py. - conformance/runners/go: the runner, with one file for each step type in cup_ops/, named as the TypeScript and Rust runners name them. It reads and writes JSON with encoding/json, and runs with GOPROXY=off. - run.py and bench.py: the go:run target. Go is left out of a run on a machine with no go. The Go baseline is kept under its own key. - CI: a go job for gofmt, vet, staticcheck, go test -race, the example, go:run, and the tests again on Go 1.22. - conformance/API.md: a Go column and notes. Also fixes a sentence that was cut in the Rust notes (since #18). - MATRIX.md and BENCH.md regenerated from these runs. - docs/go-plan.md: the plan, with its decisions and the two phases. Checked by breaking the runtime on purpose: not recording a Tap failure fails 4 cases; keeping the State of a run that did not start last fails 1; not sending OnError for the pipeline fails 2. Benchmark, Apple Silicon: a 10-step run takes 0.55 µs for Go, 0.28 for Rust, 4.8 for python:run. Sizing the executed list once in State took 10 no-op steps from 356 to 217 ns. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The third runtime,
packages/codeupipe-rs(cratecodeupipe, 0.1.0, no dependencies), built fromconformance/API.mdwith tests first and the TypeScript package as the reference. The plan and its decisions are indocs/rust-plan.md.Rust runtime
add_valvetakes the predicate first, and a Tap, Hook or Predicate gets&Payload, so only a Filter writes. Doc tests check each of these.conformance/runners/rust/, with its own JSON reader and writer so it builds offline.run.pyhasrust:run, left out when the machine has nocargo(CargoProbe, besideNodeProbe).state/cases from fix: a State belongs to one run, not to the Pipeline #13. A State belongs to one run:run(&self), so a step can hold its own pipeline (anRc) and run it again;run_with_state(&input, &mut state)for the caller's State;state()is anRc<State>snapshot of the run that started last; a State's errors areArc<dyn Error>so a State can be cloned. Breaking the "started last" rule on purpose fails one case and one unit test.rust:runis 20× faster thanpython:runand faster than compiled Python in 4 of the 5 scenarios, so the Rust compiler (phase 3) is on hold.rustjob: fmt, clippy, tests, both examples,rust:run, and a build with Rust 1.75 (the crate'srust-version, not checked on a machine before this).For all three runtimes
API.md). Payload stays the default. A pipeline can carry a class or struct instead, and the Pipeline asks only forcopy(). Lineage is kept only by a type with_stamp(label)(stampin Rust); for any other type it does nothing. Python already took such an object but crashed with lineage on. TypeScript gets the type parameter that Rust has (Filter<Order>,new Pipeline<Order>()), with@ts-expect-errorchecks that mixing types does not type-check.examples/typed_pricingin Rust gives the same result as the pricing example.payload(call(self, order)) compiled to code that raisedNameError. The rewrite now follows the parameter's own name, a predicate's parameter is no longer merged with a local calledpayload, and a refusal quotes the name as written (order.price).bench.py --checkstays for a machine with fixed resources, and the Rust baseline has its own key.For the reviewer
betaafter fix: a State belongs to one run, not to the Pipeline #13, refactor: the Python code passes its own linter in strict mode #14 and refactor(conformance): the judge is a codeupipe pipeline #15. It also fixes CI for the TypeScript and Rust jobs:conformance/run.pyimportscodeupipesince refactor(conformance): the judge is a codeupipe pipeline #15, and those jobs did not install it (the TypeScript job failed on refactor(conformance): the judge is a codeupipe pipeline #15 as well). The three new helpers inbench.pycarrycup: allow-plain, soPLAIN_BY_DESIGNgoes from 60 to 63.npm pack; 75 Rust tests, clippy, fmt, both examples; conformance on all 8 targets, every one with nothing unsupported;scorecard.differential --count 2000with 0 failures;bench.py.rust-version) runs only in CI; only 1.89 is installed here. It passed on the run before the State change.🤖 Generated with Claude Code