Repository navigation
support for p3 components - #22
Open
chenyan2002 wants to merge 21 commits into
Open
chenyan2002 wants to merge 21 commits into
chenyan2002 wants to merge 21 commits into
Conversation
…mode
- Codegen reads p3 bindings: skip wit-bindgen's wit_stream/wit_future
modules, keep wit_bindgen:: paths, and track async functions and the
streams and futures in each signature.
- The start function of replay, fuzz and dialog is async, so it can
await async exports.
- proxy:executor (assets/executor.{wit,rs}) runs the virtualized host's
side of streams and futures, which sync mocks cannot wait for. The
imports component exports it, and start runs it along with its own.
- Fuzz uses a local Fuzz trait instead of Arbitrary, so that streams and
futures can implement it. The host writes fuzzed chunks to the streams
it returns, and reads the guest's to the end, partway, or not at all.
Async imports return at once or after yielding, and start yields
between export calls, so the schedule is fuzzed too.
- Fuzz and dialog mocks are labeled by the handles wasmtime gives them,
set in a _resource_new override, and labeled alike in both generated
components through get-handle-mock-* conversion functions.
- Recorder interface: take-host-line(key), keyed by a task or a label, and finish. next-host-line now returns only what start performs, export calls and the host's drops of the guest's resources, and skips guest lines that may still come. Both recorders implement the new functions. - Executor: wait_until_step, with an async wait-until-step export for start. It panics if replay passes the step, which means replay diverged. - Replay codegen: - Async import mocks: a call recorded RETURNED returns at once. A STARTED one takes its Return and waits for that line's step. - start: each line it performs waits for the step before its own, takes its step and checks the number. call_export checks the guest's Started. - Arguments: borrowed arguments are dropped as the host's drops. - finish runs at the end. - Mock labels: replay now labels mocks by their trace labels (mock_labels, get-handle-mock-*).
…sion - Replay of streams, futures and error-contexts: a local `Replay` trait (src/traits/replay.rs) replays the host's values from the trace's WAVE text and consumes the guest's. The host's ends run as executor jobs that perform each trace line in its step and check what the guest wrote. Host-created streams and futures are paired with the trace's label through the new `pair` recorder function. - host_stream/host_future, shared by fuzz, dialog and replay, label a stream or future by the writer the host keeps. Only fuzz and dialog print the reader under that label, through wasm-wave's `set_handle_label`. - The host's drops of mocks, e.g. of the borrowed arguments of an export call, are marked through a new `host-drops` executor export, so they are recorded as the host's drops, without a step. - Generated components send their panics to the recorder's `fail`, as they have no stderr of their own. - Executor: `arrive-after` waits for a step and takes the next one in a single call. The executor is held in between, so that no run, e.g. for a guest's drop, can fall between the wait and the arrival. Replay would otherwise wait for that run by waking itself, which changes the order in which wasmtime delivers the guest's events. - `start` calls only the main component's exports, not the host's own functions. Those would put both ends of a stream in the generated components, where the imports executor can't take them over, as wasmtime traps intra-component copies of non-numeric payloads. - Labels: the recorder names each binding of a handle number apart as it records (trace::Labels), e.g. #stream-7, then #stream-7-2 after its end. This only looks at earlier lines, so a trace cut short by a trap keeps the same names. Replay's renaming drops generations and the `handed` map. A stream replay makes is paired with the binding after the line that hands out its reader. Replay rejects a trace that reuses a label after its end. - make test-fuzz-replay fuzzes and replays the p3 fixture as well as the p2 one, through a shared run-fuzz-replay target. - docs/p3-record-replay.md: phase 3 done, with label bindings, arrive-after, and outdated and historical text removed.
- Fuzz: an import's result is drawn from a hash of its arguments and a seed taken once per run (fuzz_data, splitmix64), rather than from the argument text itself. The same call gets the same result within a run, and another in the next run. - Executor: every wait for a step holds the executor from the run in which it ends until the waiting task resumes, or is dropped, e.g. a cancelled import's mock. Otherwise a run, e.g. for a guest's drop, could come in between, and replay, which knows only the step, would place it differently. This caused STARTED returns one step late. `arrive-after`, which held only for arrivals, is now `step-after`: a wait plus a step, taken in the imports component's task before the hold is released. - Labels: a Drop or Value of a label no line has mentioned yet ends the binding of the line that brings it in, which comes later. The host records a copy once its job sees it complete, so the guest may already have taken a mock from a future's value and dropped it. - The bundled recorder prints lines as text with PROXY_TRACE=text, for reading a fuzz run. JSON stays the default. Trace::parse panics on text without trace lines, rather than replaying an empty trace. - tests/run-wasm.sh tells a trap of the main component, e.g. a guest's panic on a fuzzed error, from a failure of the proxy. run-fuzz accepts the former, and run-fuzz-replay requires each replay to end as its fuzz run did.
- Imports component: an async import polls the host's call once. A ready result is returned at once (RETURNED). Otherwise the proxy records Started, and an executor job awaits the host's subtask. The job records the Return in the step of the run it completed in, and hands the result back. The export waits for it as replay's mock waits for its step, holding the executor until it resumes (host_result). - Executor: wait_for_job runs a future as a job and returns its value with its step; wait_for_run is built on it. The job may borrow the waiting task's arguments, e.g. a method's self. It is only polled while the task waits, and dropped with it if the guest cancels, which cancels the host's call too. - Exports component: every export call takes a step for its arrival. Record mode has no start, so async export calls run the executor while they are pending, and stop it when the last one returns; a call arriving meanwhile keeps it running (call_export_recorded, record mode's `run`). call_export and trace_guest move out of start, to be shared with record mode. - Every mode's worlds now have proxy:conversion/executor, record mode included.
- Record mode's pumps: converting a stream or future the proxy receives makes a pair of the other side's type, hands on its reader, and spawns a job between the received end and the new writer. The job records each copy (Write of the host's items, Read of the guest's), each side's Drop, and a future's Value, with their steps. A zero-length write makes it read from the sender only once the receiver reads, without read-ahead; it reads up to 4096 items at a time. SENDER says who sent the value: the caller for arguments, the callee for the result. - Labels: a pumped stream or future is labeled by the end the proxy received. Arguments are printed after conversion, so the reader handed on there is registered under that label with set_handle_label, and the items the caller sends are printed converted. - Executor: the executor's `run` callback wraps wit-bindgen's (executor_callback). instrument patches the generated bindings so that the executor's __callback_run calls it, and asserts the glue's shape. After a yield, the first poll is a probe that only wakes the executor, so wit-bindgen takes an event that came meanwhile before the real run, as when an event wakes a waiting executor during recording. Without it, a guest's write during a self-woken yield landed one step late in replay, breaking recordings whose streams carry several chunks. - tests/fixtures/p3-record: a guest whose async imports, streams and futures wasmtime implements itself. `run` makes two concurrent wait-for calls; `cat` copies stdin to stderr through WASI p3 streams. make test-record-p3 records both under `wasmtime run`, with delays for RETURNED and STARTED, and stdin in one chunk, several, or none, and replays each trace. - Fixtures: each WIT now sits next to the crates built from it: tests/fixtures/p2 (was proxy and proxy-run), tests/fixtures/p3 (was proxy-p3 and proxy-p3-run), and tests/fixtures/p3-record. - docs/p3-record-replay.md: pumps, their labels and readiness, the executor's callback, and phase 4's status.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.