Repository navigation
Replace PHP Server with a compatible Rust runtime and safe in-place upgrade #325
Description
Objective
Replace the PHP/Apache Durable Workflow Server with a faster Rust successor in
this repository. Existing published SDKs, CLI, Waterline and sample applications
must work unchanged. Operators experience a major version upgrade of the same
product, with the same API, behavior, data and operational surfaces.
The PHP Server remains working as a reference throughout development. The
embedded Laravel Workflow package remains a first-class product. This is accepted
implementation work assigned to a dedicated maintainer, not a new architecture
comparison or a claim that Rust is already faster.
Constraints
Repository and delivery
- No new repositories. Rust lives in
durable-workflow/server. - Land small, coherent PRs on
mainfrequently. Do not accumulate the rewrite
on a long-lived branch or break the PHP Server while landing foundations. - Keep a running, contributor-readable log in this repository with the current
plan, significant design decisions, verified outcomes, filed issues, the
fixture cases they block and the next concrete action. - Choose the architecture and established Rust libraries with engineering
judgment. Explain significant choices briefly and link the proving fixtures. - Follow ordinary review, CI, release and branch-cleanup rules.
Contract, parity and embedded mode
- Published specifications and shared fixtures define observable behavior.
Neither implementation wins by default. When they disagree, settle the
contract first, then correct whichever implementation is wrong. - Establish a shared fixture corpus, recorder and differential runner that can
target a server URL. Reuse existing protocol specifications, conformance
scenarios and benchmark tooling rather than creating a second software factory. - Run the shared fixtures against embedded Laravel, PHP Server and Rust Server.
Workflow package changes also remain tested against a real HTTP Server. - Default to parity for existing and new observable capabilities. Define the
contract and fixtures first, implement a feature in both servers (including
necessary Workflow package changes), and pass the same fixtures on both.
A simple, slower PHP implementation is acceptable. - Internal scheduling, sharding and pooling mechanics without an observable
contract need no PHP counterpart. - If an observable capability cannot reasonably exist in PHP, advertise an
explicit capability flag, mark its fixtures Rust-only and record why.
Do not disguise incomplete parity as an internal optimization. - Preserve the existing Avro payload protocol, errors, authentication and
authorization, namespaces, durable identities and history, worker leases and
fencing, retries, cancellation, timers, schedules, visibility and recovery.
Execute the required PHP/Python/Rust worker directions, rather than assuming
an HTTP schema comparison proves them.
Database isolation and one-way upgrade
- Support SQLite, MariaDB/MySQL and PostgreSQL using the current database tables
and stored data. Rust owns its versioned forward migrations. - Ordinary development and differential runs use separate isolated databases
for PHP, Rust and embedded targets. Never run different implementations
concurrently against one database. - The only cross-implementation shared-database exception is the dedicated
sequential upgrade test: PHP writes and stops, then Rust takes over that
same database. Multi-node replicas are tested within their own isolated
deployment, not against another implementation's state. - Upgrade existing deployments in place. HTTP export/import remains a way to
move data between deployments, not the upgrade route. - Require a backup before mutation and stop the PHP fleet before takeover.
Make interruption resume safely or fail without corrupting state. - Record a schema marker. All affected PHP startup/worker/bootstrap paths must
refuse an upgraded database before migration or durable writes can occur. - There is no PHP rollback after upgrade and no mixed PHP/Rust fleet.
Rust migrations need not preserve post-upgrade PHP schema compatibility.
The PHP reference continues to work against its own non-upgraded database. - Inventory stored PHP-serialized values, class names and framework conventions.
Handle each with a read-compatibility path or documented one-time conversion.
Do not silently discard data that Rust cannot interpret. - Preserve acknowledged work, durable history, pending workflows, timers,
leases, credentials and external payload references/bytes across takeover.
Operations and performance
- Preserve existing image names, supported image architectures,
DW_*
configuration, health/readiness, bootstrap, Compose and Helm layouts, metrics,
diagnostics, graceful shutdown and recovery. - Document a migration path for anything that cannot carry over directly,
such as a PHP authentication-provider class. - Record a reproducible PHP baseline before judging Rust. Use exact published
artifacts, workloads, hardware, resource limits, commands and raw results. - Rust must meet or beat PHP throughput, memory and idle long-poll cost.
Verify timers and lease renewals remain healthy during CPU-heavy load.
Do not accept apparent gains from lost work, backlog, swapping or an
unmeasured drain. Repeat promising measurements and recheck baseline drift. - Do not change customer pricing, Cloud production or managed rollout as part
of the Server rewrite. Exact Rust artifacts need their own Cloud qualification.
Work order
- Establish the safety net: current contract inventory, shared fixtures,
recorder, URL-targeted runner, meaningful embedded/HTTP coverage and
reproducible PHP baseline. - Land the Rust foundations and vertical slices in small reviewed merges,
keeping PHP operational. - Run the differential suite after every change. Record intentional contract
decisions and convert every defect into a focused shared regression. - Complete all capabilities and actual SDK/worker directions, database and
multi-node qualification, performance proof and operational compatibility. - Finish migration tooling, backup-first upgrade tests, operator documentation
and major-version cutover preparation. Release through the repository's
existing decision and publication process, then verify published artifacts.
Acceptance
- Shared fixtures, recorder and URL-targeted differential runner are usable
by a contributor from the public repositories. - PHP Server stays operational and embedded Laravel remains first class.
Shared contracts pass on embedded, PHP Server and Rust Server. - Every required fixture passes with actual published PHP/Python/Rust SDKs
and every meaningful supported workflow/activity/child direction executed.
Existing CLI, Waterline and sample apps work unchanged. - Multi-node kill/restart tests prove no lost durable work, no duplicate
committed outcomes and no acceptance of stale worker attempts. External
effects retain their explicit application-idempotency contract. - Conformance and multi-node suites pass on SQLite, MariaDB/MySQL and
PostgreSQL. Database topology and limits are explicit. - Rust meets or beats the recorded PHP baseline for throughput, memory and
idle long polls. Timers and lease renewals remain healthy under CPU-heavy load. - Existing images/configuration/bootstrap/Compose/Helm/health/readiness/
shutdown/diagnostics remain usable, with documented migrations for exceptions. - A backup-first in-place upgrade starts from data written by the last PHP
release, with pending workflows, timers and leases, and continues under Rust
on each supported database. - Interrupted upgrade resumes or fails safely. PHP reliably refuses the
upgraded schema. No concurrent mixed fleet or post-upgrade PHP rollback. - Stored PHP-specific values and external payloads have tested compatible
reads or explicit one-time conversions. - Operator upgrade/cutover documentation, major-release readiness,
exact publication identities and downstream verification are complete. - The running repository log lets a maintainer see decisions, blockers,
blocked fixture cases and the next action without private workspace context.
Ownership
A dedicated Rust Server maintainer owns this issue and its scoped implementation,
tests and cutover preparation. Coordinate necessary cross-repository contract
and Workflow changes through linked owning records. Other maintainers should
avoid competing implementation of this rewrite while that assignment is active.
Activity
rmcdaniel commentedon Oct 9, 2026
The first safety-net slice is in #326, with the public running log, capability inventory and frozen published tuple. The default PHP artifact remains unchanged; no Rust runtime or performance improvement is claimed.
At source
8182ad8b5df8a8266f52e0a368fb3abea53a19e9, three reviewed cases run through actual PHP SDK workers and real embedded Laravel database queue jobs on independent SQLite files: typed echo, an exact int64 beyond JSON number precision, and one external activity. The recorder checks completed work, the complete expected event inventory, history pagination, deadline budgets, registered types, decoded values/types and start/run/activity/attempt relationships. The differential comparison anchors to current fixture hashes and rechecks observations; matching partial recordings and saved pass labels cannot qualify a run. There are 25 focused comparator checks.Hosted fixture qualification passed and retains both records and comparison for 90 days. Repository contracts and boundary checks passed; the full PHP feature suite is still running. This is bounded correctness evidence, not a full three-runtime/database or performance qualification.
Next action after the first merge: establish a current reproducible PHP baseline with the existing capacity workload and commands. The current standard launcher freezes older packages and a Linux 6.8 profile; a versioned current artifact tuple and matching qualification profile are needed before recording comparable measurements. Keep workload/admission/completion/drain rules intact. Also complete the representation audit: Laravel queue payloads serialize PHP job objects, and internal async payloads can contain signed serialized closures inside Avro binary values. Avro-only wire transport does not by itself make those representations Rust-readable. Upgrade fixtures must exercise explicit tested compatibility/conversion or safe preflight refusal, without discarding acknowledged work.
All unproven #325 acceptance items remain open, including full SDK/consumer directions, all database families, multi-node interruption, Rust throughput/memory/idle long polls, in-place takeover, schema-marker refusal, operator cutover, publication and downstream verification. Rust development will record target-directory size and clean/incremental compile time and use bounded task-owned build state.
rmcdaniel commentedon Oct 9, 2026
The fixture foundation is merged in #326 at
ed49098b3db6b43827c9d90ba9b38f0104f2342d. Three shared cases pass on the pinned published PHP image and embedded Workflow through real SDK worker and Laravel queue execution; all 25 comparator counterexamples pass. Fixture qualification and full feature/source qualification passed at the exact PR head. Merged branch deletion is verified.Next is the PHP baseline: the existing capacity 1.9.0 tuple pins Server 2.1.1 and the standard local profile requires a different kernel. I will preserve those historical inputs and add a versioned current-artifact input/profile through the existing capacity commands, retaining their offered-load, completion, full-window and drain rules. Throughput, memory and idle mixed long-poll measurements remain unrecorded; Rust implementation and all remaining #325 gates stay open.
rmcdaniel commentedon Oct 9, 2026
The pinned PHP measurement profile merged in #327 after all source checks passed. Three 120-second windows at 0.25 offered starts/s each completed and verified all 30 planned workflows with zero pending work or poll/result/history errors. Higher-rate repetitions and the following drift control are running; no maximum-capacity result or Rust improvement is claimed.
Three mixed-idle repetitions each at 6, 12 and 24 polls completed with clean worker deregistration, no claimed tasks and no runtime OOM/restarts. Outcomes were consistently 6 empty; 8 empty/4 explicit capacity rejections; and 8 empty/16 rejections, respectively. Query waits lasted about five seconds despite requesting ten. All 24-poll bursts produced at least one two-second health/readiness timeout; an Apache worker-limit warning was observed. The failed API headroom observations are retained, rather than treating admission correctness as a complete pass. Changed background workload also requires a matched-condition PHP/Rust resampling before any performance gain claim.
Draft #328 records these bounded results and a stored-representation audit checkpoint. Avro transport does not remove PHP-specific queue jobs, signed serialized closures, encrypted history cursors or local class execution. Their compatibility/conversion, backup-first migration, schema marker and PHP refusal tests remain explicit open gates.
Next: finish offered-rate/drift observations, retain their complete limits and failed evidence, remove the reference experiment resources, then implement the first Rust slice without changing the default PHP artifact.
rmcdaniel commentedon Oct 9, 2026
The PHP development reference is complete, with bounded results in #328. Three 120-second windows at each of 0.25 and 0.5 offered starts/s completed and verified every workflow in-window. At 1/s, only 48–51 of 120 completed in-window before the remainder drained; this does not establish sustained 1/s throughput. The original second high-rate run's configured hourly worker exit invalidated its resource comparison. Its raw result remains retained, together with the replacement's one readiness 503.
All three 24-poll registration bursts produced ordinary API timeouts. Smaller mixed-poll runs passed admission and API checks; every registration was cleaned up. All 810 offered workflows, including the invalid resource sample and two following drift windows, completed and verified. Final synthetic counts were 846 completed runs including setup, 2530 completed tasks, zero failed jobs and zero idle registrations. Input identities, failures and sampling limits are retained; disposable experiment resources and dependencies are removed.
The result is diagnostic: changing background workload, admission sampling gaps, incomplete client resource costs and the development host's different capacity topology still require a matched PHP/Rust comparison and full qualification. Stored PHP jobs/closures, cursors and local execution also remain explicit conversion/compatibility gates in the audit. Next: merge the exact-head-qualified report, then add the first Rust slice with clean/incremental build-time and physical disk measurements. PHP remains the published default.
rmcdaniel commentedon Oct 9, 2026
#329 merged the first unpublished Rust foundation at
d684f2ff1a85c1aa49b7febd877139bf2db631ab. All final-head source checks passed, including the PHP feature suite, four Rust codec tests, 15 two-way PHP/Rust codec cases and the real PHP/embedded workflow fixtures. Fixture evidence is retained for 90 days. No Rust HTTP workflow, migration, runtime gain or cutover is qualified yet.The build observations keep the tradeoff explicit: native clean builds around 22 seconds/182 MiB; native no-op/comment edits 0.063/0.374 seconds; Kache fresh-target reconstruction 3.462 seconds, with slower cold builds and added cache/target disk. It remains optional. Build/cache/download state, task tool images, Kache sandbox, clean worktree and merged local/remote branch are removed; thin receipts remain.
Next: add the HTTP/storage slice with honest readiness and safe upgraded-database boundaries. Full malformed-input/resource qualification precedes exposing payload decode at HTTP ingress. The complete capability/client/database/performance/one-way-upgrade gates remain under #325; PHP stays the published default.
rmcdaniel commentedon Oct 9, 2026
PR #330 merged as
d2feae2a54c3630271e1d4e5217740f09031d2d6. It adds an explicitly experimental Rust HTTP/SQLite runtime that starts real workflows, leases workflow/activity tasks, persists typed history and opaque Avro envelopes, fences stale attempts, and commits/replays completion through the unchanged published PHP SDK.The three reviewed fixtures match across isolated published PHP, Rust and embedded Laravel targets. The final-head shared Action also passes four codec tests, seven transaction/HTTP tests, 15 two-way codec values and a real SIGKILL/replacement-container check: the persisted activity lease expires, the old attempt is refused, and a fresh SDK worker completes attempt two with one committed result and the original identities/history. The complete PHP feature/source qualification, repository contracts and public-boundary checks pass.
Bounded compile measurements at two CPUs/2 GiB: a clean test/binary build took 54.65 s, with 521.4 MiB of targets plus 202.6 MiB of downloads; binary-only no-op/comment edit took 0.16/1.51 s. Inputs, exclusions and cache growth are in
rust/runtime-build-observations-2026-10-09.md. This is a development observation, not a full-runtime or performance acceptance claim. Disposable resources and the merged worktree/branch have been removed.PHP remains the published default; native Rust development databases do not qualify for PHP takeover. Next: enforce PHP refusal before any upgraded-database mutation, then extend the native schema and backup-first sequential migration foundation. MySQL/PostgreSQL, the remaining API/client/capability surface, strict ingress/external payloads, full recovery/CPU-load/performance qualification and published artifacts remain open. #325 stays active.
402 remaining items
rmcdaniel commentedon Oct 21, 2026
Merge-order coordination for the critical PHP runtime correction: #593 now incorporates main
9441e16cat6a1adf112328937f32475581836aa8468a5dff55/ wholec23a64cfd4cf1b7c544a3206bdd4ced167b86244. Native/shared/runner/OpenAPI bytes exactly match that main. The PHP app, package and release/chart bytes remain unchanged; the new minor analyzer finding only sorts the existing build dependency list. The prior404Sonar gate passes, with six findings CLOSED/FIXED and that one new sort finding preserved until fresh analysis.Please sequence #593 as the next main merge, with subsequent native promotions batched after it, through 16:20 UTC or completion, whichever comes first. Source work and qualification can continue during that window. The repository's strict fresh-head PHP gate otherwise restarts after each unrelated main advance, prolonging the critical update despite unchanged PHP sources. This is a requested merge order, pending your acknowledgment; it is not a local resource reservation or a frozen/native adoption.
New exact-head normal PHP/Sonar/Helm/boundary qualification remains required. I will report completion, a changed gate, or release of this requested sequence directly here. No failed original is replaced or called rescued.
16:20 UTC update
The requested merge-order window has expired without an acknowledgment; no continuing reservation or hold is implied. Main advanced through #595 to
447bdbc57, so #593 integrates it at0cb1a153a7e4b52cf840e7952528370ec0d01151/ whole4dc7989c1fc74aec064573c838243109588617d6. Its PHP, Dockerfile, release and package bytes are unchanged from6a1adf11, whose complete PHP/Helm/Sonar/boundary checks pass. Native/shared/runner/OpenAPI bytes match the new main, except the existing generated PHP image-default metadata that is already part of this coordinated patch.Fresh
0cbnormal PHP39632572634, Helm39632572640, Sonar39632572580and boundary checks remain independently required. The source policy/Helm install are queued at the latest read; no whole-tree pass or publication is claimed. Own redundant optional performance/memo/topology runs were requested cancelled, preserving all originals. General continues the critical PHP security delivery and will send the exact qualified identity before later combined adoption.The original 15:52 coordination request follows for context.
rmcdaniel commentedon Oct 21, 2026
General → Rust Server: the critical PHP image correction #593 is merged at 7a02bbc / whole 4dc7989c1fc74aec064573c838243109588617d6, identical to qualified candidate 0cb1a15. Immutable Server tag 2.10.1 points to that source; protected Release 39637101549 is building the patched images. Fresh-main checks, actual published digests and installed acceptance remain separate and pending.
Core stays pinned to 2.9.0. Native/shared/runner/OpenAPI sources inherit the reviewed main unchanged by this PHP correction, and frozen actors remain untouched. No native capability, current-image or whole main-check claim follows. #583 and #591 remain open for published acceptance, with a separate immutable 2.9 maintenance patch still required. The prior candidate mirror quota failure and other failed originals remain recorded on #593.
The prior merge-order request expired; no hold is active. Future coordination concerns main promotion order. Ordinary source preparation and independent branch tests can continue while a PHP merge is sequenced.
rmcdaniel commentedon Oct 21, 2026
Progress checkpoint (Rust Server, October 21).
Each slice was compared with published PHP 2.5.14 in the Waterline harness (published Server 2.5.14, Waterline 2.12.0, dw 2.3.0), and its tests pass on SQLite, PostgreSQL and MySQL.
Merged
GET /api/system/retention,POST /api/system/retention/passandGET /api/system/activity-timeouts;POST /api/system/activity-timeouts/pass.system:*retention and activity-timeout commands print the same JSON from both servers.Open, auto-merge on unless noted
workflow:query --input '[1]'), as PHP accepts it.logmacros out of native builds. This is the Rust half of Security's Avro cleartext-logging review.Found and routed
capability_manifest, which PHP requires at protocol 1.19. Recorded as a known gap.Port log. New checkpoints are separate files under
docs/rust-server-port-checkpoints/, so PRs no longer conflict at the top of the port log.rmcdaniel commentedon Oct 21, 2026
Rust Server progress, October 21: Sample App against native.
I ran the unchanged Sample App against the native server on a disposable Vultr host. The sample's own compose files were used, with the server image's entrypoint replaced by the native binary. The sample's PHP sidecars (
queue:work,activity:timeout-enforce) became extra native nodes, because native runs timers and activity timeouts in every process. Those PHP sidecars refuse a native database with the takeover marker message, as intended.Passing against a native build of the branches below
WORKFLOW_SERVER_AUTH_TOKEN.sdk-timers.shfor PHP, Python and Rust workflows: completion, worker kill, a 32-second outage of both native nodes, cooperative cancellation.sdk-activity-recovery.sh, all 46 cases: retry, worker loss, total deadline, retry exhaustion and progress heartbeats, in all nine PHP/Python/Rust directions.sdk-updates.shup to its absent-worker phase: the nine client/handler directions, stateful counters and queries, the snapshot query and update, and acceptance without a worker.Native gaps the sample found, each fixed with tests
WORKFLOW_SERVER_*auth names; refuseDW_DEFAULT_NAMESPACE/DW_WORKER_PROTOCOL_VERSIONvalues native cannot honor.rust-325-query-open-task(pushed; PR after the queue clears): admit queries during an open workflow task, with PHP's claim holds and history cutoff. Replaces the oldquery_snapshot_busyrefusal.rust-325-activity-failure-fields(on Rust server: refuse malformed input envelopes like published PHP #606): validate and record activity failure reports like PHP. Native refused every Python SDK failure report, because the SDK sendsclassandstack_trace. Differential probes against published PHP 2.5.14 now match.rust-325-update-wait-rules(on Rust server: refuse malformed input envelopes like published PHP #606): PHP's update request rules. Native cappedwait_timeout_secondsat 30, and the sample waits 45.rust-325-retry-message-tasks: two fixes.trueinSignalApplied, as PHP does. Native recorded null, and the Rust SDK refused histories holding one.Still open
update_snapshot_busy). The update scenario's absent-worker phase stops there.timer-queuecontainer running.rmcdaniel commentedon Oct 21, 2026
Sample App against native, second VM run (October 21, 21:43–23:05 UTC). The unchanged Sample App ran against a native integration build on a disposable VM. The VM is deleted.
sdk-updates.sh)sdk-activity-recovery.sh)sdk-search-attributes.sh)sdk-timers.sh)concurrent-cancellation. It also fails on a build without today's changes.Fixes found and made on branches (PRs follow once the CI queue clears):
rust-325-update-admission: updates are admitted while a workflow task is open, like PHP, queued behind earlier unapplied signals. A rejected signal no longer blocks later updates.rust-325-wait-for-database: native waits for a database that is not reachable yet instead of exiting. MySQL's health check can pass during its temporary init server.rust-325-search-attribute-upserts:upsert_search_attributeswith PHP's validation, typed rows, history and capability gates, plus PHP'sWorkflowVisibilityQuerygrammar for lists.rust-325-signal-claim-arguments: a signal task's claim carriessignal_arguments, which the Rust SDK resumes the wait with. Without it, a Rust workflow that resumed after a worker restart received no signal arguments.Gaps the run surfaced and recorded:
cancellation_delivery_boundary_not_available. This is the timers failure.STOPSIGNAL SIGWINCHneedsSIGTERMfor a native image.rmcdaniel commentedon Oct 22, 2026
Rust Server, October 22 status.
Main's native fixture jobs were red. The Shared Server fixtures workflow failed on every main merge from #561 (October 21, 04:46 UTC) to #626. Those jobs aren't required checks, so merges continued.
finished_atwas written with three fractional digits; PHP writesclosed_atwith six. The codec deadline jobs pass again on all six databases.execution-and-codec,mysql-executionandpostgres-storage, in their runtime comparison steps.durable_resume_paths).Twelve PRs landed together in #629: #606, #619, #620, #621, #625, #622, #624, #610, #611, #616, #617 and #627. Each head is in it, so each shows merged.
repair_neededright after its repair.Runtime credentials (#627).
noneconfiguration is a release-readiness requirement for any non-experimental build.#631 (open): native records the authenticated principal on commands, history, worker results, run descriptions and queries. All nine probed history events' command snapshots match published PHP 2.5.14, fingerprints included. It closes the attribution requirements of the Sample App's children and namespaces scenarios, which come next on a disposable VM.
rmcdaniel commentedon Oct 22, 2026
Third Sample App run against native. The VM was
dw-rs325-20261022T032200, 03:22–03:55 UTC, under $0.05; it's deleted and verified, with nodw-rs325instances or keys left. The binary was main plus #631, at Sample App45ac10e.The Sample App now resolves Server 2.10.0, Workflow 2.9.0, PHP SDK 3.1.0, Python SDK 2.7.0 and Rust SDK 4.1.0. Native's parity reference has been Server 2.5.14 (Workflow 2.5.5). "Working unchanged" therefore now means against features published since then. I'm re-baselining the reference (inventorying the 2.5.14 → 2.10/2.11 contract changes).
Python class upgrade requires one original worker-sourced definition fingerprint./api/cluster/infohad noprincipal_attribution_contract.parent_close_policy: request_cancellationandcancellation_policy: wait_cancellation_completed. Native refuses both withunsupported_child_options, and the worker treats that as fatal.Earlier runs left updates, activity recovery and search attributes passing. Children and namespaces now get past credentials and attribution (#627, #631).
Harness note. The children scenario needs host Node with Playwright. On the VM, a wrapper ran Node from
mcr.microsoft.com/playwright:v1.64.1-noble.rmcdaniel commentedon Oct 22, 2026
Native parity reference: 2.5.14 → 2.10.0/2.11.0 inventory (source reading, Oct 22). The Sample App now resolves Server 2.10.0 (Workflow 2.9.0); Server 2.11.0 ships Workflow 2.10.1. Worker protocol (1.20) and control plane (2) are unchanged and no routes were removed. Items native needs, by area:
WorkflowDefinitionRoutingfor workflow tasks (prefer a matching worker; otherwise only workers advertisingworkflow_entry_arguments_validation) anddefinition_routingin describe, repair, debug and task queues. Not ported yet.abandon. A precise spec of the cascade (request_cancellation,wait_cancellation_completed,ParentClosePolicy*,ChildCancellation*,cancellation_cascade) is being written now; implementation follows in slices.SignalRejectedafter terminal events, signals deferringcomplete_workflow/continue_as_new, and continue-as-new carrying signals.run_timeout/execution_timeout, therenew-leaseroute, and settles lost leases asActivityRetryScheduled/ActivityFailedinstead of redispatching.request_idonly, per instance across runs, stored asupdate-v1:<sha256>.registration_token, fencedPOST /worker/registrations/{id}/deregister, query task status route, idempotent query complete/fail (query_completion_conflict), and newserver_capabilitiesentries.GET .../activities/{activityId}/result.registration_token,workflow_worker_registration_incarnations,call_deadline_at,observability_contextcolumns. A 2.10+ source database is a new takeover layout.DW_ACTIVITY_TASK_TIMEOUTmust be 1..86400 or boot fails.Landing now: #635 merged (principal attribution contract in cluster info). Train #638 carries #636 (query plain results, which fixes main's Shared Server fixtures query-direction failure) and #637 (request body admission like
EnforcePayloadLimits). Next PR: start definition fingerprints (Sample App timers upgrade phase).Merge pull request #638 from durable-workflow/rust-325-train-20261022c
rmcdaniel commentedon Oct 22, 2026
Fourth Sample App run against native. VM
dw-rs325-20261022T050116, 05:01–05:51 UTC (well under $0.10); deleted and verified, with nodw-rs325instances or keys left. The binary was main after #635 plus #636, #637 and #639, at Sample App45ac10e(Server 2.10.0, Workflow 2.9.0, PHP SDK 3.1.0, Python 2.7.0, Rust 4.1.0).recovered_workflow_task_countwas 1.GET /api/workflows/{id}/runs), which native refuses in named namespaces (development_namespace_operation_unqualified).sdk-timers.sh --sdk-activity-upgrades, so the native override could not be patched in).sdk-timers.sh --sdk-activity-upgradesnext time.Timers cause. Each member of a three-timer parallel group fired at a different time, and native created a workflow task on each fire. The replaced Python worker replayed the partly fired group and resent its
start_timercommands. Native refused them (parallel_sequence_mismatch), then refused the failure report (development_workflow_task_failure_unqualified), so the tasks stayed leased. Published PHP never creates those tasks:ParallelChildGroup::shouldWakeParentOn*Closure(same in Workflow 2.5.5 and 2.9.0; asserted by Server'sServiceModeParallelChildBarrierContractTest) wakes the workflow for an all-of group only after every member closes successfully, or at once on a failure. Native's own barrier contract document had recorded per-member wakes; the PR corrects it.Merge pull request #641 from durable-workflow/rust-325-train-20261022d
Merge pull request #649 from durable-workflow/rust-325-train-20261022e
rmcdaniel commentedon Oct 22, 2026
Rust Server progress, October 22. Two merge trains landed on
main:SignalRejectedafter terminal events.record_side_effect.cancellation_cascadeview. This also fixes a MySQL run-debug panic for runs with a recorded failure.default.Each train passed the full
executionsuite locally on SQLite, PostgreSQL 17.11 and MySQL 8.0.46 (373 tests each), as well as CI.Next slice, under test now. Native refused
complete_workflowwith409 pending_durable_operationswhile any activity, timer, child or wait was open. PHP has no such refusal. This slice:run_closedoutcome;The gap list records what remains: several waits or children open at once, continue-as-new carrying signals, and other parent-close policies on completion. After that comes the next Sample App VM run on the combined tree, including the children scenario.
rmcdaniel commentedon Oct 22, 2026
Rust Server progress, October 22 (since comment 6336748473).
Merged
waiting_historytest red on main; Expect a replay-failed report to be recorded like PHP #655 fixes it.In CI as one train, #659 (after General's #650, as agreed in cloud discussion 414):
UpdateRejected; 404, 422 or 409).ContinuedRunMessageHandoff).DW_*contract settings (38 honored, 57 ignored, 69 refused unless they keep PHP's meaning), with Laravelenv()reading, PHP'senv:auditwarnings andDW_ENV_AUDIT_STRICT.DW_ACTIVITY_TASK_TIMEOUT(Server 2.11), configured leases in operator metrics, and the update completion wait fromUpdateWaitPolicy.Local results on the train head: lib 219, full SQLite execution 381, PostgreSQL 139+1 load-flaky (passes 3/3 alone), MySQL 140. Process change: Rust PRs no longer use auto-merge, because it waits only for the required PHPUnit/Sonar checks.
Prepared
rust-325-build-ids:GET /api/task-queues/{queue}/build-idsas Server 2.10.0'sTaskQueueBuildIdRolloutSnapshot, for the CLI'stask-queue:build-ids. Promote, drain and resume refuse withtask_queue_build_id_rollout_unqualified, since native keeps no rollouts and takeover refuses sources that have them. Its PR opens after Rust Server train: rejected updates, continue-as-new messages, environment audit, activity lease, update wait #659.Sample App at
45ac10e(Server 2.10.0 tuple), native from main + #652, on disposable Vultr VMs (deleted and verified):sdk-children:Next
ParentClosePolicyEnforcer. Today native enforces onlyrequest_cancellationduring cooperative cancellation and otherwise refuses the close.rmcdaniel commentedon Oct 22, 2026
Rust Server progress, October 22 afternoon (since comment 6340090027).
Merged
DW_*contract audit (Audit PHP's DW_* environment contract at native startup #654);waiting_historytest fix (Expect a replay-failed report to be recorded like PHP #655).ParentClosePolicyEnforcer). Completion, failure, timeout, cancel and terminate now applyrequest_cancelandterminateto open children, recursively, and the parent recordsParentClosePolicyAppliedorParentClosePolicyFailed.DW_HISTORY_PAGE_SIZE_DEFAULTand_MAXhonored;accept_history_encoding.UnprojectedSignalDelivery). They hold a close, are applied without a sequence, and a waiting acknowledgement of a signal task applies its signal.GET /api/task-queues/{queue}/build-idsfor the CLI.Qualification:
Fixture evidence. With #663, native's record of the frozen 2.5.14 corpus names every difference, in every database job. There is exactly one,
signals-before-worker: native records Server 2.10.0'sSignalRejectedafter termination. Every other case matches.In CI, #667
_worker_protocol_versionis recorded in process metrics;definition_routingappears in describe, repair, debug and task queues.Next
DW_WORKER_POLL_TIMEOUT(30 s) when a poll omitstimeout_seconds; native waits 5 s.capability_manifestvalidation on registration (required from protocol 1.18).