Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions .github/workflows/server-parity.yml
Original file line number Diff line number Diff line change
Expand Up @@ -554,6 +554,7 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json
native_record_status=0
docker run --rm --user=1000:1000 --entrypoint node --network "$RESOURCE_SCOPE" \
-v "$PWD:/source:ro" -v "$PWD/parity-evidence:/evidence" -w /source \
-e DW_PARITY_TOKEN=parity-test-token -e DW_PARITY_RUNNER_REVISION -e DW_PARITY_CLI_PHAR -e MYSQL_IMAGE "$image" scripts/conformance/server-parity.mjs record \
Expand All @@ -564,7 +565,8 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/patch-created-activity-completed.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json || native_record_status=$?
compare_status=0
docker run --rm --user=1000:1000 --entrypoint node --network=none \
-v "$PWD:/source:ro" -v "$PWD/parity-evidence:/evidence" -w /source "$image" \
scripts/conformance/server-parity.mjs compare /evidence/php.json /evidence/rust.json /evidence/embedded.json \
Expand All @@ -574,7 +576,7 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/patch-created-activity-completed.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json > parity-evidence/comparison.json
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json > parity-evidence/comparison.json || compare_status=$?
bash scripts/conformance/server-parity/published-sdk-patch-profile.sh mysql http://parity-php:8080 http://parity-native-a:8080
bash scripts/conformance/server-parity/repeated-patch-profile.sh mysql http://parity-php:8080 http://parity-native-a:8080
bash scripts/conformance/server-parity/legacy-marker-profile.sh mysql http://parity-php:8080 http://parity-native-a:8080
Expand Down Expand Up @@ -686,6 +688,10 @@ jobs:
catalog php_ref > parity-evidence/catalog-after.json
cmp parity-evidence/php-before.json parity-evidence/php-after.json
cmp parity-evidence/catalog-before.json parity-evidence/catalog-after.json
# A native fixture difference fails the step only after every later
# check has run, so one difference cannot hide the rest.
test "$native_record_status" -eq 0
test "$compare_status" -eq 0
- name: Compare separate configured role-token cohort
run: bash scripts/conformance/server-parity/role-profile.sh mysql
- name: Back up, take over and resume a separate published PHP upgrade source
Expand Down Expand Up @@ -1051,6 +1057,7 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json
native_record_status=0
docker run --rm --user=1000:1000 --entrypoint node --network "$RESOURCE_SCOPE" \
-v "$PWD:/source:ro" -v "$PWD/parity-evidence:/evidence" -w /source \
-e DW_PARITY_TOKEN=parity-test-token -e DW_PARITY_RUNNER_REVISION -e DW_PARITY_CLI_PHAR "$image" scripts/conformance/server-parity.mjs record \
Expand All @@ -1061,7 +1068,8 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/patch-created-activity-completed.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json || native_record_status=$?
compare_status=0
docker run --rm --user=1000:1000 --entrypoint node --network=none \
-v "$PWD:/source:ro" -v "$PWD/parity-evidence:/evidence" -w /source "$image" \
scripts/conformance/server-parity.mjs compare /evidence/php.json /evidence/rust.json /evidence/embedded.json \
Expand All @@ -1071,7 +1079,7 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/patch-created-activity-completed.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json > parity-evidence/comparison.json
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json > parity-evidence/comparison.json || compare_status=$?
bash scripts/conformance/server-parity/published-sdk-patch-profile.sh pgsql http://parity-php:8080 http://parity-native-a:8080
bash scripts/conformance/server-parity/repeated-patch-profile.sh pgsql http://parity-php:8080 http://parity-native-a:8080
bash scripts/conformance/server-parity/legacy-marker-profile.sh pgsql http://parity-php:8080 http://parity-native-a:8080
Expand Down Expand Up @@ -1183,6 +1191,10 @@ jobs:
catalog php_ref > parity-evidence/catalog-after.json
cmp parity-evidence/php-before.json parity-evidence/php-after.json
cmp parity-evidence/catalog-before.json parity-evidence/catalog-after.json
# A native fixture difference fails the step only after every later
# check has run, so one difference cannot hide the rest.
test "$native_record_status" -eq 0
test "$compare_status" -eq 0
- name: Compare separate configured role-token cohort
env:
POSTGRES_IMAGE: ${{ matrix.image }}
Expand Down Expand Up @@ -1542,6 +1554,7 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json
native_record_status=0
docker run --rm --user=1000:1000 --entrypoint node --network "$RESOURCE_SCOPE" \
-v "$PWD:/source:ro" -v "$PWD/parity-evidence:/evidence" -w /source \
-e DW_PARITY_TOKEN=parity-test-token -e DW_PARITY_RUNNER_REVISION -e DW_PARITY_CLI_PHAR "$image" scripts/conformance/server-parity.mjs record \
Expand All @@ -1552,7 +1565,8 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/patch-created-activity-completed.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json || native_record_status=$?
compare_status=0
docker run --rm --user=1000:1000 --entrypoint node \
-v "$PWD:/source:ro" -v "$PWD/parity-evidence:/evidence" -w /source "$image" \
scripts/conformance/server-parity.mjs compare /evidence/php.json /evidence/rust.json /evidence/embedded.json \
Expand All @@ -1562,7 +1576,7 @@ jobs:
--include-fixture tests/Fixtures/ServerParityPending/patch-created-activity-completed.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-type.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-waiting-history-message.json \
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json > parity-evidence/comparison.json
--include-fixture tests/Fixtures/ServerParityPending/workflow-task-transient-retry.json > parity-evidence/comparison.json || compare_status=$?
bash scripts/conformance/server-parity/published-sdk-patch-profile.sh sqlite http://parity-server:8080 http://parity-rust:8080
bash scripts/conformance/server-parity/repeated-patch-profile.sh sqlite http://parity-server:8080 http://parity-rust:8080
bash scripts/conformance/server-parity/legacy-marker-profile.sh sqlite http://parity-server:8080 http://parity-rust:8080
Expand Down Expand Up @@ -1673,6 +1687,10 @@ jobs:
php scripts/conformance/server-parity/cooperative-restart.php finish http://parity-recovered:8080 /evidence/cooperative-restart-receipt.json > /evidence/cooperative-restart-finish.json
php scripts/conformance/server-parity/restart.php finish http://parity-recovered:8080 /evidence/restart-receipt.json > /evidence/restart-finish.json
'
# A native fixture difference fails the step only after every later
# check has run, so one difference cannot hide the rest.
test "$native_record_status" -eq 0
test "$compare_status" -eq 0
- name: Compare separate configured role-token cohort
env:
RESOURCE_SCOPE: dw-parity-${{ github.run_id }}-${{ github.run_attempt }}-${{ github.job }}
Expand Down
65 changes: 65 additions & 0 deletions docs/rust-server-port-checkpoints/2026-10-22-definition-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# October 22, workflow definition routing

- **Gap.** Native records the definition fingerprint a start's worker cohort
agrees on, with `workflow_definition_fingerprint_source: worker`, as PHP
does. It then offered that run's workflow tasks to any worker, so a worker
with another definition could take a pinned run while the recorded one was
live.
- **Source.** Server 2.10.0's `WorkflowDefinitionRouting::forWorker`,
which `WorkflowTaskPoller::matchesRecordedWorkflowDefinition` applies at
claim, and `WorkflowReplayEligibility`.
- **Decision.** For a workflow task of a worker-pinned run:
- a worker advertising the recorded fingerprint for the type takes it
(`matching_definition`);
- without a recorded fingerprint (`recorded_definition_missing`), or
without one from the worker (`worker_definition_missing`), nobody
does;
- a worker with another definition waits while an eligible peer
advertises the recorded one (`matching_definition_preferred`);
- otherwise it takes the task only if it advertises
`workflow_entry_arguments_validation` (`replay_required`, else
`entry_validation_unavailable`).

An unpinned run goes to any worker (`not_worker_pinned`). Native
refuses to start with `DW_V2_PIN_TO_RECORDED_FINGERPRINT` off, so
pinning is always on.
- **Eligible peers** (`eligibleWorkers`). Active registrations on the
run's namespace and queue that:
- match the build the task or run requires;
- support the type, with workflow capacity above zero;
- replay the run (`WorkflowReplayEligibility::allows`). That means a
heartbeat inside the stale window and cooperative-cancellation support
when the run has a request, at the protocol version the worker last
reported. It also means the memo and typed search attribute
capabilities its history needs.
- **Protocol version.** Each registration and heartbeat records its
protocol version under `_worker_protocol_version` in the worker's process
metrics, after the metrics it reported. A heartbeat without metrics
keeps the stored ones. Worker listings therefore show the key, as PHP
2.10.0's do; 2.5.14 had no such key.
- **Reports** (`inspect` and `forQueue`).
- Describe, repair and debug report `definition_routing` for the run's
next due ready task:
- the context (`pinning_enabled`, `worker_pinned`,
`recorded_fingerprint`, type, queue and required build);
- PHP's status, the counts of eligible workers by definition, up to ten
example workers, `blocked` and `runnable_task`;
- PHP's resolution text.
- A blocked route adds debug's `workflow_definition_routing_blocked`
warning and `diagnostic_status: needs_attention`.
- Task queue describe samples up to ten runs with a due ready task. A
blocked sample sets `repair.needs_attention`.
- **Tests** (`definition_routing.rs`).
- A pinned run goes to the worker with its recorded definition. Workers
with another definition, or without one, wait even when they validate
entry arguments.
- Once the recorded definition leaves, only an entry-validating worker
takes the run.
- An unpinned run goes to any worker.
- Registrations and heartbeats record the protocol version with the
reported metrics.
- Describe, debug, task queue and repair report a blocked route, then the
route an entry-validating replacement unblocks.
- The worker listing test now expects the protocol key.
- Full SQLite suite: 398 passed. Routing, worker, poll, child, cooperative,
debug, task queue, repair and describe tests: PostgreSQL 134, MySQL 134.
9 changes: 0 additions & 9 deletions docs/rust-server-port.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,15 +182,6 @@ contract fixture before its correction.
then did not stop within the 5-second shutdown budget. Forty local
repetitions of that case alone passed. Find what holds the query thread
after the fence before changing either side.
- **Workflow definition routing (Server 2.10.0).** Native records the
definition fingerprint a start's worker cohort agrees on, as PHP does.
Published PHP 2.10.0 also routes a task for such a run by that
fingerprint. It prefers a live worker advertising the recorded definition.
Otherwise it offers the task only to a worker that advertises
`workflow_entry_arguments_validation`. It reports `definition_routing` in
describe, repair, debug and task queues, and stores each worker's protocol
version under `_worker_protocol_version` in its process metrics. Native
offers such a task to any worker and reports none of this.
- **Start build-id pin.** Published PHP pins a new run to its start cohort's
build id: a promoted build-id rollout, else the single build id among the
live workers for the type, else among all the queue's live workers. Native
Expand Down
Loading
Loading