Skip to content

Investigate intermittent approved-update HTTP 500 in published PHP MariaDB corpus #825

Closed

Description

@rmcdaniel

General accepts the PHP investigation handed off by the native Server owner. This is an intermittent published-PHP qualification failure; its root cause is not yet established.

The owner handoff identifies update-validation / approved_update returning HTTP500 instead of the expected202 on the PHP side of the MariaDB10.11 corpus in #822 run40578777402, job122454294740, using published Server2.11.5. The earlier #816 run40561089085, MariaDB job122392116543 reported the same response shape on published2.11.4. Neighboring full corpus runs pass; that does not explain either500.

The retained PHP log is a tail and does not retain the correlated request exception. A later external-payload exception belongs to a different fixture and must not be used as this failure's diagnosis. This is distinct from a demonstrated native-source regression. The normal #822 failed-job rerun is owned by the native maintainer; General will retain the original failures regardless of that rerun's result.

Current evidence: the owner correlated a later state-updates HTTP 500 with SQLSTATE 40001 / InnoDB 1213 while inserting workflow_tasks, reported in the request-specific receipt. The original approved-update 500s above remain uncorrelated. Both original published MariaDB probe contracts passed once in a controlled baseline, which does not explain the intermittent failures.

General pushed 517e2647512e7f94722050ace5d33fecceb48ef6 with owned-root MySQL/MariaDB admission retries after rollback. Twenty real-engine controls and 82 related tests pass; source and qualification receipt. Follow-up head adaad79ac327fda10d40dd4df96f5ce5e7b7dc1f also fences the selected run inside retried admission. A real deadlock concurrent with continue-as-new previously let the direct path record an unknown-update rejection after the requested run changed. All 24 real-engine controls, 78 related tests and 54 controller tests pass; the strengthened final four race cases pass with 140 assertions. See follow-up qualification. The approved path already refused stale validation. Task-owned database fixtures/network have been removed.

Merged PR: #828, main 759abaa3b4eb7d931cf256b1aab2950768c2c0a9. The combined PR head passed 16 checks with 7 conditional skips; the normal merge group passed all 43 Shared Server jobs and its full PHP feature qualification. Source review and the specific Sonar maintainability decision are recorded. The admission correction is included in published 2.11.7; the exact full-corpus scope and remaining historical uncertainty are described below.

Published follow-through: Server 2.11.7 / Core 2.15.7 completed the first finite PHP-only MariaDB corpus with 74 of 82 cases passing and eight differences; update-validation and many-updates pass. The first record is retained. Subsequent classification separates four masked-route-cache fixture differences, missing writable /state for storage-test, the existing MariaDB #569 health GROUPBY defect whose allowance needs the truthful database-image hint, a default-prefix fixture assumption, and the #770 reference correction accepted in #867. Corrected observations and targeted follow-through preserve each scope. In particular, the operator tuple's third field is health namespace, not task namespace; metrics remain200. No task-isolation finding or repaired-health claim is supported.

The explicit PHP timing-helper handoff is implemented in PR #870, current combined head c108ed328de8360aa05589f01a66eed35c331977: elapsed time now covers the logical waiting request across lock-pressure attempts/backoff using a monotonic clock. Actual HTTP tests fail both retry-duration cases on the old helper and pass all four on the fix; focused Node26/26 and existing PHP retry6 tests/70 assertions pass, as do lint/actionlint. Frozen timing bounds, retry policy and product runtime are unchanged. Source review is accepted. The prior combined a9f head passed the full 43-job Shared Server run and entered the normal queue. Main #874 then merged, requiring one comparator-command conflict resolution. Current c108 merges main9670, retains the exact test argv union with one argument per line, and passes114 focused Node tests plus actionlint. Fresh current-head qualification is green: Shared40978243986 succeeds and all57 PR checks complete without failures. PR870 merged normally as6c1e63dd at01:14:27 UTC after successful protected merge-group Shared/PHP checks. Normal-main Shared40984638681/PHP40984638686/Sonar40984638635 all succeed on6c1e63dd. The historical500 disposition remains separate and this issue stays open. Timer helper/test bytes remain unchanged from5edd.

Fresh full follow-through is now 83/83 PASS, 2026-10-29 20:50:00.262–20:57:15.518 UTC, using PR870 runner5edd, actual published2.11.7/2.15.7, a fresh database/default prefix, stock container layout and truthful MariaDB hint. Exact full-record receipt and limits. All82 old cases remain (81 old fixture hashes unchanged; accepted#867 updates waterline-import); the already-merged child-descriptions fixture is the83rd. Known published-defect allowances and MariaDB#569 health500 remain modeled; metrics are200. No lock-pressure retries were reported in this full run.

Next action: preserve the accepted first/full records and complete cleanup of PR870's task resources now that protected merge and normal-main Shared/PHP/Sonar qualification have passed. Retain a narrowly scoped disposition for the original historical approved-update500s, whose exception remains missing. A passing rerun does not explain their cause; this issue remains open.

Acceptance:

  • Preserve the original response, exact published PHP/Core tuple, database version, fixture runner and workload; qualify whether the input and expected202 remain valid.
  • Reproduce or retain correlated evidence that distinguishes product storage/validation failure from fixture/environment failure. Do not assign a cause from a neighboring log entry.
  • Correct any confirmed product defect with focused regression coverage, normal source qualification and published-consumer follow-through. Keep required checks and original failing evidence.
  • Verify the affected full MariaDB corpus after repair; a passing rerun alone does not close this investigation.

Activity

  1. rmcdaniel commented on Oct 27, 2026

    @rmcdaniel
    MemberAuthor

    General accepted the published-PHP investigation from the Rust owner. Original artifact metadata independently confirms artifact 12773370526, mysql-execution-mariadb-10.11-corpus, run 40578777402, at source 0e7e955244d7a30fb19694bb04c263b90c44e582. The archive is 15,382 bytes and is not expired. Its normal GitHub download timed out, so I have not independently inspected its contents or correlated the HTTP 500 with an exception.

    The reports for published PHP 2.11.4 and 2.11.5 remain separate evidence. The unrelated payload exception in the retained log tail is not a root-cause finding. A passing failed-job rerun would not establish the cause. Next: recover the original request/case correlation and exercise that published-PHP MariaDB path independently, retaining any first failure before changing code.

  2. rmcdaniel commented on Oct 27, 2026

    @rmcdaniel
    MemberAuthor

    The Rust owner supplied a later, request-correlated report from run 40578777402, attempt 2, job 122482789008: the published PHP 2.11.5 record returned HTTP 500 for the state-updates fixture's selected-run set_payload update at 21:40:40 UTC. The reported PHP exception was SQLSTATE 40001, MariaDB error 1213, on insertion into workflow_tasks, with maintenance running. The published-PHP recording failed before the native recording ran. This is an owner-reported correlation; General has not independently recovered that log. It does not establish the exception behind the earlier approved_update failures.

    Source inspection traced remote workflow updates through Server's ExternalWorkflowUpdateAdmission, including the outer approval transaction in ValidatedExternalWorkflowUpdateAdmission. Their default ControlPlaneMutationRetrier path only retries SQLite, so the reported MariaDB path is a candidate retry gap. Core's embedded update admission also uses a single-attempt transaction, but it is a separate path and is not yet implicated in this remote fixture failure. Existing Core command-deadlock coverage exercises MySQL signal/cancellation cases and explicitly skips MariaDB. No reproduced root cause or delivered fix is claimed.

    Next: exercise the exact original probe inputs against published PHP 2.11.5/Core 2.15.1 on isolated MariaDB, retain correlated request and exception evidence, then qualify a scoped retry correction with rollback, idempotency, caller-owned transaction, and unrelated-error controls. Passing reruns do not close this issue.

  3. rmcdaniel commented on Oct 27, 2026

    @rmcdaniel
    MemberAuthor

    The isolated published-PHP reproduction harness is working. Original state-updates and update-validation probes from runner source 0e7e955244d7a30fb19694bb04c263b90c44e582 each completed once and passed the canonical checkObservation contract. The tuple is published Server 2.11.5, Core 2.15.1, normal locked PHP SDK 2.2.6, MariaDB 10.11.19. The database was bootstrapped with the frozen published 2.5.14 layout and migrated with the published current image. The repair/schedule maintenance loop stayed active; the isolated server's deadlock counter remained zero.

    This first pair did not reproduce either intermittent HTTP 500. It is not a root-cause finding or closure evidence. The exact selected images/input hashes and small original observations are retained only for this active investigation; disposable data, raw logs and task resources will be removed after the correlated diagnostic receipt.

    The next useful check is a deterministic rollback/deadlock control at the actual Server external-admission boundary. A correction must retry an owned root transaction, preserve a caller-owned transaction, and avoid repeating validator execution or committed effects. The validated path has an outer approval transaction around parent admission, so blindly enabling nested retries would be insufficient. No product source has changed for this issue yet.

  4. rmcdaniel commented on Oct 27, 2026

    @rmcdaniel
    MemberAuthor

    Implemented and pushed the scoped correction at 517e2647512e7f94722050ace5d33fecceb48ef6, including actual merged #824. The application change is b8c685e73 on fix/825-external-update-deadlocks; only the two external-update admission classes and a regression test differ from current main.

    Server now asks Laravel to retry owned root MySQL/MariaDB admission transactions using the existing configured attempt budget. Nested approval admission remains owned by its outer approval transaction. Caller-owned transactions, other storage connections, unrelated failures and after-commit failures retain their original behavior. Validator approval remains outside the retried admission, and the retained approval is rechecked on each attempt.

    Before/after evidence:

    • A real two-connection InnoDB lock cycle makes update admission the deadlock victim. Both direct admission and admission with an already-approved validator receipt fail on the original code: 2 tests / 42 assertions / 2 failures, with MariaDB error 1213 while accessing workflow_tasks.
    • The correction passes 10 tests / 234 assertions on MariaDB 10.11.19 and 10 tests / 222 assertions on MySQL 8.0.46, with no errors, failures or skips. Controls cover rollback and one committed command/update/history/task, retained-key deduplication, an unchanged validator receipt, configured single-attempt failure, caller-owned rollback, unrelated constraint errors and failures after commit.
    • Five existing update/routing/validation/retry suites pass 82 tests / 963 assertions. Changed PHP files pass Pint.

    This reproduces and fixes the root-transaction retry gap at the actual Server external-admission boundary. It does not independently identify the exception behind the earlier uncorrelated approved_update 500s, or establish acceptance of a newly published Server image. Those failures remain retained. The original published 2.11.5 baseline still needs replacement only after a qualified publication and coordination with the native fixture owner.

    No PR or release has been created yet while CI throughput repair #826 is under qualification. Next: defensive exact-source review, protected checks/merge, qualified publication and published-artifact verification of the reported corpus cases.

  5. rmcdaniel commented on Oct 28, 2026

    @rmcdaniel
    MemberAuthor

    General pushed follow-up head adaad79ac327fda10d40dd4df96f5ce5e7b7dc1f on fix/825-external-update-deadlocks.

    A controlled real InnoDB deadlock now also exercises a worker's actual continue-as-new operation between rollback and retry. Before the run guard, MariaDB's direct case retried twice, answered unknown_update HTTP 404 and left one durable update row after the requested run changed. The approved case already refused stale validation. The first loose probe allowed admission to commit before continuation and was discarded as an invalid interleaving; the retained negative control holds the history range so rollback actually precedes continuation.

    The controller now passes the selected run ID into external admission. Fresh admission rechecks it while holding the instance/run locks on each retry, and refuses a changed run with the existing historical_run_command_rejected HTTP 409. The guard runs after retained-key recovery, preserving committed receipts, and before validation can start for a different selected run. Ordinary successful response metadata is unchanged. No native or frozen-fixture source changed.

    Validation on the source logic: all 24 real-engine controls passed on MySQL 8.0.46 and MariaDB 10.11.19 (604 assertions), all 78 related request/validation/routing/fence/retrier tests passed (934 assertions), and all 54 control-plane/controller tests passed (819 assertions). After strengthening the race assertions, its four final cases passed with 140 assertions: one rolled-back insert attempt, correct original run/reason, and zero admitted command/update/history rows. Pint passed all four changed PHP files. This supplements the original rollback-only retry proof; it does not correlate the original approved-update HTTP 500 reports.

    Next: exact-source review, normal CI/PR progression after the current CI bottleneck repair, then published-image acceptance and the full affected MariaDB corpus. No PR, tag or release is claimed yet.

  6. rmcdaniel commented on Oct 28, 2026

    @rmcdaniel
    MemberAuthor

    Merged #828 through the normal queue as 759abaa3b4eb7d931cf256b1aab2950768c2c0a9. All 43 merge-group Shared Server jobs and the merge-group PHP feature qualification pass. The PR head had 16 successful checks and 7 conditional skips, with its source/Sonar review recorded in the PR.

    This puts the demonstrated owned-transaction retry and selected-run corrections on main. It is not a published-image acceptance result. Next: qualify main, publish through the protected release path with the appropriate qualified Core dependency, and repeat the full affected MariaDB corpus against the exact published image. The originally reported approved-update HTTP 500s remain uncorrelated and this issue stays open.

  7. rmcdaniel commented on Oct 29, 2026

    @rmcdaniel
    MemberAuthor

    Handoff from #854's merge-queue run 40841285248 (MariaDB 10.11 corpus lane, job 123339409484). General owns the PHP follow-through.

    • Published PHP: Server 2.11.5 with Workflow 2.15.1 (tests/Fixtures/ServerParityBaselines/php-2.11.5.json; linux/amd64 durableworkflow/server@sha256:bcc672859a399d1df234fc304531ab06b7bfa8acad7cb2c9da95f8e7ab5c98b4).
    • Database: MariaDB 10.11.19.
    • What happened: the many-updates probe sent its 16th POST /api/workflows/{id}/update/set (wait_for: accepted, with a request_id) while the installed maintenance loop (workflow:v2:repair-pass then schedule:evaluate) was running. The answer was an uncaught 500 with body {"message":"Server Error"}.
    • Server log: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction, on a select * from workflow_tasks where workflow_run_id = … in the update admission.
    • Workaround in Rust #325: describe runs after more than 64 updates #854: the probe now sends the identical submission (same request_id) again, at most three times in all. Each retry writes a value-free note (many_updates_submit_retry: index, attempt, status) that the recorder keeps even when the probe passes. So a recovered 500 stays visible as a defect signal, not error-free qualification.
  8. rmcdaniel commented on Oct 29, 2026

    @rmcdaniel
    MemberAuthor

    Correction, October 29: I initially misread the third operator comparison field as a task namespace. It is health_namespace; the corrected interpretation appears below. The original recorded values are unchanged.

    Published PHP-only MariaDB follow-through completed on October 29, 17:56:11–18:02:42 UTC. This is the first retained record: 74 of 82 selected cases pass; eight differences remain unclassified, recorder exit 1 / product-fail. No full-pass or historical approved_update 500 root-cause claim follows from this run. update-validation and many-updates pass in this record; the original intermittent failures above remain open.

    Exact tuple: published Server 2.11.7, source/runner f55f90e, immutable Linux/amd64 child sha256:be7c2288ff935396262d743590530b204fac11b76582e56a525cc81a8da840d3; Core 2.15.7 / 3aa98b8473b130dbc141457f76ccc32e4aa58062; normal Composer PHP SDK 2.2.6 / 290fa427a76ffc6ea226bcd59749374aa7a89efa; published CLI 2.2.0, SHA256 fce532e6a49737aef9cb8a137fd66219928b2595764462be33b12a0fdf5abc7c. Real MariaDB 10.11.19 used a fixed 1 CPU / 1 GiB memory-and-swap limit, 128 MiB InnoDB buffer, TLS fixture, and no host ports. The database was initialized through the published layout and normal bootstrap path. Real repair/schedule maintenance ran during the finite corpus. This was a PHP-only HTTP profile, without native Server execution.

    The eight differences are:

    • operator-metrics: the health response namespace is null because health returns HTTP 500 after an activity exists. This matches the known MariaDB PHP Server health returns 500 on MariaDB (ONLY_FULL_GROUP_BY in task queue build-id snapshot) #569 health defect; the metrics response itself retains HTTP 200 and the correct namespace. The client omitted MYSQL_IMAGE, so the probe did not identify the MariaDB family for its existing documented disposition.
    • schedule-catalog, schedule-fixed-rate-one-occurrence, schedule-manual-lifecycle, schedule-policies: four schedule audit route-name differences.
    • start-search-attributes: supplied custom workflow-ID prefix differs from a hardcoded reference prefix.
    • storage-test: storage roundtrip returns HTTP 500 rather than 200.
    • waterline-import: recorded history bytes are 4713 rather than 5431; reference/meter intent belongs to the existing coordinated reference work.

    Next: preserve and classify these differences using the first observations and correlated diagnostics, distinguish fixture/reference assumptions from product behavior, and obtain request-correlated evidence for the storage 500. No rerun has replaced this record. A passing repeat alone will not close either historical admission failure. This does not infer an authorization/isolation failure from null metadata.

    The finite client is removed. Task-owned diagnostics and the isolated database are retained for classification; maintenance has been stopped after preserving the run's application log. Cleanup checkpoint is completion of classification and durable evidence retention. The normal public 2.11.7 release and its specific Bridge acceptance are already documented in #861; that acceptance does not imply this full corpus passed.

  9. rmcdaniel commented on Oct 29, 2026

    @rmcdaniel
    MemberAuthor

    General also accepted the native owner's explicit PHP helper handoff: update-validation-probe.php must measure the complete logical request, including lock-pressure retries, while preserving frozen timing bounds and product runtime behavior. Add focused retry and cached-receipt controls. This helper correction is separate from identifying the original intermittent admission 500. It follows the current Laravel security-floor qualification; native reference/frozen changes remain with their owner.

  10. rmcdaniel commented on Oct 29, 2026

    @rmcdaniel
    MemberAuthor

    The storage-test HTTP 500 in the October 29 first corpus now has request-correlated evidence. Both POST /api/storage/test requests at 18:00:23 UTC match two application exceptions at the same time from RuntimeLocalExternalPayloadStorage::__construct, reached through NamespaceExternalPayloadStorage::driverFor and StorageController::test: the configured external payload root could not be created.

    The runner's storage-test-probe.php explicitly configures file:///state/storage-test-<target>/<namespace>. The normal server-parity.yml PHP container provides writable /state. This local disposable fixture omitted that mount, so its unprivileged application could not create the configured root. The first record remains retained unchanged; a corrected-envelope targeted repeat is still pending. This explains the fixture setup difference and does not establish a storage runtime regression.

    This is separate from the historical approved_update failures: those still have no request-correlated exception, and neither these storage exceptions nor a later unrelated external-payload exception is their diagnosis. The other seven first-record differences remain for classification.

    The finite maintenance process is stopped. The isolated database and bounded request/access/error diagnostics remain only until classification and durable evidence retention are complete; then remove the task-owned fixture.

  11. rmcdaniel commented on Oct 29, 2026

    @rmcdaniel
    MemberAuthor

    The first 82-case record remains 74 pass / eight differences. Targeted follow-through on the same published Server 2.11.7 / Core 2.15.7 / runner f55f90e54a213872f1c164a1b22c082d859c18aa now classifies those observations more precisely:

    • Restoring the stock image's route cache makes the four unchanged schedule cases pass: catalog, fixed-rate occurrence, manual lifecycle and policies. The selected record ran 19:25:09–19:25:27 UTC. The first fixture had masked the baked cache, leaving route names null; the stock routes have generated names that the existing comparator normalizes.
    • Providing the required writable /state makes the same namespace's real storage diagnostic return HTTP 200, with both 64-byte and 70,000-byte roundtrips passing. The first run's two 500s are correlated with failure to create that configured storage root.
    • Correction to my earlier label: the operator comparison's null field is health_namespace. Metrics is HTTP 200 with the correct namespace throughout. Health becomes 500 after an activity exists, and the actual exception is MariaDB 1055 / workflow_tasks.compatibility not in GROUP BY, the existing #569 defect. My client omitted the MYSQL_IMAGE hint used by the probe to identify MariaDB. Supplying the exact inspected database image makes the unchanged operator case pass its existing known-defect disposition at 19:52:45–19:52:56 UTC. This does not repair health; PHP Server health returns 500 on MariaDB (ONLY_FULL_GROUP_BY in task queue build-id snapshot) #569 remains open. The earlier “task namespace” interpretation was incorrect and is corrected in the first receipt.
    • The unchanged start-search-attributes case passes using the documented default parity-v1 prefix. The earlier custom prefix exposed its hardcoded expected workflow ID, a fixture comparison limitation rather than an incorrect returned workflow ID.
    • The Waterline byte difference remains in the reference/meter alignment owned by #770 and #867. No frozen/native files were edited for these follow-ups.

    These selected records do not replace the first full record or constitute a clean full-corpus qualification. Next: qualify the whole finite PHP MariaDB profile with the normal image envelope, truthful database-family metadata, fresh database/default prefix, and the accepted reference alignment. Preserve the historical approved-update 500s without assigning either this storage exception or #569's health exception as their cause.

    The selected clients are removed and maintenance is stopped after each finite record. The isolated task database and small diagnostics remain for full follow-through; remove them once that result and remaining dispositions are durable.

  12. rmcdaniel commented on Oct 29, 2026

    @rmcdaniel
    MemberAuthor

    Fresh full published-PHP MariaDB qualification passes 83/83 cases, 2026-10-29 20:50:00.262–20:57:15.518 UTC, exit0. This is PHP-only acceptance of the current reviewed corpus, including its explicit published-defect allowances.

    Exact tuple: Server2.11.7 sourcef55f90e54a213872f1c164a1b22c082d859c18aa, actual stock amd64 imagesha256:be7c2288ff935396262d743590530b204fac11b76582e56a525cc81a8da840d3, installed Core2.15.7/3aa98b8473b130dbc141457f76ccc32e4aa58062, PHP SDK2.2.6/290fa427a76ffc6ea226bcd59749374aa7a89efa; visibility verifies published CLI2.2.0 PHAR SHAfce532e6a49737aef9cb8a137fd66219928b2595764462be33b12a0fdf5abc7c. Runner is PR870 head5eddc262dea97b3c99f5188ba9577eb15c692a48, based on accepted current reference6af963db. Real MariaDB10.11.19 uses pinned imagesha256:7db29378d4fdab73f8123bbc2b48905c90d1a4b00cf848b028f1e81e623257f2, inspected memory+swap1GiB/1GiB. Fresh database was bootstrapped using the normal published2.5.14 layout then2.11.7 bootstrap; defaultparity-v1 prefix, truthful actual MYSQL_IMAGE, normal stock container layout with writable/state.

    The original74/82 record is unchanged (SHA7da1483736cecfba3e755ab65f2fc7d3953377d9b958bc58f5d11922a76023e4). Current full record SHA8a950d2bbea3bfc3f0dcc58b9c3a39d413822aa2ae129e4d8829b061b0e1b41c. Both artifact tuples are identical. All82 original cases remain;81 original fixture hashes are identical, while the owning Rust maintainer's accepted#867 canonical-meter change updates waterline-import. Its already-merged child-descriptions fixture adds the83rd case. General changed no frozen fixture or native source.

    All eight earlier differences now have bounded dispositions supported by the retained first record and targeted controls: four masked-route-cache fixture differences; missing writable/state; the omitted MariaDB image hint for the existing#569 allowance; a custom-prefix fixture assumption; and the accepted#770 canonical-meter reference update. Whole-profile schedule/storage/start-search/update-validation/waterline checks now pass. Approved update is202 (0.314183821s logical request); this run reported no lock-pressure retry notes, so retry timing itself is proved by PR870's actual-child real-HTTP negative/positive controls, not inferred from this no-retry run.

    Known PHP defects remain modeled: cancellation-scopes, record-local-activity, workflow-import#765 and worker-sessions. In operator-metrics, metrics are200 in every stage; health is200 in the three pre-activity stages and still500/null health namespace in the four later stages under the documented MariaDB#569 allowance. This is no repaired-health or task-isolation claim. Bridge checks exercise the published2.11.7 correction. The normal hosted PR870 qualification remains pending.

    The original historical approved-update500s remain uncorrelated and this issue stays open. A passing full record does not supply their missing exception or prove a cause. Next: finish PR870 hosted qualification/merge, preserve the reviewed evidence and retain a narrowly scoped historical-failure disposition. The finite client was reaped and its maintenance process stopped; task-only resource cleanup follows review. Retained local records are for this active investigation, with removal reconsidered after the historical disposition and qualification receipts are accepted.

  13. rmcdaniel commented on Nov 2, 2026

    @rmcdaniel
    MemberAuthor

    Closing: the HTTP 500 was published-PHP-only, PHP Server is frozen (operator direction, November 1), and native 3.0.1 can't produce it.

    • Every 500 in this issue was the published PHP reference (2.11.4 and 2.11.5), on MariaDB SQLSTATE 40001 / 1213 against workflow_tasks. PHP's fix (Retry owned external update admissions after InnoDB deadlocks #828) shipped in 2.11.7. Native recordings in those runs passed. The MariaDB lane of merge-group run 41514975042 (the source of native-3.0.1) recorded outcome: pass for PHP, embedded and native.
    • Native at 923d45b3: update admission (admit_update, rust/src/runtime/updates.rs) is one transaction. The validator wait runs outside it, and the approved path re-enters admission and rechecks the run and the approval. Any database error, a deadlock included, rolls back. RuntimeError's response mapping (rust/src/runtime/mod.rs) then answers it as 503 storage_unavailable, never 500, with a redacted runtime_storage_unavailable log line carrying the driver code.
    • Not this issue: the {"many_updates_submit_retry":…"status":500} lines in PHPUnit logs come from ServerParityManyUpdatesRetryTest's fake 500s, not from a server.

    Weaker relative in native, not a 500: a deadlock on update admission isn't retried in-process and isn't answered with the typed, retryable backend_lock_pressure + Retry-After some other routes use. Clients get a plain 503. There's also no native two-connection InnoDB deadlock test for update admission. If that becomes worth doing, it's a native issue of its own: retry or typed lock pressure on admission, plus a deadlock test proving rollback and no duplicate under one request_id.

  14. rmcdaniel commented on Nov 2, 2026

    @rmcdaniel
    MemberAuthor

    The concurrency test noted above is tracked in #960.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions