Skip to content

fix(copy): defer foreign key checks to the commit for a whole copy - #3859

Merged
datlechin merged 15 commits into
TableProApp:mainfrom
MrWeb:fix/copy-to-foreign-keys
Oct 27, 2026
Merged

datlechin merged 15 commits into
TableProApp:mainfrom
MrWeb:fix/copy-to-foreign-keys

Conversation

@MrWeb

@MrWeb MrWeb commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • New PluginSessionPurpose.relaxedForeignKeys session scope: the driver turns off the target's foreign key checks for that session, and closing it puts back the setting the session had. MySQL and MariaDB implement it; PostgreSQL does not, since session_replication_role also stops user triggers and needs a superuser.
  • The table order was already parents first. What failed with 1452 was a cycle, a self reference with a child row ahead of its parent, or a source row whose parent is gone.
  • Rows are written with checks off only under Stop and roll back, inside a transaction the copy owns, when the copy creates every table it fills and every key points at a table it fills with all its rows (ObjectCopyPlanner.keysInside). Otherwise they stay on: turning them back on checks nothing already written, and a table the target keeps (Data only, Add rows to it) may hold keys the source cannot speak for.
  • Before it commits, a copy that wrote with checks off counts each key's rows that point at nothing (ObjectCopyPlanner.keyChecks, one anti-join per key in the target's quoting). A source row whose parent is gone, or one read while the source was changing, fails the copy and rolls it back instead of committing past keys nothing checks again. The checks run once on the still-empty tables before the relax; an account that cannot run them keeps enforcement on.
  • Stop and roll back writes every table's rows in one transaction, so a failure takes all of them back. Committed table by table, a cycle's first table kept rows pointing at a second that then failed. A MySQL table whose engine keeps rows through a ROLLBACK (MyISAM) is still counted.
  • Stop and keep what ran and Skip and continue commit each batch as it goes, with checks on. Wrapped per table, PostgreSQL committed the aborted transaction as a ROLLBACK and lost every batch ahead of the failure. A table that fails or is stopped part way counts the batches it kept, on either path, including a target with no transactions and a driver that throws a cancellation.
  • Both data paths lease the target before the source, so two copies in opposite settings cannot each hold the lease the other waits for.
  • The structure phase uses the same scope around table DDL, for a replacement or a copy whose keys stay inside it, instead of the hand-run disable/enable pair that forced checks on afterwards. A CREATE whose key names a table outside the copy keeps the checks, so it still fails when that table is missing.
  • Clears for a Replace keep checks on, so a table outside the copy keeps its own ON DELETE rule. A table whose clear failed gets no rows.
  • New On error picker in the Copy To sheet: Stop and roll back, Stop and keep what ran, Skip and continue.
  • A structure phase that throws, or fails to commit, rolls back before the connection returns to the pool, which never ends a transaction itself. Before, a failed CREATE SCHEMA on PostgreSQL left the next lease inside an aborted transaction.
  • Tables are ordered parents first across every schema in the copy, so duplicating a PostgreSQL database with a key into another schema no longer fails. Names match exactly: Shop and shop are two MySQL databases.
  • Compare & Sync orders MySQL and PostgreSQL tables parents first: a key's referenced schema named the endpoint's own namespace and never matched the snapshot. Drops now see target-only tables' keys.
  • Copy To reads table structure in bulk when the selection covers every listed table, instead of one table at a time.
  • SQLite and local libSQL get a session scope: .preserve saves foreign_keys, and .relaxedForeignKeys returns nil inside a transaction, where the pragma is a no-op. A relax whose read-back fails turns foreign_keys back on. SQL import already held .preserve; Save now holds it around its prologue, outside the user's transaction only: inside one the pragma does nothing, and PostgreSQL would put a SET LOCAL value back session-wide. Neither leaves a SQLite connection enforcing foreign keys afterwards.
  • PostgreSQL's SET session_replication_role runs in a DO block that swallows insufficient_privilege, so a non-superuser import or save no longer fails or aborts the open transaction.
  • libSQL remote and Cloudflare D1 no longer claim a foreign key toggle: a pragma per request does nothing there.
  • PluginKit: additive enum case, kit stays 41 (unreleased).

Test plan

  • ObjectCopyTransactionTests with separate source and target drivers: relax on the target only, after the clears, closed on throw and cancel, none under a session transaction; keys outside the copy keep the checks (keysOutsideTheCopyKeepTheChecks, creationRelaxesOnlyInsideTheCopy); every table in one transaction under Stop and roll back (rollbackRunsTheRowsInOneTransaction); Stop and keep what ran commits each batch and counts what a failure kept (stopAndKeepCommitsEachBatch); a rollback counts nothing and a target with no transactions counts what it kept (rolledBackFailureCountsNothing, untransactionalTargetCountsWhatItKept); keys are checked before commit and a broken one rolls back, and one the account cannot run keeps the checks (relaxedCopyChecksKeysBeforeCommit, brokenKeyRollsTheCopyBack, uncheckableKeyKeepsTheChecks); target leased first (leasesTheTargetFirst); a MyISAM-style table keeps its count through a rollback (tableThatCannotRollBackKeepsItsCount); a thrown cancellation keeps its count (thrownCancellationKeepsTheCount); a failed clear gets no rows; a failed schema rolls back (failedSchemaRollsBack).
  • ObjectCopyPlannerOrderingTests (ordering, keysInside, keyChecks), ObjectCopyPlannerStructureReadTests, CompareForeignKeyOrderTests, SQLiteSessionScopeTests (SQLiteDriverTests and TableProPluginKitTests), MySQLSessionScopeTests, PostgreSQLSessionScopeSQLTests, DataWriteExecutorTests/planWithoutPrologueOpensNoScope and saveInsideTheUsersTransactionOpensNoScope.
  • Updated on purpose, for the new scope events: ObjectCopyDDLSessionTests (5 cases), ObjectCopySessionScopeTests/runnerPinsSameEngineCopy and runnerLeavesCrossEngineCopyUnscoped, DataWriteExecutorTests togglesRunOutsideTheTransaction, togglesRunAfterRollback, aRequiredPrologueThatFailsStopsTheRunBeforeAnyStep.
  • The 1452 cases and the parents-first order were measured on MySQL 8.4.
  • Unit plan: all pass except AppearancePaneListTests/themesListHoldsStillWhileAccessibilityReadsIt and GutterNumberOffsetTests/windowGutterKeepsItsMargin, which fail on main on this Mac too (locked screen, Dark Mode).
  • Docs: docs(features): foreign key checks during Copy To and the On error setting docs#134

Screenshots

The configure step of the Copy To sheet, rendered from the test host (the Mac was locked, so XCUITest could not drive it).

Light Dark
Before Before light Before dark
After After light After dark

Fixes #3858

@datlechin datlechin changed the title fix(copy): turn off foreign key checks while Copy To writes rows fix(copy): write rows with foreign key checks off and put back what the session had Oct 27, 2026
@datlechin datlechin changed the title fix(copy): write rows with foreign key checks off and put back what the session had fix(copy): defer foreign key checks to the commit for a whole copy Oct 27, 2026
@datlechin
datlechin merged commit 86651d7 into TableProApp:main Oct 27, 2026
8 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Copy To: fails on foreign key constraints (no FK_CHECKS disable, alphabetical table order) + very slow compare phase

2 participants