From 8087f64e9162579665c6a627683dc7e5f0575050 Mon Sep 17 00:00:00 2001 From: Simon Binder Date: Tue, 20 Oct 2026 12:26:45 +0200 Subject: [PATCH 1/2] Draft: Direct tables --- architecture/client-architecture.mdx | 1 + .../advanced/performance-optimization.mdx | 28 +++++++++++++++++++ client-sdks/advanced/raw-tables.mdx | 8 ++---- client-sdks/cascading-delete.mdx | 5 ++-- client-sdks/high-performance-diffs.mdx | 2 ++ 5 files changed, 37 insertions(+), 7 deletions(-) diff --git a/architecture/client-architecture.mdx b/architecture/client-architecture.mdx index 80403f326..dffce579f 100644 --- a/architecture/client-architecture.mdx +++ b/architecture/client-architecture.mdx @@ -36,6 +36,7 @@ The PowerSync Client SDK automatically maintains the following tables in the SQL | `ps_data__` | This contains the data for each "table", in JSON format. Since JSON is being used, this table's schema does not change when columns are added, removed or changed in the Sync Streams/Rules and client-side schema. | | `ps_data_local__
` | Same as the previous point, but for [local-only](/client-sdks/advanced/local-only-usage) tables. | | `
` (`VIEW`) | These are views on the above `ps_data` tables, with each defined column in the client-side schema extracted from the JSON. For example, a `description` text column would be `CAST(data ->> '$.description' as TEXT)`. | +| `
` (`TABLE`) | For [direct tables](/client-sdks/advanced/performance-optimization#direct-tables), PowerSync replaces the `ps_data__` storage table and view with a singl table matching your schema. | | `ps_untyped` | Any synced table that is not defined in the client-side schema is placed here. If the table is added to the schema at a later point, the data is then migrated to `ps_data__
`. | | `ps_oplog` | This is the operation history data as received from the [PowerSync Service](/architecture/powersync-service), grouped per bucket. | | `ps_crud` | The client-side upload queue (see [Writing Data](#writing-data-via-sqlite-database-and-upload-queue) below) | diff --git a/client-sdks/advanced/performance-optimization.mdx b/client-sdks/advanced/performance-optimization.mdx index b48b3e07f..11e2402e4 100644 --- a/client-sdks/advanced/performance-optimization.mdx +++ b/client-sdks/advanced/performance-optimization.mdx @@ -50,6 +50,34 @@ If this reveals it took too long for a read connection to become available, cons For contention on the write connection, note that the PowerSync client processes changes from the PowerSync Service in a single write transaction after they've been downloaded. Due to consistency requirements, this transaction cannot be split into multiple steps, and especially for a large initial sync it can hold a write lock for several seconds. +## Direct tables + + + **Availability** + + Direct tables described in this section were introduced in the following versions of our client SDKs: + + - **JavaScript** (Node: `todo`, React-Native: `todo`, Web: `todo`) + - **Dart**: Version `todo` of `package:powersync`. + - **Kotlin**: Version todo. + - **Swift**: Version todo. + - **Rust**: Version todo. + - **.NET**: Version todo + + +By default, PowerSync stores all data as [JSON rows](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure): Synced rows are copied into local tables without modifications, and local updates modify an existing JSON object. This format is very highly flexible and avoids schema migrations, but comes with a performance cost: + +- Every column read forces SQLite to decode the entire JSON object, making queries slower than queries on "real" SQLite tables. +- JSON objects occupy more storage space than a native SQLite row, making databases slightly larger and the initial sync slower. + +[Raw tables](/client-sdks/advanced/raw-tables) fix this by making PowerSync into tables you define. With raw tables, you are responsible for creating and migrating them. +Direct tables are raw tables that PowerSync can manage for you, like those based on views. In supported SDKs, direct tables +can be enabled with a simple flag: + +TODO: snippets + +TODO: Limitations (no view name override, no high-performance diffs, eperimental) + ## Fixing Slow Queries If a query itself is expensive and takes a long time to run, several options can improve performance. Some of these require a restructuring of your app's schema. diff --git a/client-sdks/advanced/raw-tables.mdx b/client-sdks/advanced/raw-tables.mdx index 4a3a55e03..1f9a9b89f 100644 --- a/client-sdks/advanced/raw-tables.mdx +++ b/client-sdks/advanced/raw-tables.mdx @@ -4,10 +4,6 @@ description: "Access raw SQLite tables to bypass JSON view limitations for nativ sidebarTitle: "Raw SQLite Tables" --- - - Raw tables are not currently compatible with the [High Performance Diffs](/client-sdks/high-performance-diffs) feature available in our JavaScript SDKs. We have a [public proposal](https://docs.google.com/document/d/12bvZDJF2aaTOqkxHyo3W-TYGQTaS9FQFdu-MX2yOuqI/edit) open to discuss this addition - your feedback is welcome! - - By default, PowerSync uses a [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) where data is stored schemalessly in JSON format and then presented through SQLite views based on the client-side schema. Raw tables allow you to define native SQLite tables in the client-side schema, bypassing this. This eliminates overhead associated with extracting values from the JSON data and provides access to advanced SQLite features like foreign key constraints and custom indexes. @@ -32,7 +28,9 @@ Consider raw tables when you need: - **Indexes** - PowerSync's default schema has basic support for indexes on columns, while raw tables give you complete control to create indexes on expressions, use `GENERATED` columns, etc. - **Improved performance** for complex queries (e.g., `SELECT SUM(value) FROM transactions`) - raw tables more efficiently get these values directly from the SQLite column, instead of extracting the value from the JSON object on every row. - **Reduced storage overhead** - eliminate JSON object overhead for each row in `ps_data__
.data` column. -- **To manually create tables** - Sometimes you need full control over table creation, for example when implementing custom triggers. +- **To manually create tables** - Sometimes you need full control over table creation, for example when implementing custom triggers or if you use an existing library like [Room](/client-sdks/orms/kotlin/room) managing the schema for you. + +If you only need raw tables for their performance and reduced storage overhead, [direct tables](/client-sdks/advanced/performance-optimization#direct-tables) (experimental) may be a better option. These use the same architecture as a raw tables, but are fully managed by PowerSync (so no explicit migrations are necessary). **Advanced SQLite features** like `FOREIGN KEY` and `ON DELETE CASCADE` constraints need [special consideration](#using-foreign-keys). diff --git a/client-sdks/cascading-delete.mdx b/client-sdks/cascading-delete.mdx index 4d01e366e..d5fa08aaa 100644 --- a/client-sdks/cascading-delete.mdx +++ b/client-sdks/cascading-delete.mdx @@ -7,9 +7,9 @@ keywords: - "delete" --- -PowerSync [uses SQLite views](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) instead of standard tables, so SQLite features like foreign key constraints and cascading deletes are not available. +By default, PowerSync [uses SQLite views](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) instead of standard tables, so SQLite features like foreign key constraints and cascading deletes are not available. -There's no built-in support for cascading deletes on the client, but you can achieve this in two ways: +There's no built-in support for cascading deletes on the client, but you can achieve this in three ways: 1. Manual deletion in a transaction - Delete all related records in a single transaction (recommended for most cases) @@ -19,6 +19,7 @@ There's no built-in support for cascading deletes on the client, but you can ach You create triggers on the internal tables (not the views defined by the client schema), similar to what is done [here](https://github.com/powersync-ja/powersync-js/blob/e77b1abfbed91988de1f4c707c24855cd66b2219/demos/react-supabase-todolist/src/app/utils/fts_setup.ts#L50). +3. Raw tables - PowerSync can sync into existing tables defined by your app, and those tables can use [foreign key constraints](/client-sdks/advanced/raw-tables#using-foreign-keys). ## Example: Manual Transaction diff --git a/client-sdks/high-performance-diffs.mdx b/client-sdks/high-performance-diffs.mdx index b8e1265a2..6a6f9a1fc 100644 --- a/client-sdks/high-performance-diffs.mdx +++ b/client-sdks/high-performance-diffs.mdx @@ -22,6 +22,8 @@ While [basic/incremental watch queries](/client-sdks/watch-queries) enable react - API design and developer experience - Additional features or optimizations needed + The API does not support [raw](/client-sdks/advanced/raw-tables) or [direct](/client-sdks/advanced/performance-optimization#direct-tables) tables yet. We have a [public proposal](https://docs.google.com/document/d/12bvZDJF2aaTOqkxHyo3W-TYGQTaS9FQFdu-MX2yOuqI/edit) open to discuss this addition - your feedback is welcome! + Join our [Discord community](https://discord.gg/powersync) to share your experience and get help. From fb4e1888c1fd3600e26c67325c0d4c47d486eac0 Mon Sep 17 00:00:00 2001 From: Simon Binder Date: Tue, 20 Oct 2026 15:59:17 +0200 Subject: [PATCH 2/2] AI feedback --- architecture/client-architecture.mdx | 2 +- .../advanced/performance-optimization.mdx | 69 +++++++++++++++---- client-sdks/advanced/raw-tables.mdx | 2 +- 3 files changed, 56 insertions(+), 17 deletions(-) diff --git a/architecture/client-architecture.mdx b/architecture/client-architecture.mdx index dffce579f..365f2af97 100644 --- a/architecture/client-architecture.mdx +++ b/architecture/client-architecture.mdx @@ -36,7 +36,7 @@ The PowerSync Client SDK automatically maintains the following tables in the SQL | `ps_data__
` | This contains the data for each "table", in JSON format. Since JSON is being used, this table's schema does not change when columns are added, removed or changed in the Sync Streams/Rules and client-side schema. | | `ps_data_local__
` | Same as the previous point, but for [local-only](/client-sdks/advanced/local-only-usage) tables. | | `
` (`VIEW`) | These are views on the above `ps_data` tables, with each defined column in the client-side schema extracted from the JSON. For example, a `description` text column would be `CAST(data ->> '$.description' as TEXT)`. | -| `
` (`TABLE`) | For [direct tables](/client-sdks/advanced/performance-optimization#direct-tables), PowerSync replaces the `ps_data__` storage table and view with a singl table matching your schema. | +| `
` (`TABLE`) | For [direct tables](/client-sdks/advanced/performance-optimization#direct-tables), PowerSync replaces the `ps_data__` storage table and view with a single table matching your schema. | | `ps_untyped` | Any synced table that is not defined in the client-side schema is placed here. If the table is added to the schema at a later point, the data is then migrated to `ps_data__
`. | | `ps_oplog` | This is the operation history data as received from the [PowerSync Service](/architecture/powersync-service), grouped per bucket. | | `ps_crud` | The client-side upload queue (see [Writing Data](#writing-data-via-sqlite-database-and-upload-queue) below) | diff --git a/client-sdks/advanced/performance-optimization.mdx b/client-sdks/advanced/performance-optimization.mdx index 11e2402e4..e4a23bfdf 100644 --- a/client-sdks/advanced/performance-optimization.mdx +++ b/client-sdks/advanced/performance-optimization.mdx @@ -50,33 +50,72 @@ If this reveals it took too long for a read connection to become available, cons For contention on the write connection, note that the PowerSync client processes changes from the PowerSync Service in a single write transaction after they've been downloaded. Due to consistency requirements, this transaction cannot be split into multiple steps, and especially for a large initial sync it can hold a write lock for several seconds. -## Direct tables +## Direct Tables **Availability** - Direct tables described in this section were introduced in the following versions of our client SDKs: + Direct tables are experimental. They were introduced in the following versions of our client SDKs: - - **JavaScript** (Node: `todo`, React-Native: `todo`, Web: `todo`) - - **Dart**: Version `todo` of `package:powersync`. - - **Kotlin**: Version todo. - - **Swift**: Version todo. - - **Rust**: Version todo. - - **.NET**: Version todo + - **JavaScript** (React Native: `2.3.4`, Web: `2.6.0`, Node.js: `1.2.1`) + - **Dart**: Version `2.6.0`. + - **Kotlin**: Version `1.16.0`. + - **Swift**: Version `1.17.0`. + - **.NET**: Not supported yet. + - **Rust**: Version `0.1.2`. -By default, PowerSync stores all data as [JSON rows](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure): Synced rows are copied into local tables without modifications, and local updates modify an existing JSON object. This format is very highly flexible and avoids schema migrations, but comes with a performance cost: +By default, PowerSync stores all data as [JSON rows](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure): synced rows are copied into local tables without modifications, and local updates modify an existing JSON object. This format is highly flexible and avoids schema migrations, but comes with a performance cost: - Every column read forces SQLite to decode the entire JSON object, making queries slower than queries on "real" SQLite tables. -- JSON objects occupy more storage space than a native SQLite row, making databases slightly larger and the initial sync slower. +- JSON objects occupy more storage space than a native SQLite row, making databases slightly larger and increasing writes during large sync downloads. -[Raw tables](/client-sdks/advanced/raw-tables) fix this by making PowerSync into tables you define. With raw tables, you are responsible for creating and migrating them. -Direct tables are raw tables that PowerSync can manage for you, like those based on views. In supported SDKs, direct tables -can be enabled with a simple flag: +[Raw tables](/client-sdks/advanced/raw-tables) fix this by letting PowerSync sync into actual tables you define. With raw tables, you are responsible for creating and migrating them. Direct tables are raw tables that PowerSync manages for you, like the default JSON-based tables. In supported SDKs, you enable direct tables with a flag on the table definition: -TODO: snippets + + +```typescript TypeScript +const schema = new Schema({ + users: new Table({ name: column.text }, { direct: true }) +}); +``` + +```dart Dart/Flutter +const schema = Schema([ + Table.direct('users', [Column.text('name')]) +]); +``` + +```kotlin Kotlin +val schema = Schema( + Table("users", listOf(Column.text("name")), direct = true) +) +``` + +```swift Swift +let schema = Schema( + Table(name: "users", columns: [.text("name")], direct: true) +) +``` + +```rust Rust +let mut schema = Schema::default(); +schema + .tables + .push(Table::create("users", vec![Column::text("name")], |tbl| { + tbl.direct = true + })); +``` + + + +Direct tables aren't based on views, so you can't set the `viewName` option on them. In the JavaScript SDKs, [High Performance Diffs](/client-sdks/high-performance-diffs) don't support direct tables yet. + +### Migrations + +To switch a table between the JSON-based format and a direct table, change the flag in your schema. PowerSync SDKs then migrate the existing data automatically. Schema changes on direct tables can take longer than on JSON-based tables: PowerSync must migrate the table itself, whereas JSON-based tables only need their views re-created. -TODO: Limitations (no view name override, no high-performance diffs, eperimental) +When you add or remove columns on an existing direct table, PowerSync detects the change and runs `ALTER TABLE` statements automatically. New columns contain `NULL` values until the next completed sync. ## Fixing Slow Queries diff --git a/client-sdks/advanced/raw-tables.mdx b/client-sdks/advanced/raw-tables.mdx index 1f9a9b89f..70cebf669 100644 --- a/client-sdks/advanced/raw-tables.mdx +++ b/client-sdks/advanced/raw-tables.mdx @@ -30,7 +30,7 @@ Consider raw tables when you need: - **Reduced storage overhead** - eliminate JSON object overhead for each row in `ps_data__
.data` column. - **To manually create tables** - Sometimes you need full control over table creation, for example when implementing custom triggers or if you use an existing library like [Room](/client-sdks/orms/kotlin/room) managing the schema for you. -If you only need raw tables for their performance and reduced storage overhead, [direct tables](/client-sdks/advanced/performance-optimization#direct-tables) (experimental) may be a better option. These use the same architecture as a raw tables, but are fully managed by PowerSync (so no explicit migrations are necessary). +If you only need raw tables for their performance and reduced storage overhead, [direct tables](/client-sdks/advanced/performance-optimization#direct-tables) (experimental) may be a better option. They use the same storage format as raw tables, but PowerSync creates and migrates them for you. **Advanced SQLite features** like `FOREIGN KEY` and `ON DELETE CASCADE` constraints need [special consideration](#using-foreign-keys).