Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ jobs:
- name: Idempotent reinstall
run: |
DB="postgres://$POSTGRES_USER:$POSTGRES_PASSWORD@$POSTGRES_HOST:$POSTGRES_PORT/$POSTGRES_DB"
psql -v ON_ERROR_STOP=1 -q -f vendor/cel4postgres--*.sql "$DB"
psql -v ON_ERROR_STOP=1 -q -1 -f vendor/cel4postgres--*.sql "$DB"
for f in sql/*.sql; do
psql -v ON_ERROR_STOP=1 -q -f "$f" "$DB"
psql -v ON_ERROR_STOP=1 -q -1 -f "$f" "$DB"
done

- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
Expand Down Expand Up @@ -87,8 +87,10 @@ jobs:
run: docker compose logs --no-color

# The release channels, proven on every run rather than only at
# publish time: the all-in bundle initdb-installs on the bare
# pinned image; the wrapped artifact installs through pg_tle via
# publish time: every artifact installs in its promised shape
# (plain inside a caller's transaction, -tx on its own); the
# all-in bundle initdb-installs on the bare pinned image; the
# wrapped artifact installs through pg_tle via
# CREATE EXTENSION; and the grants template actually separates
# the two roles (a reader can check but not write).
artifacts:
Expand All @@ -104,6 +106,9 @@ jobs:
- name: Build artifacts
run: ./scripts/build-release.sh

- name: Install every artifact in its promised shape
run: ./scripts/smoke-artifacts.sh

- name: Install the bundle on a bare database
run: |
version=$(sed -n "s/^VALUES ('\([0-9][0-9.]*\)');$/\1/p" \
Expand Down
43 changes: 5 additions & 38 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ permissions:
# between jobs -- fewer moving parts, and nothing to pin.

jobs:
# The plain-scripts channel: the all-in bundle initdb-installs
# on the bare pinned image and answers fga.version(); the
# engine-only file installs over the vendored cel4postgres
# bundle the same way compose layers them.
# The plain-scripts channel: scripts/smoke-artifacts.sh installs
# every artifact in the shape it promises -- each plain file inside
# a transaction it does not own, each -tx file with a bare psql -f,
# the engine-only files over the vendored cel4postgres bundle.
smoke:
if: >-
github.repository == 'emfga/fga4postgres' &&
Expand All @@ -38,40 +38,7 @@ jobs:
- name: Build and install artifacts
run: |
./scripts/build-release.sh
version=$(sed -n \
"s/^VALUES ('\([0-9][0-9.]*\)');$/\1/p" \
sql/010_install.sql)
image=$(sed -n 's/^ *image: \(postgres:[^ ]*\)$/\1/p' \
compose.yaml)
smoke() {
docker run -d --rm --name relcheck \
-e POSTGRES_DB=fga -e POSTGRES_USER=fga \
-e POSTGRES_PASSWORD=pw "$@" \
"$image" >/dev/null
# -h 127.0.0.1: over the socket, psql reaches
# initdb's temporary server mid-install; only the
# final server listens on TCP.
got=
for _ in $(seq 60); do
got=$(docker exec relcheck psql -h 127.0.0.1 \
-U fga -d fga \
-tAc 'SELECT fga.version()' 2>/dev/null) && break
sleep 2
done
docker rm -f relcheck >/dev/null
if [ "$got" != "$version" ]; then
echo "expected $version, got '$got'" >&2
exit 1
fi
}
init=/docker-entrypoint-initdb.d
echo "== fga4postgres--$version.sql"
smoke -v "$PWD/dist/fga4postgres--$version.sql:$init/install.sql:ro"
echo "== fga4postgres-engine--$version.sql over vendored cel"
smoke \
-v "$PWD/vendor:/vendor:ro" \
-v "$PWD/dist/fga4postgres-engine--$version.sql:/engine.sql:ro" \
-v "$PWD/scripts/release-engine-initdb.sh:$init/000.sh:ro"
./scripts/smoke-artifacts.sh

# The flagship channel: the all-in bundle wrapped by
# scripts/pgtle-wrap.sh into pgtle.install_extension, installed
Expand Down
20 changes: 13 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ a future contributor will otherwise reopen.
verbatim JSON per model id plus a normalized internal form for fast
resolution. Owner-confirmed.
4. **CEL is consumed from cel4postgres, never reimplemented.** The
pinned release artifact (`vendor/cel4postgres--0.0.1.sql`, from
pinned release artifact (`vendor/cel4postgres--0.0.2.sql`, from
github.com/emfga/cel4postgres) installs before `sql/` in initdb and
in the release story. Conditions evaluate through `cel.*`; the
OpenFGA dialect (`ipaddress`, `in_cidr`, typed-parameter coercion)
Expand Down Expand Up @@ -149,10 +149,16 @@ gitignored `dist/`.
simultaneously install order and initdb order (initdb does not descend
into subdirectories). fga4postgres scripts start at `010` because the
`000` slot belongs to the vendored cel4postgres bundle, bind-mounted as
a single file in compose.yaml. Every script opens its own `BEGIN;` …
`COMMIT;` and is idempotent (`CREATE OR REPLACE`, `IF NOT EXISTS`,
`ON CONFLICT`), so plain concatenation builds a release and re-running
the installer is the upgrade path.
a single file in compose.yaml. No script opens or closes a
transaction — the caller owns the boundary (`psql -1`, a migration
tool's transaction, pg_tle's `CREATE EXTENSION`), which is what lets
a plain release artifact be embedded where a transaction is already
open; `build-release.sh` refuses a `BEGIN;`/`COMMIT;` line in `sql/`
or the vendored bundle, and each artifact's `-tx` twin adds exactly
one pair around the whole file. Every script is idempotent
(`CREATE OR REPLACE`, `IF NOT EXISTS`, `ON CONFLICT`), so plain
concatenation builds a release and re-running the installer is the
upgrade path.

**Schema naming.** Everything lives in schema `fga`. Public entry
points are unprefixed (`fga.check`, `fga.version`); every internal
Expand Down Expand Up @@ -241,8 +247,8 @@ docker compose logs --no-color # on failure
Installing by hand into any database:

```bash
psql -v ON_ERROR_STOP=1 -f vendor/cel4postgres--0.0.1.sql "$DB_URL"
for f in sql/*.sql; do psql -v ON_ERROR_STOP=1 -f "$f" "$DB_URL"; done
psql -v ON_ERROR_STOP=1 -1 -f vendor/cel4postgres--0.0.2.sql "$DB_URL"
for f in sql/*.sql; do psql -v ON_ERROR_STOP=1 -1 -f "$f" "$DB_URL"; done
```

The engine version lives in exactly one place: the
Expand Down
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,24 +45,26 @@ Distribution is plain SQL scripts and

## Installation

From a release, one file installs everything:
From a release, one file installs everything, in one transaction:

```bash
psql -v ON_ERROR_STOP=1 -f fga4postgres--<version>.sql "$DB_URL"
psql -v ON_ERROR_STOP=1 -1 -f fga4postgres--<version>.sql "$DB_URL"
```

From a checkout:

```bash
psql -v ON_ERROR_STOP=1 -f vendor/cel4postgres--*.sql "$DB_URL"
psql -v ON_ERROR_STOP=1 -1 -f vendor/cel4postgres--*.sql "$DB_URL"
for f in sql/*.sql; do
psql -v ON_ERROR_STOP=1 -f "$f" "$DB_URL"
psql -v ON_ERROR_STOP=1 -1 -f "$f" "$DB_URL"
done
```

The scripts are idempotent; re-running the installer is the upgrade
path. [docs/INSTALL.md](docs/INSTALL.md) covers the pg_tle channel
and the consumer privilege template.
The scripts are idempotent and open no transaction of their own;
re-running the installer is the upgrade path.
[docs/INSTALL.md](docs/INSTALL.md) covers the `-tx` artifacts,
embedding in a migration tool, the pg_tle channel and the consumer
privilege template.

## Development environment

Expand Down
7 changes: 5 additions & 2 deletions docker/000_install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,14 @@
# fga4postgres script, in name order. This exists because initdb runs
# only a flat directory and a single-file bind mount cannot be layered
# over a read-only directory mount -- so sql/ and vendor/ are mounted
# at their own paths and this script sequences them.
# at their own paths and this script sequences them. The scripts
# open no transaction of their own, so --single-transaction keeps
# each file all or nothing -- which matters on the bench's
# persistent volume, not on compose's tmpfs.
set -eu

for f in /vendor/cel4postgres--*.sql /sql/*.sql; do
echo "fga4postgres initdb: applying $f"
psql -v ON_ERROR_STOP=1 \
psql -v ON_ERROR_STOP=1 --single-transaction \
--username "$POSTGRES_USER" --dbname "$POSTGRES_DB" -f "$f"
done
35 changes: 29 additions & 6 deletions docs/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,33 +12,56 @@ PostgreSQL 18 is the floor: the engine uses the native
## Plain SQL (works everywhere)

Download `fga4postgres--<version>.sql` from a release (verify
against `SHA256SUMS`) and run it:
against `SHA256SUMS`) and run it in one transaction:

```sh
psql -v ON_ERROR_STOP=1 -f fga4postgres--<version>.sql "$DB_URL"
psql -v ON_ERROR_STOP=1 -1 -f fga4postgres--<version>.sql "$DB_URL"
```

That single file bundles the pinned cel4postgres release and the
whole engine. If the database already runs the pinned
cel4postgres, use `fga4postgres-engine--<version>.sql` instead —
it contains only the engine and expects schema `cel` to exist.

Each file comes in two shapes. The plain file contains no
transaction control, so it runs inside a transaction someone else
opens: psql's `-1` (`--single-transaction`), a migration tool's, or
pg_tle's `CREATE EXTENSION`. Its `-tx` twin
(`fga4postgres-tx--<version>.sql`,
`fga4postgres-engine-tx--<version>.sql`) wraps the same content in
exactly one `BEGIN;` and one `COMMIT;`, for a bare `psql -f` that
should still be all or nothing.

Every script is idempotent; re-running the installer against a
live database is the upgrade path.

From a checkout instead of a release:

```sh
psql -v ON_ERROR_STOP=1 -f vendor/cel4postgres--*.sql "$DB_URL"
psql -v ON_ERROR_STOP=1 -1 -f vendor/cel4postgres--*.sql "$DB_URL"
for f in sql/*.sql; do
psql -v ON_ERROR_STOP=1 -f "$f" "$DB_URL"
psql -v ON_ERROR_STOP=1 -1 -f "$f" "$DB_URL"
done
```

## Inside a migration tool

Migration tools (Flyway, Liquibase, Kysely, Alembic, Rails, …)
usually run each migration inside a transaction they own. Embed the
plain `fga4postgres--<version>.sql` as the body of one migration
and execute it as a single multi-statement string, without bind
parameters — the file is many statements with `$$`-quoted function
bodies, which PostgreSQL accepts only over the simple query
protocol. The whole install then commits or rolls back with the
migration. Do not embed a `-tx` file: its `COMMIT;` would end the
tool's transaction halfway through.

## pg_tle (RDS, Aurora, and anywhere pg_tle is allowed)

Wrap the release artifact into a `pgtle.install_extension` call
and install it as a real extension:
Wrap the plain release artifact (not a `-tx` one: transaction
control is not allowed inside `CREATE EXTENSION`, and the script
refuses it) into a `pgtle.install_extension` call and install it
as a real extension:

```sh
./scripts/pgtle-wrap.sh fga4postgres <version> \
Expand Down
7 changes: 4 additions & 3 deletions internal/bench/loader.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,10 @@ func installFiles(root string) ([]string, error) {
return append(cel, engine...), nil
}

// execFile runs one install script. The scripts hold their own
// BEGIN/COMMIT and multiple statements, so this goes through the
// simple protocol.
// execFile runs one install script. A script is many statements,
// so this goes through the simple protocol, which also runs the
// whole string as one implicit transaction: the scripts open none
// of their own, and a failure leaves the file unapplied.
func execFile(
ctx context.Context, pool *pgxpool.Pool, path string,
) error {
Expand Down
49 changes: 36 additions & 13 deletions scripts/build-release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,21 @@
# than keeping a copy that could drift (CLAUDE.md: no version
# copies).
#
# Two shapes, because plain SQL scripts and pg_tle are both
# first-class distribution channels:
# the all-in bundle inlines the vendored cel4postgres release
# so one `psql -f` installs everything on a bare database; the
# engine-only file serves databases that already run the pinned
# cel4postgres. Plain concatenation is safe because every script
# opens and closes its own transaction. Install with
# ON_ERROR_STOP so a failure stops between them:
# Two bundles, because plain SQL scripts and pg_tle are both
# first-class distribution channels: the all-in bundle inlines the
# vendored cel4postgres release so one file installs everything on a
# bare database; the engine-only file serves databases that already
# run the pinned cel4postgres.
#
# psql -v ON_ERROR_STOP=1 -f fga4postgres--<version>.sql
# Each comes in two shapes. The plain file carries no transaction
# control, so it runs inside whatever transaction the caller already
# holds -- a migration tool's, pg_tle's CREATE EXTENSION, or psql's
# --single-transaction:
#
# psql -v ON_ERROR_STOP=1 -1 -f fga4postgres--<version>.sql
#
# The -tx file is the same bundle between exactly one BEGIN; and one
# COMMIT;, for a plain `psql -f` that should still be all or nothing.

set -eu

Expand All @@ -40,27 +45,45 @@ case $(printf '%s\n' "$vendored" | wc -l) in
;;
esac

# Neither the sources nor the vendored cel4postgres bundle may open
# or close a transaction; that is what lets the plain artifacts run
# inside a caller's transaction. A BEGIN; or COMMIT; line would
# silently commit that transaction halfway, so refuse to build
# rather than ship it.
# shellcheck disable=SC2086
if grep -n -x -e 'BEGIN;' -e 'COMMIT;' "$vendored" sql/*.sql; then
echo "install scripts must not open or close transactions" >&2
exit 1
fi

rm -rf dist
mkdir -p dist

# Concatenate the named files, each behind a banner naming its
# source, so an error line in a bundle is traceable to a script.
# Writes <name>--<version>.sql and its <name>-tx--<version>.sql
# twin.
bundle() {
out=$1
name=$1
shift
for f in "$@"; do
printf -- '-- ---- %s ----\n\n' "$f"
cat "$f"
printf '\n'
done >"dist/$out"
done >"dist/$name--$version.sql"
{
printf 'BEGIN;\n\n'
cat "dist/$name--$version.sql"
printf 'COMMIT;\n'
} >"dist/$name-tx--$version.sql"
}

engine=$(ls sql/*.sql)

# shellcheck disable=SC2086
bundle "fga4postgres--$version.sql" "$vendored" $engine
bundle fga4postgres "$vendored" $engine
# shellcheck disable=SC2086
bundle "fga4postgres-engine--$version.sql" $engine
bundle fga4postgres-engine $engine

(cd dist && sha256sum -- *.sql >SHA256SUMS)

Expand Down
15 changes: 10 additions & 5 deletions scripts/pgtle-wrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,9 @@
#
# pgtle-wrap.sh <extname> <version> <artifact.sql>... > out.sql
#
# The artifacts run unchanged except for one transformation: their
# top-level BEGIN;/COMMIT; lines are dropped, because a pg_tle
# script executes inside CREATE EXTENSION's own transaction, where
# transaction control is not allowed.
# The artifacts run unchanged. Pass the plain ones, never a -tx
# twin: a pg_tle script executes inside CREATE EXTENSION's own
# transaction, where transaction control is not allowed.

set -eu

Expand All @@ -21,7 +20,13 @@ tag='_fga_pgtle_'
tmp=$(mktemp)
trap 'rm -f "$tmp"' EXIT

cat -- "$@" | grep -v -x -e 'BEGIN;' -e 'COMMIT;' >"$tmp"
cat -- "$@" >"$tmp"

if grep -q -x -e 'BEGIN;' -e 'COMMIT;' "$tmp"; then
echo "input controls transactions; pass the plain artifacts," \
"not the -tx ones" >&2
exit 1
fi

# The bundle becomes one dollar-quoted literal; the quoting breaks
# if its tag ever appears in the content, so refuse instead of
Expand Down
12 changes: 0 additions & 12 deletions scripts/release-engine-initdb.sh

This file was deleted.

Loading
Loading