Repository navigation
Fast reads: compiled relations and list caps (0.1.0) - #6
Merged
Merged
Conversation
Timings vary between runs, so a performance regression in the resolver cannot fail a test on them. A call count can: whether list_objects evaluates a condition per candidate, or whether a deny reads usersets the model cannot produce, is a number that does not move with machine load. The helper reads pg_stat_xact_user_functions before and after the statement, inside one transaction, and reports the difference. Read once, that view is not scoped to the transaction: until the backend flushes its counters it still carries earlier transactions' calls, so on a pooled connection each count would include the previous test's.
Every BFS level filtered its candidates against a text[] of every node seen so far with "<> ALL(seen)", which is a linear scan per candidate: quadratic in the size of the search, and the bulk of a list_objects on a 5,000-object hierarchy. The seen set is now kept as _lo_node values and filtered with NOT IN, which the planner runs as a hashed subplan. The condition evaluator also ran for every candidate, because a LEFT JOIN LATERAL calls its function before the ON clause discards the row. Unconditioned rows now bypass it entirely; only rows that carry a condition reach _eval_condition. A cost test pins that at zero calls for an unconditioned model, counted from the function statistics rather than timed. On a synthetic 5,000-environment hierarchy, a platform admin's capped list went from ~640 ms to ~105 ms locally. Answers do not change; the list_objects corpus and imported runners prove it.
Both list functions computed their whole answer and truncated it to 1,000 at the end. list_objects did stop its BFS after the level that crossed the cap, but then sent every candidate needing confirmation through a forward check, and list_users folded every row into its result set, re-deduplicating the whole set on each fold. A capped request now stops as soon as the cap is reached: confirmation ends at the cap, and a list_users node reached only through unions stops reading rows and children once it holds the cap. Operands of an intersection or exclusion are still computed in full, since the operation can remove users. Upstream also stops collecting at the cap, so a condition error in a row never reached is not raised there either. The BFS also counted unconfirmed candidates toward the cap. When 1,000 of them arrived on one level and all failed their forward check, the search ended before reaching objects one level further out and answered nothing where upstream answers those objects. A new differential probe pins that case on both sides; the cap now counts confirmed objects only. The cost tests count calls instead of timing: 5,005 candidates needing confirmation make exactly 1,000 confirmations, and 5,005 direct viewers fold exactly 1,000 users (5,005 before; the test ran 15.9 s before and 1.5 s after, setup included).
Upstream caps ListObjects and ListUsers at 1,000 by server config, not by the API, and an operator can raise the cap or lift it with 0. The engine hard-coded 1,000 in both functions, so a database had no way to match a server configured otherwise. fga.setting holds the two values, one row each, read through fga._setting_int. It is database-wide rather than per store because upstream's config is per server: one install, one configuration. A CHECK refuses negatives, and the defaults are inserted with ON CONFLICT DO NOTHING so re-running the installer, which is the upgrade path, never resets a value an operator chose. A missing row raises instead of falling back, so deleting it cannot silently lift a cap. The test changes the settings only inside transactions it rolls back, making the list call in the same transaction, because other tests rely on the default while it runs.
Upstream's StreamedListObjects ignores the max-results setting, so
it is the way to list more objects than the cap allows. The engine
had no counterpart: the test adapter replayed the unary, capped
answer as a stream, which differed from upstream as soon as more
than 1,000 objects were accessible. A new differential probe shows
it: 1,005 accessible docs stream as 1,005 at the oracle.
The search moves into fga._list_objects with the cap as an argument,
and both public functions call it, so validation order, error codes
and answers stay identical by construction rather than by keeping
two copies in step. The streamed function returns one jsonb row per
object in the shape of upstream's stream message, {"object": ...},
which the adapter now reads row by row instead of replaying the
unary result.
It is uncapped, not incremental: PL/pgSQL produces the whole answer
before the first row reaches the caller, so a condition error still
fails the call where upstream may have streamed some objects first.
A store opted in with enable_compiled_relations gets three generated SQL functions per relation in a schema the application owns, and every model write regenerates them in the same transaction. Stores that do not opt in, including every conformance store, behave exactly as before. The registry records each generated function as a regprocedure, so disable and delete_store drop exactly what the engine created and never a function the application keeps in the same schema. The schema must already exist: the engine never creates schemas, which keeps ownership and grants with the application and turns a typo into an error instead of an empty schema. Every relation starts on the delegated strategy, which keeps the generated signature and calls the generic resolver pinned to the model the function was compiled from. Faster strategies replace it relation by relation.
A relation built only from direct grants, wildcards, usersets,
computed usersets and tuple-to-userset, with no cycle, set operation
or condition on its path, now compiles to a UNION of fixed joins over
fga.tuple. Each arm that reaches another relation calls that
relation's generated function, and PostgreSQL inlines the whole tree
into the caller's query, which is what turns a list page into one
planned query instead of a resolver loop. Anything else stays
delegated, with the reason in the registry, so no answer depends on
an approximation.
Names follow the readable <type>__<relation>__<kind> form only when
both names are plain lower-case words joined by single underscores;
otherwise a sanitised prefix and a hash of the exact names keep every
function distinct within 63 bytes. Excluding leading and trailing
underscores as well as "__" closes a meeting the "__" rule alone
leaves open ("a_" + "b" and "a" + "_b"), and the generator still
refuses any collision instead of trusting identifier truncation.
The bodies carry no SET clause, because one stops inlining, so every
table, function, type and operator is schema-qualified; a decoy tuple
table and always-true uuid operators ahead of pg_catalog do not move
an answer. Functions that survive a model change are replaced in
place, keeping their oid for plans that inlined them.
The generated functions must answer exactly what check, list_objects and list_users answer, and the conformance corpora are the widest statement of that the project has. Each corpus case now gains a third subtest, compiled, whose engine side opts its store in and answers every assertion through the generated functions, compared with both the corpus expectation and the upstream oracle. The generated functions take uuids rather than request strings, so a request the public API refuses during validation has no generated counterpart; the compiled client hands it to the generic API, which must refuse it, and reports that it did. A routable request the generic API would answer is an error, never a quiet fallback. Relations the generator leaves delegated, assertions carrying contextual tuples, and list_users answers containing a wildcard (which need a registered subject source to become a final list) are all named through the skip register with their reason, so the coverage the compiled strategies do not have yet prints on every run instead of hiding behind green delegated answers.
A denied check visits every node on every path, and each direct node paid for a wildcard probe and a userset read even when the relation admits neither. No such row can exist: stored rows are filtered by the type restrictions and contextual tuples are refused without a matching one, so the restrictions decide up front whether either read can return anything. One lookup of them now replaces two read calls per node on models without usersets or wildcards. The count test pins the skip in both directions, so a relation that does admit usersets or wildcards still reads them. The deep-deny benchmark is the four-level hierarchy the timings were taken on.
_check_ttu looked up model_relation once per linked parent to skip a parent whose type lacks the computed relation. The answer depends on the parent's type only, so it is read once per call and applied as a filter on the tupleset read. Filtering earlier changes nothing a caller sees: rows are grouped by parent type, so a skipped type is always a whole group, and a skipped group never held an error.
A direct child raises when its relation admits conditioned tuples or userset dispatch, and union and intersection must hold that error until a sibling decides. Nothing in the corpus exercised it: running every direct child of a union outside its exception block left all check suites green. This probe compares the engine with the oracle on exactly those children, so removing the subtransaction around one that can raise now fails.
A generated __objects is the set the application pages in its own query, so it mirrors list_objects without the result cap. The delegated strategy answered through fga.list_objects and silently returned the first 1,000 objects of a larger set: a page past that point would simply be missing rows. It now reads streamed_list_objects, the uncapped entry point.
streamed_list_objects is a reader entry point and the compiled relations opt-in belongs to the writer, so the grants template names them. A missing grant fails nothing on its own: PostgreSQL grants EXECUTE to PUBLIC by default, so the template could drift silently until a consumer hardens their database and finds a role locked out. CI therefore asserts that every public fga function appears in the template, rather than probing one call. The caps and the streamed function are documented where installers look, and streamed list-objects leaves the exclusion list.
Consumers catch these SQLSTATEs, so an undocumented one is a contract change nobody can see. The test reads every 'YFnnn' literal in sql/ and fails when docs/ERRORS.md has no row for it; the frozen-scheme rule in that file was until now only prose. Compiled relations reuse existing codes, so the file gains a table saying which conditions raise which code there.
Generated functions answer the API's user:* question through an explicit p_any_subject flag and refuse the nil uuid as an id, the way the public API does. Nil is the engine's internal wildcard sentinel: a function that accepted it would let a caller with an unset uuid ask, and be granted, the wildcard question. The test pins the refusal to the public API's own error, refuses the flag together with an id, and compares the flag's answers with the oracle's check of user:* across a direct wildcard, one inherited through a tuple-to-userset, and a delegated exclusion.
The compiled relations' delegated __subjects reads list_users, and list_users stops at list_users_max_results (default 1,000). That cap belongs to the upstream-shaped API; __subjects is a set the application filters in its own query, where a silently truncated answer drops rows. fga._list_users takes the cap as a parameter (0 = unlimited), the same split _list_objects already has, and fga.list_users becomes a wrapper passing the setting, so its signature and answers do not change.
__subjects promises concrete subjects: wildcards expanded over the store's registered subject source minus exclusions, usersets expanded to members. The delegated path missed that in two ways. It read the capped fga.list_users, so an object with more than 1,000 subjects lost the rest; it now reads the uncapped _list_users. And it returned list_users' concrete users and the expanded wildcard as two result sets, so a user granted both ways came back twice; the two are now one UNION. The delegated path keeps list_users' search and post-processes its upstream shape: object users are their ids, userset users (when the filter names a relation) their userset ids, and a typed wildcard is replaced by the source rows check allows, since upstream's "user:*" hides who is excluded from it. A relation whose __subjects can meet a wildcard of a type with no registered source raises when called; the registry reason now says so from the model write on, for flattened and delegated relations alike, and the note disappears once a source is registered.
The compiled list_users corpus mode skipped the 36 assertions whose upstream answer holds user:*, because __subjects returns the final list and the corpus has none to compare with. Each such assertion now registers a subject source for the filter type (every id the stages and the request mention, plus ids nothing grants to directly) and requires __subjects to equal the source subjects the engine's check allows, one at a time. The oracle still answers the corpus expectation as before. The registry's missing-source note is stripped from the delegated report, since corpus stores register no source and the note would split every delegated reason in the skip register.
Applications prepare page queries and build views over generated functions, and PostgreSQL inlines the function body into both. A model write has to reach them. The test prepares a forced generic plan and a view over doc#viewer's __objects, moves the relation through two rewrites and from the flattened to the delegated strategy, and requires both to answer for the new model at once. Measured while writing it: a generator that dropped and re-created functions would still pass the prepared statement, which PostgreSQL re-analyses by name after the invalidation, but it fails every model write once a view depends on a function. Replacing in place is what keeps the view working.
The planner walks the relation graph to find cycles and dispatch depth; the recursive emitter needs the same edges to find the sub-graph a relation reaches. One function defines them, so the two can never disagree about what a relation can reach.
Relations on a cycle of the relation graph (same-type parent chains, nested groups, a#m -> b#m -> a#m), relations reaching one, and chains deep enough to meet check's depth limit all delegated to the generic resolver. They now compile to a WITH RECURSIVE walk over the tuple graph; UNION deduplicates its nodes, so cyclic data ends the walk. The walk covers the whole sub-graph the relation reaches instead of calling the reached relations' generated functions: __check has to count dispatches from its own root, and a callee's __check would start counting again at zero. Each kind spends depth like its counterpart. __objects charges none, like list_objects. __check records each node's dispatch count: a grant within 25 dispatches is check's true, since check follows every simple path of that length. check raises only from a simple path of 25 dispatches whose last node has a row to dispatch; that needs a walk row at 26 and 26 distinct nodes, so without both a false is exact, and with both the call goes to fga.check, whose error it then is. __subjects does the same at list_users' boundary, which refuses on entering a node 25 dispatches away. Corpus delegation in the check replay drops from 208 relations to 142; what remains is set operations and conditions.
Compiled relations exist so an application can page authorised rows in its own query, which only pays off when PostgreSQL inlines the generated function into that query. A SET clause on a function blocks inlining, so the rule that every function pins its search_path cannot hold for them. They keep the protection the rule gives by another route: every name and operator in a generated body is schema-qualified, and two tests fail if either half slips. The scope list also gains streamed list-objects, the result-cap settings and compiled relations, which this release adds.
Compiled relations need a measured answer to two questions the existing scenarios cannot ask: how much faster the generated __check and __objects are than the generic engine on a model of role ladders and parent chains, and what a consumer's own page filtered through __objects costs as the subject's visible set grows. The tenant scenario is the synthetic four-level hierarchy those questions were first prototyped on, with 5,000 leaves at 100k (the PR run) and 100,000 at 1m. Its cases are its own list rather than the standard thirteen: the generic check and list_objects on a store that is not opted in, then the same query streams through __check, through fga.check on the opted-in store (the case that moves once fga.check dispatches to compiled functions), through __objects, and as a name-sorted page of 50 rows. Each compiled case opts the store in for its own duration, so a generic case never measures an opted-in store, and a run killed mid-case is undone at the start of the next. The consumer table is loaded with the fixture and analyzed with fga.tuple, since every page shape needs those statistics.
The compiled page was designed against a prototype that put a platform admin's page at ~6.6 ms for 5,000 leaves and ~108 ms for 100,000. The engine's generated functions are what ships, so the numbers a reader needs are theirs: they sit well outside that envelope today, and the planning/execution split shows why (planning alone is ~16 ms per call, and the admin's set is deduplicated by several nested unions). Writing it down with the machine context keeps the gap visible until the generator closes it, rather than letting the prototype's figures stand for the product. The workflow comment carries the tenant cases' share of the PR budget, since the timeout is sized by that arithmetic.
Intersection and exclusion were the largest delegated group, and a conditioned grant or a single contextual tuple sent a relation, or a call, back to the generic resolver. Set operations now compile to INTERSECT / EXCEPT for __objects and AND / AND NOT for __check. They are exact only over operands that answer plainly true or false, so the plan keeps a set operation delegated when it reaches a cycle (a cycled false on the subtract side must not read as "not excluded") or a relation that may raise (an operand error a sibling would swallow). __subjects mirrors list_users' algebra: concrete candidates plus a wildcard flag, then filtered by the relation's own check, so a wildcard on either side needs no subject source unless the answer really is "everyone". Conditioned grants on a relation's own plain or wildcard restrictions compile too. Conditions go through _eval_condition, so the two paths cannot disagree on one; the list functions raise the first condition error they meet, as the list APIs do, and __check holds errors until nothing grants, as check does. Conditioned usersets and tuplesets stay delegated: which of their errors the list APIs report depends on the search's visiting order. The plan composes them with the recursive strategy: the recursive walk reads tuples itself and knows only plain rewrites, so a cyclic or deep relation reaching a set operation or a conditioned relation delegates, and a set operation or conditioned relation on or reaching a cycle delegates rather than going recursive. The leftover helpers nothing calls (_compiled_no_ctuples, _compiled_rows) are gone. Contextual tuples are read by the flattened, conditioned and set-op bodies as a second source beside fga.tuple, validated with each API's own error codes, with check's exact-key replacement. A recursive body still hands such a call to the generic resolver: its walk joins fga.tuple on the previous step's node, and a second source there would cost the index path every step relies on. Three shapes keep the common call without them as fast as before: callee-calling arms drive a LATERAL union so the callee is inlined once and the index path survives; a check's contextual half is a function call rather than a subplan, since a non-inlined body starts every subplan on every call; and the contextual source is costed as the handful of rows it is, or the plan cache re-planned each nested check.
The compiled replay answered only stored-tuple runs, so nothing showed the generated functions reading contextual tuples, and each assertion carrying some was reported as handed to the delegated path. A fourth mode, compiledCtxTuples, replays upstream's second run — the stage tuples handed as contextual tuples — through the generated functions, the wildcard list_users path included (it now hands the stage tuples to both the reference checks and __subjects). The skip register follows what is still not compiled in that mode: a recursive relation answers calls carrying contextual tuples through the generic resolver, and each one used is named. A relation reaching one it cannot compose with is now described as that, since the reached relation may itself be compiled.
The INSTALL walkthrough is the contract a consumer copies, so a test executes its SQL blocks in order and asserts what the prose says they answer. The mistake it is built to catch is the one a reader cannot see: a generated function called in the select list instead of FROM returns the same rows but computes the whole set first, so the test also requires every __objects and __subjects example to inline. README and CONFORMANCE describe the surface as additive and say what the corpus proves about it: compiled answers equal the engine's, and list_users keeps upstream's shape.
An opted-in store compiles every relation of its latest model, but the public entry points still resolved generically, so the deny cost the generated functions remove stayed in fga.check. check, batch_check (through check), list_objects and streamed_list_objects now look the relation up in fga.compiled_relation after their own request validation and call the registered function. The registry is keyed by model id, so a request pinned to an older model finds no row and stays generic. Delegated rows are skipped: their functions call fga.check. That is not the only hand-off, though: a recursive function given contextual tuples or nearing the depth limit also calls fga.check or streamed_list_objects for its own relation. The dispatch helpers therefore set fga._dispatching for their duration, and every entry point answers generically while it is on; without it such a call recursed until the stack limit. list_objects keeps its cap on top of the uncapped __objects. Past the cap the generic search tolerates condition errors where a generated body refuses on the first it meets, so a capped call the function refuses is answered by the generic search instead. list_users is not dispatched (its response shape differs from __subjects); the test asserts it calls no generated function.
The compiled corpus mode answered every assertion through the generated functions only, so nothing replayed fga.check and fga.list_objects as they now behave on an opted-in store. Check and ListObjects also ask the public API on the same store and refuse to answer when the two disagree in value or refusal code; flipping the dispatched answer turns 928 compiled and 716 compiledCtxTuples subtests red. A capped list compares by size only, since any 1,000 objects are a valid answer.
The release adds a public surface — compiled relations, the streamed list function and database-wide settings — and changes what an opted-in store needs from the caller's privileges, so it is a minor bump rather than a patch.
A flattened relation composed other relations' generated functions, and PostgreSQL inlines each call separately: a callee reached by several routes was parsed and planned once per route, and every composed UNION added its own dedup. On the tenant bench's environment#can_view that was 80 dedup nodes and ~16-23 ms of planning per page, and at 100k leaves the set passed five nested Sort + Unique steps. The emitter now walks the relation graph at generation time into grant places: a path of tuple-to-userset and userset hops plus every relation reached at its end through computed usersets. Relations sharing a path share one fixed join, so the ladder is one arm per (place, restriction) and the body has a single UNION. Only a set operation reached on the way is still called, since its operands are not plain unions. A check runs as its own statement, so its arms share one EXISTS (every subplan starts on every call). Contextual tuples no longer double the inlined arms: a call carrying them runs the same arms over both sources as a dynamic statement, which costs nothing to plan for the common call that carries none.
The comment claimed replacing in place keeps a cached plan from pointing at a dropped function. Measured, a prepared statement re-resolves the name after a drop as well; what in-place replacement actually protects is everything holding the function by oid, an application's views above all, which a drop would refuse (or need CASCADE for) and so fail the model write.
Two model writes to one opted-in store, each in an open transaction, regenerate the same functions. The second waited on the first's rows and then carried on from a registry it had read before the first committed, so the first model's relations stayed registered, and their functions stayed in the schema, under the second model. Regeneration and drop now take a transaction-scoped advisory lock on the store first; the second write waits, and every statement after the lock reads what the first committed. The key uses the two-int form with a class id hashed from a fixed string, so it cannot meet an application's single-bigint advisory locks and is unlikely to meet its two-int ones.
A generated function that hands a question to the generic resolver did so through the public fga.check / streamed_list_objects. Called directly on an opted-in store, that entry point dispatched straight back to the same generated function, which handed off again: a recursive relation's __check with contextual tuples ran twice and its __objects three times. The hand-off helpers now set fga._dispatching for their own duration, the guard dispatch already sets around a generated call, so the entry point they call answers generically the first time.
An application that drops its schema before deleting the store (or opting out) drops the generated functions with it, while the registry still holds their oids. Cleanup then tried DROP FUNCTION on an oid that names nothing, failed with a syntax error, and left the store opted in; tests that drop their schema first piled such stores up. Cleanup and regeneration now drop only registered functions that still exist.
A generated body is parsed whenever the statement calling it is planned: inlined __objects and __subjects as part of the caller's plan, and __check because the planner tries to inline it before finding the EXISTS that stops it. The dynamic statement a call with contextual tuples runs was a literal in each body, two thirds of its 34 kB on the tenant bench, so every planning paid to parse it. Re-planned per call (a plpgsql EXECUTE, as dispatch does), a check cost 0.47 ms with it and 0.38 ms without. The statements now live in fga.compiled_contextual, written by the same regeneration that writes the functions and keyed like them, and the body names its key. Opt-out and drop remove them with the registry rows.
PostgreSQL never inlines a check body that reads tuples, since the EXISTS stops it, but it learns that only after parsing and analysing the whole body, on every planning of the statement that calls it. With the ladder in one body that analysis is most of a dispatched check: fga.check calls __check through a plpgsql EXECUTE, which plans afresh each time, and a hit cost 0.37 ms there against 0.08 ms from a statement planned once. A check body that cannot be inlined now starts with an empty statement: the inliner refuses a multi-statement body right after the raw parse, before any analysis. The same call costs 0.21 ms; a statement planned once is unchanged. A SET clause would skip even the parse, but generated functions carry none (the project's decision 9).
Cleanup now tolerates a schema the application dropped first, and two model writes to one opted-in store wait for each other instead of leaving the first's relations behind. Both are visible to an application, so the install guide says so where it describes opting out.
Before and after numbers for environment#can_view on the same machine, runs alternated so both sides share its load. The page now plans in ~2 ms instead of 16-23 and sits inside the prototype's envelope at 5,000 leaves. At 100,000 leaves a platform admin's page is still about twice the prototype's, and the difference is measured to be the parent-edge condition test the prototype skipped: a covering index closes it, the engine's current index makes every edge a heap fetch. Checks trade: a miss no longer calls every callee (1.5 ms to 0.25), a direct hit starts the whole plan (0.07 ms to 0.14). The dispatched fga.list_objects rows answer the slowdown dispatch had measured for a few-subject list (25 ms to 2).
Flattening enumerates every path of tuple-to-userset and userset hops a relation reaches, and the count multiplies with the branches along a chain: four parent relations over nine levels is 87,381 places, and a model write on an opted-in store then ran past any reasonable timeout. Bounding the model write is not enough; the queries must stay usable too. Planning tracks places times chain depth: on a deep binary chain a statement calling __objects planned in 8 ms at 15 places, 95 ms at 63, 0.47 s at 127 and 2.8 s at 255, where a dispatched check took about 3 s. The cap is 64. The plan now counts a relation's places before flattening it. Past the cap it compiles to the recursive walk, which reaches the same grants without listing paths, or is delegated with the reason when it reaches a set operation or a condition. The count walks lazily and stops once it has seen more rows than the cap times the most relations of a type, so sizing a pathological relation costs no more than sizing one at the cap. The emitter refuses a cut-short list rather than emit part of a relation. The upstream corpus needs at most 24 places, so no corpus relation changes strategy. The installer also drops _compiled_no_inline before defining its helpers: the change after this one stops using it, and the drop keeps an upgrade from leaving it behind.
A __check never inlines: it is scalar and reads tuples. Without a SET clause PostgreSQL still parses its body on every planning of the statement calling it, only to find that out, and dispatch from fga.check calls it through a plpgsql EXECUTE, which plans on every call. On a four-level ladder fixture an EXECUTE of __check went from 0.56 to 0.26 ms, and fga.check from 0.85 to 0.47 ms; a check from a statement planned once, or called per row, is unchanged. __check now carries SET search_path = fga, pg_temp; __objects and __subjects stay SET-free, since they must inline, and every body stays fully qualified. The shape tests assert the rule per kind, across every strategy. The empty statement a check body started with, which made the planner give up inlining after the raw parse, no longer measures anything once the SET clause stops the planner before the parse, so it is gone.
An exact parent edge must be an unconditioned row, and condition_name was not in tuple_reverse_idx, so a broad compiled page made one heap fetch per object it listed. The reverse index now INCLUDEs condition_name and is named tuple_reverse_cond_idx; the installer builds it and drops the old one, so re-running it converges from any earlier state. No CONCURRENTLY: the bundle has to keep running inside a transaction (migration tools, psql -1, the -tx artifact, pg_tle), so the rebuild blocks writes to fga.tuple while it runs. The tests replay the installer over every starting state in a rolled back transaction, and check that a page over a broad ladder reads its parent edges from the index alone.
Record what INCLUDE (condition_name) costs and buys, measured on two databases with identical data, one per index, alternated over five rounds: a platform admin's page at 100,000 leaves drops from 241 to ~139 ms, narrow subjects do not move, and a 100-tuple write plus delete stays within noise. The index grows 8-10% on fixtures without conditions. INSTALL.md says what the upgrade costs: re-running the installer rebuilds the index inside its transaction, blocking tuple writes, then reads once the old index is dropped, until it commits (~1.7 s per million tuples here). No CONCURRENTLY path is offered yet.
The notes still said a check body starts with an empty statement to stop the inlining attempt. That statement is gone: __check now carries SET search_path, which stops the planner before it parses the body, and the dispatched fga.check on the ladder fixture went from 0.85-0.90 ms to 0.47-0.53 ms.
The cycle test compared each __objects answer with streamed_list_objects on the same opted-in store, which dispatches back to the generated function: it compared the function with itself. The reference query now runs with dispatch turned off, so the generic resolver answers it.
Re-running the installer is the upgrade path, and this release changes an index on the way, yet the smoke only ever installed each artifact into an empty database. Each artifact now installs a second time over itself, so a script that stops being idempotent fails the release instead of a consumer's upgrade.
The exemption from the pinned search_path exists so set-returning generated functions can be inlined into the caller's query. __check returns a boolean from a subquery, which PostgreSQL never inlines, so the exemption bought it nothing and cost a re-parse of its body on every planning: an EXECUTE-dispatched check spent 0.21 ms in the function without the clause and 0.087 ms with it. The exemption now names only __objects and __subjects.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.1.0: fast reads. fga4postgres becomes fast enough to be an
application's only source of authorization truth, for point checks and
for list pages the application sorts and pages itself. It writes nothing
beyond the tuples (no materialised closure).
What changes
List functions (phase 0)
list_objectsdedups with a hashed anti-join instead of a growingarray, and evaluates a condition only for tuples that carry one. A
5,000-leaf platform-admin list went from ~640 ms to ~105–135 ms.
fixes a bug: candidates still waiting for their forward check counted
toward the cap, so
list_objectscould answer[]where upstreamreturns objects (
TestProbeListObjectsCapCountsConfirmed).fga.setting:list_objects_max_results,list_users_max_results; default 1000,0 = unlimited).
fga.streamed_list_objects: uncapped, one row per object,differential against upstream
StreamedListObjects. It is no longerin the exclusions.
parent-type lookup is hoisted. The measured gain is 5–8%, smaller than
estimated: removing the subtransactions was measured slower and
reverted. The fast deny is the opted-in path below.
Compiled relations (phase 1). A store opts in with
fga.enable_compiled_relations(store, target_schema, subject_sources).From then on, every model write regenerates, in the same transaction,
three plain
LANGUAGE sqlfunctions per relation in a schema theapplication owns:
__objects,__subjectsand__check.id IN (SELECT x FROM app_authz.document__viewer__objects('user', $1) x) ORDER BY title LIMIT 50plans as one query.set operations, conditioned, and delegated. Delegated is used where no
other strategy is exact; it calls the generic resolver, and the
registry records its reason.
<type>__<relation>__<kind>, or a hashed form when aname would exceed 63 bytes or collide.
__objectsand__subjectscarry noSETclause, because thatblocks inlining;
__check, which never inlines, keeps it. Everytable, function, type and operator in a generated body is
schema-qualified.
CLAUDE.mddecision 9 is amended in its owncommit, and tests enforce it, using a decoy table and a decoy
=operator.
__subjectsreturns the final list: wildcards are expanded over aregistered subject table, exclusions applied and groups expanded.
it, a concurrent write could leave the wrong functions registered.
Dispatch (phase 2). On an opted-in store,
fga.check,batch_check,list_objectsandstreamed_list_objectsanswer throughthe compiled functions when the request targets the latest model.
list_usersdoes not, because its response keeps upstream's shape.How it is proven
modes,
compiledandcompiledCtxTuples. Every assertion is answeredby the generated function and through the dispatched public API, and
compared with the corpus and the oracle.
existed, 222 corpus relations were delegated; now 60 are.
mutation turns it red.
pg_stat_xact_user_functionsdeltas) replacetimings where a cost regression should fail a test.
TestInstallDocExamplesruns the INSTALL walkthrough as written,asserts the answers the prose states, and requires the example page
queries to inline.
TestErrorCodesDocumentedrequires adocs/ERRORS.mdrow for everyraised code. Compiled relations add no new codes.
EXECUTE is granted to PUBLIC by default, so a missing grant would not
otherwise fail a call.
Numbers
Tenant bench, synthetic 4-level model, shared laptop, p50 medians;
full tables in
docs/BENCHMARKS.md:fga.checkdeny, generic → opted infga.list_objectsfew grants, opted inThe covering reverse index grows the index ~9% and leaves write cost
unchanged within noise (pgbench 100-tuple write+delete: 25.1 → 24.9
ms at 100k, 26.9 → 26.8 ms at 1m).
Decisions taken in review
fga.checkandlist_objectsneedUSAGEon the target schema. A role without itgets a clear "permission denied" instead of a silent fallback to the
slow path. INSTALL documents the one-time grant, and the doc test
runs it.
tuple_reverse_cond_idxaddsINCLUDE (condition_name)and replacestuple_reverse_idx.covers old, new, both and neither.
CONCURRENTLY, so the bundle still runsinside a transaction (migration tools,
psql -1,-tx, pg_tle).fga.tuplewhile it builds, about1.7 s per million tuples; INSTALL has the note.
__checkkeepsSET search_path: it never inlines, and theclause saves a re-parse on every planning (a dispatched check went
from 0.85–0.90 to 0.47–0.53 ms on the ladder fixture). Only
__objectsand__subjectsstay without the clause;CLAUDE.mdis amended in its own commit.
compiles to the recursive walk, or is delegated with the reason.
Measured on a deep chain, planning cost was 8 ms at 15 places,
95 ms at 63 and 2.8 s at 255. The corpus needs at most 24.
Every artifact now also reinstalls over itself in the smoke run,
because re-running the installer is the upgrade path.
Follow-ups, not in this release
_read_exact/_read_usersetsnow dominate the per-node cost (~28 µs per call).Inlining them and merging
_check_node's two lookups should bemeasured first.