Rest api + docs + namespaces by vshekar · Pull Request #6 · NSLS2/recap

Shekar V (vshekar) · 2026-09-16T16:19:21Z

Created infrastructure for single user REST API
Updated docs to use Great docs
Use namespaces similar to Tiled

Replace global template and run identities with namespace-scoped constraints. Keep Campaign linkage for process runs until Task 8.

Remove Campaign domain APIs after validating migrated namespace ownership. Rebuild process_run on SQLite, preserve existing IDs and assignments, then drop campaign data structures.

Require exact Apikey credentials and compare them in constant time. Redact configured secrets from representations and authentication failures.

Generate trusted request IDs and sanitize failure responses so caller input and internal details cannot cross API boundaries. Restrict mutation audit records to identifiers and closed reason codes.

Persist actor-scoped command results and sanitized mutation audits in the same transaction as domain changes. Add revision CAS support and command infrastructure migration.

…and path

…ated resources

Copilot

🟡 Changes recommended

Critical and moderate authorization, lifecycle, migration, and REST error-handling issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a single-user REST API foundation with authentication, namespace-aware storage and authorization, lifecycle/revision support, migrations, tests, and Great Docs updates.

Changes:

Added REST server, client, query, command, and authentication infrastructure.
Added namespaces, authorization, auditing, idempotency, lifecycle handling, and process-run lineage.
Expanded tests and migrated documentation tooling and content.

File summaries

File	Reviewed changes
`recap/utils/namespace.py`	Namespace path and ancestry helpers
`recap/utils/migrations.py`	Migration configuration
`recap/utils/general.py`	General utility exports and conversions
`recap/tests/transport_factories.py`	Transport test factories
`recap/tests/test_strawberry_types.py`	Strawberry type tests
`recap/tests/test_set_campaign_perf.py`	Campaign performance tests
`recap/tests/test_server_config.py`	Server configuration tests
`recap/tests/test_schema_registry.py`	Schema registry tests
`recap/tests/test_revisions.py`	Revision behavior tests
`recap/tests/test_rest_resource_templates.py`	Resource-template REST tests
`recap/tests/test_rest_process_templates.py`	Process-template REST tests
`recap/tests/test_rest_lifecycle.py`	REST lifecycle tests
`recap/tests/test_resource_tree_perf.py`	Resource-tree performance tests
`recap/tests/test_resource_freeze.py`	Resource freeze tests
`recap/tests/test_resolvers.py`	Resolver tests
`recap/tests/test_request_backend_scope.py`	Request backend scope tests
`recap/tests/test_query_loaders.py`	Query loader tests
`recap/tests/test_public_errors.py`	Public error tests
`recap/tests/test_public_api.py`	Public API tests
`recap/tests/test_protocol_split.py`	Protocol separation tests
`recap/tests/test_process_run_commands.py`	Process-run command tests
`recap/tests/test_platemate_parity.py`	PlateMate parity tests
`recap/tests/test_namespace_path.py`	Namespace path tests
`recap/tests/test_namespace_models.py`	Namespace model tests
`recap/tests/test_lifecycle.py`	Lifecycle tests
`recap/tests/test_include_properties_perf.py`	Property-loading performance tests
`recap/tests/test_graphql_integration.py`	GraphQL integration tests
`recap/tests/test_get_resource_perf.py`	Resource retrieval performance tests
`recap/tests/test_field_predicates.py`	Field predicate tests
`recap/tests/test_experiment.py`	Experiment tests
`recap/tests/test_exceptions.py`	Exception tests
`recap/tests/test_end_to_end.py`	End-to-end tests
`recap/tests/test_database_config.py`	Database configuration tests
`recap/tests/test_container.py`	Container tests
`recap/tests/test_command_models.py`	Command model tests
`recap/tests/test_command_errors.py`	Command error tests
`recap/tests/test_canonical_models.py`	Canonical model tests
`recap/tests/test_campaign_update.py`	Campaign update tests
`recap/tests/test_builder_identity.py`	Builder identity tests
`recap/tests/test_builder_command_count.py`	Builder command-count tests
`recap/tests/test_authentication_models.py`	Authentication model tests
`recap/tests/test_audit.py`	Audit tests
`recap/tests/test_attribute_value.py`	Attribute value tests
`recap/tests/test_array_default_value.py`	Array default-value tests
`recap/tests/test_actions.py`	Action tests
`recap/tests/fixtures/authorization.yml`	Authorization fixture data
`recap/server/strawberry_types.py`	Strawberry API types
`recap/server/strawberry_schema.py`	Strawberry schema assembly
`recap/server/security.py`	API-key security
`recap/server/rest_models.py`	REST request and response models
`recap/server/query_models.py`	Query models
`recap/server/errors.py`	Server errors
`recap/server/error_handlers.py`	Error handlers
`recap/server/dependencies.py`	Server dependencies
`recap/server/config.py`	Server configuration
`recap/server/audit.py`	Server auditing
`recap/server/__main__.py`	Server entry point
`recap/server/__init__.py`	Server package exports
`recap/schemas/namespace.py`	Namespace schemas
`recap/schemas/attribute.py`	Attribute schemas
`recap/schemas/__init__.py`	Schema exports
`recap/lifecycle.py`	Lifecycle definitions
`recap/exporters/registry.py`	Exporter registry
`recap/exporters/protocol.py`	Exporter protocol
`recap/exporters/__init__.py`	Exporter exports
`recap/exceptions.py`	Package exceptions
`recap/dsl/builder_state.py`	DSL builder state
`recap/dsl/attribute_builder.py`	Attribute builder
`recap/dsl/__init__.py`	DSL exports
`recap/db/step.py`	Database step models
`recap/db/namespace.py`	Namespace persistence and hierarchy
`recap/db/migrations/versions/c9d2e8f4a1b7_add_process_run_lineage.py`	Process-run lineage migration
`recap/db/migrations/versions/b47f5e2a9c10_add_command_infrastructure.py`	Command infrastructure migration
`recap/db/migrations/env.py`	Migration environment
`recap/db/engine.py`	Database engine setup
`recap/db/campaign.py`	Campaign persistence
`recap/db/base.py`	Database base definitions
`recap/db/audit.py`	Database auditing
`recap/db/__init__.py`	Database package exports
`recap/commands/models.py`	Command models
`recap/commands/idempotency.py`	Idempotency handling
`recap/commands/errors.py`	Command errors
`recap/commands/context.py`	Command context
`recap/commands/audit.py`	Command auditing
`recap/commands/__init__.py`	Command package exports
`recap/client/permissions.py`	Client permissions
`recap/client/connection_state.py`	Client connection state
`recap/client/__init__.py`	Client package exports
`recap/authorization/scopes.py`	Authorization scopes
`recap/authorization/query.py`	Authorized query filtering
`recap/authorization/__init__.py`	Authorization exports
`recap/authentication/protocols.py`	Authentication protocols
`recap/authentication/models.py`	Authentication models
`recap/authentication/errors.py`	Authentication errors
`recap/authentication/api_key.py`	API-key authentication
`recap/authentication/actors.py`	Authentication actors
`recap/authentication/__init__.py`	Authentication exports
`recap/adapter/query_loaders.py`	Adapter query loading
`pyproject.toml`	Project and documentation configuration
`mkdocs.yml`	Documentation site configuration
`docs/tutorials/resource-examples.qmd`	Resource examples tutorial
`docs/tutorials/index.qmd`	Tutorials index
`docs/tutorials/complete-local-provenance.qmd`	Local provenance tutorial
`docs/resource_examples.md`	Resource examples
`docs/reference/server-configuration.qmd`	Server configuration reference
`docs/index.md`	Documentation index
`docs/how-to/use-revisions-and-idempotency.qmd`	Revisions and idempotency guide
`docs/how-to/use-remote-authentication.qmd`	Remote authentication guide
`docs/how-to/update-and-retire-templates.qmd`	Template update and retirement guide
`docs/how-to/resource-template.qmd`	Resource-template guide
`docs/how-to/quick-start-create-and-store-data-locally.qmd`	Local data quick start
`docs/how-to/query-templates-and-namespaces.qmd`	Template and namespace queries
`docs/how-to/query-resources.qmd`	Resource queries
`docs/how-to/query-process-runs.qmd`	Process-run queries
`docs/how-to/provenance-queries.qmd`	Provenance queries
`docs/how-to/process-template.qmd`	Process-template guide
`docs/how-to/performance.qmd`	Performance guide
`docs/how-to/nested-resource-templates.qmd`	Nested resource-template guide
`docs/how-to/manage-resource-lifecycle.qmd`	Resource lifecycle guide
`docs/how-to/index.qmd`	How-to index
`docs/how-to/handle-errors-and-request-ids.qmd`	Errors and request IDs guide
`docs/how-to/edit-resource-properties.qmd`	Resource property editing guide
`docs/how-to/choose-query-shape-and-loading.qmd`	Query shape and loading guide
`docs/how-to/build-process-runs.qmd`	Process-run construction guide
`docs/how-to/add-runtime-children.qmd`	Runtime children guide
`docs/getting_started/03-process-workflow.qmd`	Process workflow guide
`docs/getting_started/00-orientation.qmd`	Getting started orientation
`docs/explanation/rest-query-architecture.qmd`	REST query architecture
`docs/explanation/resource-loading.qmd`	Resource loading explanation
`docs/explanation/query-model.qmd`	Query model explanation
`docs/explanation/index.qmd`	Explanations index
`docs/explanation/data-organization.qmd`	Data organization explanation
`docs/explanation/core-concepts.qmd`	Core concepts explanation
`docs/explanation/authentication-authorization.qmd`	Authentication and authorization explanation
`docs/explanation/audit-and-errors.qmd`	Auditing and errors explanation
`CONTRIBUTING.rst`	Contribution guidance
`.gitignore`	Ignore rules
`.github/workflows/docs.yml`	Documentation workflow

Review details

Suppressed comments (8)

recap/commands/service.py:1323

Unlike copy_resource, this canonicalization is outside the try block. An invalid destination path therefore escapes as a raw ValueError and the REST endpoint returns a generic 500 instead of the command validation 422 used by the other copy API.
recap/commands/service.py:1759
All denied scope checks are recorded with resource_type="resource", including denied process-template, process-run, and resource-template operations. This misclassifies authorization audit records and prevents consumers from identifying which object type was denied; derive the type from the scope or pass it from each caller.
recap/commands/service.py:468
Suppressing denial auditing for the destination check means a copy rejected for insufficient destination write access produces no denied record; _emit_failure explicitly skips AuthorizationDenied, so only source-side denials are recorded. Do not disable the denial audit for this check.
recap/commands/service.py:275
CreateResource accepts a client-supplied id, but that field is omitted from _CreateResourceFingerprint here. Reusing an idempotency key with the same other fields and a different requested ID therefore replays the first resource instead of reporting a fingerprint conflict, silently ignoring the new identity. Include the requested ID in the fingerprint.
recap/commands/service.py:877
Updating a process template is also authorized through _authorize, which checks namespace:write rather than the separately defined process-template:write scope. This lets namespace-only writers mutate process-template definitions in multi-user mode.
recap/commands/service.py:953
Resource-template creation is guarded by namespace:write through _authorize, even though scopes are separate for namespace and resource-template mutations. A namespace-only writer can therefore create resource templates without resource-template:write. Use the operation-specific scope.
recap/commands/service.py:1046
This update path checks only namespace:write, not the distinct resource-template:write scope. It grants namespace-only principals mutation access to resource-template definitions in multi-user deployments.
recap/db/migrations/versions/c9d2e8f4a1b7_add_process_run_lineage.py:35
The downgrade uses op.create_unique_constraint directly after dropping the index. SQLite does not support ALTER TABLE ADD CONSTRAINT, so downgrading this revision on the SQLite databases used by the project raises NotImplementedError instead of restoring the prior uniqueness constraint. Use batch_alter_table for the downgrade as well.

Files reviewed: 106/243 changed files
Comments generated: 14
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+        stmt = select(Resource).where(Resource.id.in_(subtree_ids)).options(
+            *self._resource_subtree_loaders()
        )
        return list(session.scalars(stmt).unique())


+                if target_status is LifecycleStatus.ACTIVE:
+                    if isinstance(obj, ProcessRun):
+                        obj.finalize(bump_revision=False)
+                    else:
+                        obj.activate(bump_revision=False)
+                else:
+                    obj.archive(bump_revision=False)


+                )
+                if namespace is None:
+                    raise CommandNotFoundError("Namespace not found")
+                template = session.get(ResourceTemplate, template_id)


+            canonical_path = canonicalize_namespace_path(namespace_path)
+        except ValueError as error:
+            raise CommandValidationError(str(error)) from error
+        self._authorize(context, canonical_path, mutation="create_process_template")


+                self._authorize_scope(
+                    context,
+                    template.namespace.path,
+                    Scope.PROCESS_TEMPLATE_READ,
+                    "create_process_run",


+        "process_run",
+        ["namespace_id", "name"],
+        unique=True,
+        sqlite_where=sa.text("copied_from_id IS NULL"),


+        ancestor_paths = namespace_ancestors(path)[:-1]
+        parent_path = ancestor_paths[-1]
+        parent = self._session.scalar(


+            "namespace_id",
+            "name",
+            unique=True,
+            sqlite_where=(copied_from_id.is_(None)),


+    actor: Annotated[RequestActor, Depends(authenticate_request)],
+    namespace_path: str = "",
+) -> list[str]:
+    path = canonicalize_namespace_path(namespace_path)


+        is required; environment variables and direct kwargs override YAML.
        Raises FileNotFoundError if the config file does not exist.


Shekar V (vshekar) added 30 commits July 27, 2026 22:04

chore: ignore local worktrees

ec09d47

feat: add namespace lifecycle primitives

fea45fa

feat: add namespace domain model

a6a3520

feat: migrate campaign data to namespaces

9c1a86e

feat: scope provenance entities by namespace

d9a1071

Replace global template and run identities with namespace-scoped constraints. Keep Campaign linkage for process runs until Task 8.

feat: query within namespace context

7d8d72a

feat: freeze referenced provenance aggregates

9c701c4

feat: copy frozen resources by namespace

acb7b2c

feat: replace campaigns with namespaces

2b91a12

Remove Campaign domain APIs after validating migrated namespace ownership. Rebuild process_run on SQLite, preserve existing IDs and assignments, then drop campaign data structures.

feat: define authentication and scope contracts

38ad077

feat: authenticate requests with API key

a75a757

Require exact Apikey credentials and compare them in constant time. Redact configured secrets from representations and authentication failures.

feat: compile namespace authorization grants

0235dd3

feat: load authorization snapshot generations

6c7bb3d

feat: authorize namespace operations

0457c00

style: modernize auth source type check

843c7f9

feat: add secure request context and audit

f86ab70

Generate trusted request IDs and sanitize failure responses so caller input and internal details cannot cross API boundaries. Restrict mutation audit records to identifiers and closed reason codes.

feat: enforce namespace policy in GraphQL

da892a5

feat: authenticate namespace GraphQL client

ae75422

feat: add request-scoped command spine

41ab642

feat: add idempotent audited commands

a5087d8

Persist actor-scoped command results and sanitized mutation audits in the same transaction as domain changes. Add revision CAS support and command infrastructure migration.

style: satisfy command infrastructure lint

f595930

feat: expose namespace write commands

b133c4c

feat: write process templates atomically

3ff2713

feat: write resource templates atomically

45d627d

fix: preserve resource template update atomicity

05754d7

feat: add resource copy-on-write API

e2265ca

fix: preserve resource copy metadata

4db1e95

feat: write process runs through commands

aba3322

refactor: submit builders as aggregate commands

117e25e

fix: preserve resource builder properties

11414b5

Shekar V (vshekar) added 25 commits August 19, 2026 10:25

docs: document REST query architecture

55af350

test: cover canonical identity merges

e651c11

fix(db): add process run lineage migration

b792bfb

test(commands): cover process run copy route

f048b55

test(identity): preserve repeated array values

133847b

test: remove obsolete legacy checks

61ef50a

refactor: Added a NamespaceContext class to encapsulate namespace id …

3ad36ec

…and path

feat: finalize bumps the revision of process_run and activates associ…

af37b9d

…ated resources

Add existing child to a resource

e8cf2dd

feat: add builder transaction state

2d6c7ff

feat: preserve builder provisional ids

e0cca29

fix: stabilize reused builder ids

32cc79b

feat: defer template builder commits

985f336

feat: defer resource builder commits

4882b69

fix: resolve resource builder review findings

20ade28

fix: guard unloaded resource parents

5e29042

fix: close resource builder review gaps

1f74014

fix: defer process run lifecycle commits

724ade4

fix: preserve process run draft baseline

f9b78b5

refactor: normalize builder commit lifecycle

ded416b

fix: close builder review gaps

4623a4f

fix: add_child in resource builder

6742e68

Documentation corrections

fb8bc93

Update pyproject and pixi

cda37b5

Added doc build workflow

2d0188b

Copilot AI lite review requested due to automatic review settings September 16, 2026 16:19

Copilot started reviewing on behalf of Shekar V (vshekar) September 16, 2026 16:19 View session

github-actions Bot requested a deployment to pr-6 September 16, 2026 16:21 In progress

Shekar V (vshekar) merged commit b3f801f into main Sep 16, 2026
6 checks passed

Copilot AI reviewed Sep 16, 2026

View reviewed changes

Navigation Menu

Rest api + docs + namespaces - #6

Rest api + docs + namespaces#6
Shekar V (vshekar) merged 121 commits into
mainfrom
rest-api

Shekar V (vshekar) commented Sep 16, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

This branch is being deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		is required; environment variables and direct kwargs override YAML.
		Raises FileNotFoundError if the config file does not exist.

Conversation

Shekar V (vshekar) commented Sep 16, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (8)

Uh oh!

This branch is being deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants