Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/hub-infrastructure.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Hub infrastructure

on:
pull_request:
push:
branches: [main]

permissions:
contents: read

defaults:
run:
working-directory: infrastructure/azure

jobs:
terraform:
name: Terraform ${{ matrix.command }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
command: ["fmt -check -recursive", "validate", "test"]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
with:
terraform_version: 1.13.4
terraform_wrapper: false
- run: terraform init -backend=false -input=false -lockfile=readonly
- run: terraform ${{ matrix.command }}

release:
name: Release orchestration tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
- run: node --test deploy-hub/index.test.ts
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Commits and pull requests follow the [contribution conventions](../CONTRIBUTING.
## Project guidance

- [Hub documentation](../hub/docs/README.md) covers starting and checking the TypeScript API and frontend, and what is not built yet.
- [Azure Hub installation](../infrastructure/README.md) covers Terraform, release ordering, ownership, and recovery.
- [Starter instructions](../starter/AGENTS.md) and [setup guide](../starter/SETUP.md) cover the copyable prototype app and skills.
- [Build action setup](../CONTRIBUTING.md#replica-build-action) covers the shared action.
- [Controls documentation](../hub/docs/prototype-controls.md) covers the shared engine and definition schema owned by Hub.
Expand Down
8 changes: 5 additions & 3 deletions hub/docs/artifact-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The artifact storage foundation saves immutable bytes behind a provider-neutral

## Composition

Call `createArtifactStorage(settings)` from `server/domains/artifacts/create-artifact-storage/index.ts` during startup. The asynchronous factory validates settings and returns `read(key)`, `write(key, bytes)`, and `delete(key)`. `parseArtifactStorageSettings(input)` validates unknown configuration separately when startup needs a typed selection before composition.
Call `createArtifactStorage(settings)` from `server/domains/artifacts/create-artifact-storage/index.ts` during startup. The asynchronous factory validates settings and returns `isReady()`, `read(key)`, `write(key, bytes)`, and `delete(key)`. `parseArtifactStorageSettings(input)` validates unknown configuration separately when startup needs a typed selection before composition.

Choose exactly one settings shape:

Expand Down Expand Up @@ -35,11 +35,13 @@ Alternatively, startup can call `createConfiguredArtifactStorage(environment)` f

The wrapper constructs the public-cloud HTTPS account URL and calls the typed factory. Missing or invalid providers, incomplete configuration, and mixed-provider settings fail with a safe error. An empty value still counts as a supplied setting when checking mixed providers. Unrelated environment variables are ignored. Startup may leave storage uncomposed when artifact storage is not configured; calling the wrapper without a provider always rejects.

API, combined, and independent worker entry points validate and construct configured storage before startup. Any supplied provider, directory, account, or container setting opts in, so incomplete settings fail rather than being ignored. Setup does not need storage. Without storage settings, development keeps working unchanged. This startup check does not read or write artifacts, prove Azure access, or connect an unimplemented product flow.
API, combined, and independent worker entry points validate and construct configured storage before startup. Any supplied provider, directory, account, or container setting opts in, so incomplete settings fail rather than being ignored. Setup does not need storage. Without storage settings, development keeps working unchanged. Construction alone does not prove Azure access or connect an unimplemented product flow.

Unknown typed settings, relative directories, account credentials, URL query strings, and unsupported endpoints fail validation with a safe error. Azure selection never falls back to local storage.

The Azure endpoint accepts public-cloud HTTPS account URLs only. The adapter uses `ManagedIdentityCredential`, with either the system-assigned identity or the explicitly selected user-assigned identity. It never uses account keys, connection strings, developer credentials, or `DefaultAzureCredential`. Operators provision the account and container and grant the runtime identity `Storage Blob Data Contributor` at the intended container scope. Construction does not contact Azure; missing identity, permissions, or container availability is reported when an operation runs.
The Azure endpoint accepts public-cloud HTTPS account URLs only. The adapter uses `ManagedIdentityCredential`, with either the system-assigned identity or the explicitly selected user-assigned identity. It never uses account keys, connection strings, developer credentials, or `DefaultAzureCredential`. Operators provision the account and container and grant the runtime identity `Storage Blob Data Contributor` at the intended container scope.

The API's `/api/ready` includes configured storage. `isReady()` has a two-second deadline and makes a non-writing container-properties request through the selected managed identity, or verifies the filesystem directory. Provider failure or timeout fails readiness with a safe diagnostic. The Azure check verifies container read access, not write/delete permission, available capacity, or product operations. It never creates or deletes a probe artifact. Standalone workers have no readiness endpoint.

## Bytes and results

Expand Down
16 changes: 16 additions & 0 deletions hub/docs/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,4 +68,20 @@ npm run db:generate

Use `db:migrate` only with a local development database. It needs permission to create a shadow database to calculate migrations. `npm run db:deploy` applies committed migrations without creating new ones; database tests use this command so constraints in committed migrations are exercised. It is not a production deployment setup. `npm run db:studio` opens Prisma's database editor and should be used only with an intended development database.

## Deployment database roles

The Azure installation separates database provisioning, migrations, and normal data access:

| Principal | Access |
| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Administrator | One-time database and role provisioning; never passed to Hub or its setup job. |
| `replica_migration` | Can create schemas in this database and owns `public`, `bullmq`, and their migration-created objects. Runs setup, not the long-running app. |
| `replica_runtime` | Uses both schemas, reads/writes application and queue data, uses sequences and executes queue functions. Cannot create schemas, change permanent schema objects, create roles, truncate tables, or change migration ledgers. BullMQ requires read access to its ledger and temporary-table permission for bulk jobs. |

`server/scripts/provision-database-access/` creates the two login roles and schema ownership on a fresh database. It requires administrator access over the private database network and separate passwords of at least 16 characters. Existing roles, public tables, or a pre-existing BullMQ schema cause failure and rollback rather than adopting unknown permissions. Use a reviewed ownership migration for existing installations.

When `DATABASE_RUNTIME_ROLE=replica_runtime`, setup runs `server/scripts/grant-runtime-database-access/` after Prisma and BullMQ setup. That script initializes the two queues' lazy ID sequences before granting data access, so runtime never needs permanent schema creation rights. It refreshes grants for newly migrated tables, sequences, and functions, excludes the Prisma ledger, and permits only reading BullMQ's ledger. Product additions that need other database capabilities must explicitly update and verify this contract.

Local development keeps its existing database user and leaves `DATABASE_RUNTIME_ROLE` unset. See the [Azure installation guide](../../infrastructure/README.md) for private-network provisioning, secret references, release ordering, and recovery.

The [database-enabled test projects](testing-and-operations.md#parallel-database-tests) use separate databases for every project, worker, and test run. Setup deploys the committed migrations and clears records before each case. Tests do not use the development database for records.
5 changes: 4 additions & 1 deletion hub/docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Publishing is an explicit operator step, not a side effect of CI. From the appro
| `RUNTIME_MODE` | `api` for source development; the image defaults to `combined`. `on-demand`, `scheduled`, and `setup` are separate roles using the same image. |
| `HOST`, `PORT` | Source defaults to `127.0.0.1:3001`; the image binds `0.0.0.0:3001`. |
| `DATABASE_URL` | PostgreSQL connection URL, required for combined, worker, and setup modes. Configure verified TLS for remote databases. |
| `DATABASE_RUNTIME_ROLE` | Setup-only opt-in for separate database roles: exactly `replica_runtime`. See [database access](database.md#deployment-database-roles). |
| `CLIENT_DIRECTORY` | Absolute directory containing built frontend assets and `index.html`; the image uses `/app/dist/client`. Leave unset for the development API. |
| `SHUTDOWN_TIMEOUT_MS` | Request and worker drain deadline, default 30000 milliseconds. Allow another five seconds for telemetry flushing. |
| `ON_DEMAND_JOB_CONCURRENCY`, `SCHEDULED_JOB_CONCURRENCY` | Independent worker concurrency, as described in [Background jobs](background-jobs.md). |
Expand All @@ -21,7 +22,7 @@ Configuration errors stop startup without printing credentials. Logging and opti

The runtime serves built frontend navigation and static files on the API port. Unknown `/api` endpoints remain JSON 404 responses. Missing files with extensions do not return the frontend document.

`/api/health` is liveness. `/api/ready` returns 503 during initialization, shutdown, or a failed dependency check. In combined mode it requires both workers to initialize and a successful PostgreSQL query. Worker readiness records initialization only, not continuous consumer health: a later worker connection error can be logged while readiness still succeeds. The PostgreSQL query checks current database availability. Readiness is an operational signal, not authorization.
`/api/health` is liveness. `/api/ready` returns 503 during initialization, shutdown, or a failed dependency check. In combined mode it requires both workers to initialize and a successful PostgreSQL query. Worker readiness records initialization only, not continuous consumer health: a later worker connection error can be logged while readiness still succeeds. The PostgreSQL query checks current database availability. When [artifact storage](artifact-storage.md) is configured, both API and combined modes also require its bounded, non-writing availability check. Readiness is an operational signal, not authorization.

On termination, combined mode stops all three roles and drains work within the configured deadline. A required role failing or stopping unexpectedly fails the container. Roles remain independently runnable for future deployments; cross-instance WebSocket delivery is not implemented.

Expand All @@ -41,4 +42,6 @@ The deployment orchestrator owns serialization of the complete release, includin

CI checks the image against disposable PostgreSQL, including setup, frontend/API serving, readiness, and termination. This is not a supported local-production installation. The existing devcontainer remains the development environment.

The [Azure reference installation](../../infrastructure/README.md) uses Terraform and a serialized maintenance-window release. It consumes the same image and runtime settings without adding Azure deployment logic to Hub.

An actual cloud deployment must additionally verify credentials, networking, data survival, backup and restore, domain configuration, and release ordering in an authorized environment. Never commit secrets, tenant-specific parameter files, or generated service addresses.
2 changes: 1 addition & 1 deletion hub/docs/testing-and-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ npm run storybook:build

The API health check proves only that Express responds. Product API routes that have not been built must return 404 rather than pretending to work.

Run `npm run image:check` with a Docker-compatible daemon to build the production image and exercise setup, readiness, serving, and shutdown against disposable PostgreSQL. It creates uniquely named resources and removes them afterward. The image compiles the Hub runtime only; the normal `build` command also checks the frontend, test tooling, and shared extension.
Run `npm run image:check` with a Docker-compatible daemon to build the production image and exercise role provisioning, repeated setup, runtime data access and denied schema changes, queue processing, readiness, serving, and shutdown against disposable PostgreSQL. It creates uniquely named resources and removes them afterward. The image compiles the Hub runtime only; the normal `build` command also checks the frontend, test tooling, and shared extension.

Installation generates Prisma Client without a database connection. Client tests check reuse across development reloads, production behavior, and errors for missing configuration.

Expand Down
Loading
Loading