Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions hub/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ DATABASE_URL=postgresql://replica:replica@127.0.0.1:5432/replica
# Only database-enabled tests require this URL and CREATEDB permission.
TEST_DATABASE_URL=postgresql://replica:replica@127.0.0.1:5432/postgres

# Local reference artifact storage. The default is ~/.replica/artifacts, outside the checkout.
# Production deployments should mount durable storage here.
# HUB_ARTIFACT_STORAGE_PATH=/var/lib/replica/artifacts

# GitHub user authorization is optional until both values are configured.
# These credentials establish user identity and are separate from source integration.
# HUB_GITHUB_AUTH_CLIENT_ID=
Expand Down
1 change: 1 addition & 0 deletions hub/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Hub is being rebuilt. The [Hub overview](../README.md) describes what the baseli
- [GitHub integration](github-integration.md): linked repository sources, reconciliation, webhooks, staleness, and permission mapping.
- [Membership and administration](membership-and-administration.md): Hub admission, invitations, roles, lifecycle, bootstrap, and recovery.
- [Prototype authorization](prototype-authorization.md): workspaces, prototypes, immutable versions, Hub overrides, synchronized baselines, and policy precedence.
- [Artifact ingestion](artifact-ingestion.md): workload publishers, the versioned manifest, storage, immutable publishing, idempotency, and promotion.
- [Logging](logging.md): application events, automatic request records, and payload safety.
- [Background jobs](background-jobs.md): PostgreSQL queues, workers, schedules, setup, and test isolation.
- [Testing and operations](testing-and-operations.md): baseline checks and their limits.
Expand Down
6 changes: 3 additions & 3 deletions hub/docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ Hub contributors are rebuilding the app on `build/hub/new` before merging it int

The team has chosen TypeScript, Express for the API, React and Vite for the frontend, and Vitest for tests. The old backend and the frontend code that connected to it have been removed rather than rewritten.

The API responds to a health check and provides GitHub authentication, Hub sessions, installation membership administration, Hub-native prototype authorization, and opt-in GitHub-synchronized admission and prototype baselines. The frontend displays sign-in, administrator enrollment and recovery, and a minimal membership administration page with light and dark themes. Storybook keeps the existing components and pages available for reuse.
The API responds to a health check and provides GitHub authentication, Hub sessions, installation membership administration, Hub-native prototype authorization, opt-in GitHub-synchronized admission and prototype baselines, and provider-neutral artifact publishing. The frontend displays sign-in, administrator enrollment and recovery, and a minimal membership administration page with light and dark themes. Storybook keeps the existing components and pages available for reuse.

Contributors can use PostgreSQL and Prisma locally or in the devcontainer. The [identity system](identity.md) separates Hub users, external identities, installation membership, authentication-provider configuration, sessions, authorization attempts, security records, and GitHub source-integration credentials. GitHub user authorization, Hub-owned admission, provider-neutral sessions, invitations, membership lifecycle, and administrator recovery work when user-auth credentials are configured. The [background-job foundation](background-jobs.md) provides PostgreSQL-backed queues and workers without product jobs. The [prototype authorization system](prototype-authorization.md) adds workspaces, prototypes, immutable version records, direct roles, explicit deny, synchronized GitHub baselines, and one policy evaluator. The [GitHub integration](github-integration.md) stores stable repository source records and reconciles current permissions with installation credentials. Deployment has not been built. This app does not ingest artifacts, host previews, browse source, or support reviews.
Contributors can use PostgreSQL and Prisma locally or in the devcontainer. The [identity system](identity.md) separates Hub users, external identities, installation membership, authentication-provider configuration, sessions, authorization attempts, security records, and GitHub source-integration credentials. GitHub user authorization, Hub-owned admission, provider-neutral sessions, invitations, membership lifecycle, and administrator recovery work when user-auth credentials are configured. The [background-job foundation](background-jobs.md) provides PostgreSQL-backed queues and workers without product jobs. The [prototype authorization system](prototype-authorization.md) adds workspaces, prototypes, immutable version records, direct roles, explicit deny, synchronized GitHub baselines, and one policy evaluator. The [GitHub integration](github-integration.md) stores stable repository source records and reconciles current permissions with installation credentials. The [artifact ingestion system](artifact-ingestion.md) authenticates prototype-scoped workloads, validates a shared manifest, stores bytes behind an adapter, creates immutable versions, and moves promotion pointers. Deployment has not been built. This app does not host previews, browse source, or support reviews.

## Backend

API code lives under `server/`. As features are added, route handlers will read requests, call domains, and return responses. The [code conventions](backend-architecture.md) explain repositories, integrations, utilities, domains, and which layers may import others. Put these directories under `server/` when they are needed; do not add empty implementations.

`GET /api/health` returns `{ "status": "ok" }`. Authentication routes start and complete GitHub authorization, restore or revoke Hub sessions, and perform fresh verification. Administration routes list memberships and invitations, create or revoke invitations, and change membership roles or lifecycle. Product routes create and read workspaces, prototypes, and immutable versions; archive prototypes; choose Hub-managed or GitHub-synchronized prototype authorization; explain effective access; and administer direct access rules. Administration routes link repository sources, opt them into admission, inspect synchronization state, and trigger reconciliation. A verified GitHub App webhook route marks affected sources dirty or removed. Unknown paths return a JSON 404 response; invalid JSON is rejected before routing. `createApp()` builds the Express app without starting a server. `server/index.ts` composes the authentication, administration, prototype authorization, and GitHub synchronization runtime, starts the API at a local address, and schedules bounded periodic reconciliation.
`GET /api/health` returns `{ "status": "ok" }`. Authentication routes start and complete GitHub authorization, restore or revoke Hub sessions, and perform fresh verification. Administration routes list memberships and invitations, create or revoke invitations, change membership roles or lifecycle, link repository sources, opt them into admission, inspect synchronization state, and trigger reconciliation. Product routes create and read workspaces and prototypes; read immutable versions; provision workload publishers; archive prototypes; choose Hub-managed or GitHub-synchronized prototype authorization; explain effective access; and administer direct access rules. Publishing routes ingest artifacts, read idempotent status, and move `current` or `preview`. A verified GitHub App webhook route marks affected sources dirty or removed. Unknown paths return a JSON 404 response; invalid JSON is rejected before routing. `createApp()` builds the Express app without starting a server. `server/index.ts` composes the authentication, administration, prototype authorization, GitHub synchronization, and artifact publishing runtime, starts the API at a local address, and schedules bounded periodic reconciliation.

The [database guide](database.md) defines the multi-file Prisma schema and shared client in integrations. The health route remains a process check, not a database readiness check.

Expand Down
90 changes: 90 additions & 0 deletions hub/docs/artifact-ingestion.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Artifact ingestion and prototype versions

Hub accepts one provider-neutral artifact contract from workload producers. GitHub Actions, Azure DevOps, Jenkins, local scripts, Copilot workflows, and other systems use the same API. Producers build outside Hub; Hub authenticates the workload, validates and stores the bytes, creates an immutable prototype version, records provenance, and may move a channel pointer.

## Identity and access

A prototype administrator creates a workload publisher with `publish` and may also grant `read-status`:

| Action | Authority |
| ------------- | ---------------------------------------------------------------------------- |
| `publish` | Publish an artifact and move the prototype's `current` or `preview` channel. |
| `read-status` | Read the result associated with that publisher's idempotency key. This action supplements `publish`; a status-only publisher would have no publication to read. |

Hub returns a bearer credential once and stores only its SHA-256 digest. The credential is scoped to one prototype through its workload publisher. It cannot create a browser session, view a prototype through user APIs, administer access, or publish to another prototype. A producer's source URL, revision, ref, pipeline, and run ID describe where a build came from; they never grant access.

Browser users provision credentials through `POST /api/prototypes/:prototypeId/publishers`. This requires a fresh prototype-administrator session and returns the raw token only in that response. `GET /api/prototypes/:prototypeId/publishers` returns administrator-only publisher and credential metadata without token digests or raw tokens, so another administrator can discover and revoke existing access. An administrator may temporarily disable or re-enable a publisher with `PATCH /api/prototypes/:prototypeId/publishers/:publisherId`, or permanently revoke one credential with `DELETE /api/prototypes/:prototypeId/publishers/:publisherId/credentials/:credentialId`. Hub audits each state change. Re-enabling a publisher does not restore a revoked credential.

Publication, promotion, publisher state changes, and credential revocation use the same prototype lock. Publication and promotion re-read credential expiry, revocation, scope, action, token digest, and publisher state after acquiring that lock. A revocation or disable that locks first prevents an already-started workload request from committing; a workload operation that locks first completes before the administrator mutation.

Archiving a prototype blocks new publishers, publication, publisher re-enablement, and `current` or `preview` promotion. Administrators may still disable publishers and revoke credentials after archival. Existing versions and publication status remain readable through their authorized interfaces.

## Manifest contract

`POST /api/publishing/prototypes/:prototypeId/versions` requires `Authorization: Bearer ...`, an `Idempotency-Key` header, and this JSON shape:

```json
{
"manifest": {
"schemaVersion": 1,
"prototypeId": "00000000-0000-0000-0000-000000000000",
"artifact": {
"format": "replica.prototype.bundle",
"version": "1",
"digest": "sha256:<64 lowercase hexadecimal characters>",
"sizeBytes": 1234,
"builtAt": "2026-10-27T20:00:00.000Z"
},
"provenance": {
"provider": "jenkins",
"sourceUrl": "https://ci.example/jobs/prototype",
"revision": "abc123",
"ref": "refs/heads/main",
"pipeline": "prototype-build",
"runId": "451"
}
},
"artifactBase64": "<base64 bytes>",
"promoteChannel": "preview"
}
```

The manifest and its nested objects are closed to unknown fields. Hub bounds every string, rejects NUL characters, accepts only HTTP or HTTPS source URLs without embedded usernames, passwords, query strings, or fragments, limits decoded artifacts to 8 MiB in this reference layer, verifies canonical base64, checks the decoded size, and recalculates the SHA-256 digest before storing bytes. Valid UUIDs are normalized before scope and idempotency checks, so hexadecimal casing does not change identity. The manifest prototype ID must match the route.

Publishing requests are rate-limited before JSON parsing. The version-upload route alone accepts the bounded upload-sized JSON body; status and promotion retain the API's default JSON limit.

## Storage boundary

`ArtifactStorage` separates ingestion from byte storage. The reference adapter writes content-addressed files under `HUB_ARTIFACT_STORAGE_PATH`, defaulting to `~/.replica/artifacts` outside the source checkout and Vite development root. Database rows store only the storage key and validated metadata, never the artifact bytes.

The local adapter is suitable for one Hub process with a durable mounted filesystem. It rejects relative configured paths and symbolic-link roots, shards, and artifact paths, normalizes root spellings before validation, resolves directories under the configured root, opens artifact files without following a final symbolic link, and syncs file and directory entries through the storage-root boundary before reporting a successful write. Operators must keep the storage directory writable only by the Hub workload; the local adapter does not provide replication, remote durability, encryption-key management, hostile concurrent filesystem isolation, or multi-region consistency. A production object-store adapter must preserve the same content-addressed, immutable `put`, `read`, and `remove` boundary.

Storage happens before the database transaction. A failed transaction can therefore leave an unreferenced content-addressed file. Hub never deletes referenced bytes automatically. Operators may remove only files that have no matching `PrototypeVersion.storageKey`; retain unreferenced files long enough to exclude in-flight transactions and backups. Version metadata has no deletion API and must not be rewritten silently.

## Idempotency and immutability

Hub hashes the idempotency key and stores it with a digest of the validated publication request. Retrying the same request with the same publisher and key returns the existing version. Reusing the key for different manifest or promotion input returns `idempotency_conflict`. Publishing the same content digest again to the prototype returns `artifact_already_published`.

Publishing locks the prototype while it assigns the next sequence. Concurrent requests therefore create one ordered history. PostgreSQL rejects every update or deletion of a prototype-version row, including accidental writes outside the application.

## Promotion and rollback

`current` and `preview` are mutable pointers to immutable versions. A publish request may move one pointer in the same transaction that creates the version. `PUT /api/publishing/prototypes/:prototypeId/channels/:channel` moves a pointer to any version owned by that prototype. Rollback uses that endpoint to point the channel at an earlier version; it never mutates the version.

## Status, audits, and failures

`GET /api/publishing/prototypes/:prototypeId/status` reads the publication associated with the caller's `Idempotency-Key`. It requires `read-status`. Publishing responses are private and non-cacheable because the idempotency key is carried in a request header rather than the URL.

Version creation, idempotency ownership, optional promotion, and the `prototype.version.publish` security audit commit in one transaction. Promotion writes `prototype.channel.promote` in its transaction. Audit failure rolls back database state. Responses expose validated version metadata and stable error codes, not credentials, storage paths, or artifact contents. Publishing routes omit headers and request payloads from structured request logs.

## Local producer

The local CLI proves that publishing does not depend on GitHub:

```sh
cd hub
REPLICA_PUBLISH_TOKEN=... npm run publish:artifact -- \
http://127.0.0.1:3001 PROTOTYPE_ID ./prototype.tar preview
```

The CLI prints its generated or configured idempotency key before sending the request. If the response is lost, retry with that value in `REPLICA_IDEMPOTENCY_KEY` to recover the committed result. Optional `REPLICA_PROVENANCE_PROVIDER`, `REPLICA_SOURCE_URL`, `REPLICA_REVISION`, `REPLICA_REF`, `REPLICA_PIPELINE`, and `REPLICA_RUN_ID` values populate provenance without changing authorization.
4 changes: 3 additions & 1 deletion hub/docs/database.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Hub database

Hub uses PostgreSQL and Prisma. The schema stores the provider-neutral [identity foundation](identity.md) and the [prototype authorization](prototype-authorization.md) product core.
Hub uses PostgreSQL and Prisma. The schema stores the provider-neutral [identity foundation](identity.md), the [prototype authorization](prototype-authorization.md) product core, and [artifact publication metadata](artifact-ingestion.md).

[Background jobs](background-jobs.md) use the same database with BullMQ-owned tables and functions in the `bullmq` schema. Prisma does not own those objects. `npm run setup` applies Prisma migrations before BullMQ setup and schedule registration.

Expand Down Expand Up @@ -60,6 +60,8 @@ GitHub synchronization uses separate records:

`Prototype.authorizationMode` defaults to `hub-managed`. `sourceRepositoryId` is nullable and applies only when the mode is explicitly `github-synchronized`. Synchronized records never share rows with workspace or prototype grants, so reconciliation cannot overwrite manual Hub decisions.

A database trigger rejects prototype version updates and deletions so history cannot be rewritten or silently removed. Workload credentials store token digests plus optional expiry and revocation timestamps; publishers have an independent active flag for temporary suspension. Publication rows bind a publisher-scoped idempotency-key digest to one version, and channel rows keep mutable pointers separate from immutable version metadata. Artifact bytes stay outside relational columns behind the storage boundary.

Installation membership `source` distinguishes linked-source synchronization from the `legacy-synchronized` compatibility policy. Any administrator lifecycle change or assignment of Hub administration converts a synchronized membership to `manual`, so reconciliation cannot undo the Hub override or orphan administration.

## Shared client
Expand Down
Loading
Loading