Skip to content

feat(hub): add provider-neutral artifact ingestion - #149

Open
alliscode wants to merge 13 commits into
alliscode-github-synchronized-accessfrom
alliscode-artifact-ingestion-layer
Open

alliscode wants to merge 13 commits into
alliscode-github-synchronized-accessfrom
alliscode-artifact-ingestion-layer

Conversation

@alliscode

Copy link
Copy Markdown
Collaborator

Why

Prototype builds can come from GitHub Actions, Azure DevOps, Jenkins, local tools, Copilot workflows, or another producer. Hub needs one publishing boundary for all of them instead of tying artifact ingestion to a source provider or a browser user session.

What changes

Hub now accepts a versioned, provider-neutral artifact manifest from prototype-scoped workload credentials. Publishing validates bounded metadata and bytes, verifies the content digest and size, stores artifacts through a content-addressed storage abstraction, and creates an immutable prototype version. Idempotency keys make retries return the same version and reject conflicting replays.

Workload credentials are separate from browser sessions and product access. Prototype administrators can disable publishers or permanently revoke individual credentials, and publication rechecks that authority while holding the same prototype lock used by those administration changes. Archived prototypes cannot receive artifacts or promotion changes. Source provenance records the provider, source URL, revision, ref, pipeline, and run ID, but never grants authorization.

The current and preview channels are atomic pointers to immutable versions, so promotion and rollback move a pointer rather than rewriting history. The local reference storage rejects symbolic-link escapes and keeps artifact bytes outside relational columns behind a future object-store boundary. A local CLI fixture proves that a non-GitHub producer can publish the same contract.

This layer does not call GitHub or Azure APIs and does not add product UI. Workload OIDC and other credential providers remain the next identity layer above this publishing contract.

Copilot AI auto review requested due to automatic review settings October 28, 2026 18:53
@alliscode
alliscode requested review from a team and nshki as code owners October 28, 2026 18:53
@alliscode
alliscode added this pull request to stack #122 October 28, 2026 18:54
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical security issues remain: private artifacts are accessible without authorization, and unauthenticated publishing requests can force large allocations and JSON parsing before rate limiting.

12 open issues
What changed in this PR

Adds a provider-neutral publishing boundary to Hub, separating artifact ingestion from source providers and browser sessions.

Changes:

  • Adds workload-authenticated publishing, idempotent retries, publication status, and channel promotion.
  • Introduces content-addressed storage, publisher lifecycle controls, and database-backed version history and audits.
  • Adds a local producer CLI, automated coverage, and operating documentation.
File Description
hub/​server/​routes/​types.ts Adds the publishing service dependency.
hub/​server/​routes/​publishing/​index.ts Handles publication, status, and promotion requests.
hub/​server/​routes/​publishing.test.ts Covers routing, limits, and log safety.
hub/​server/​routes/​product/​index.ts Adds publisher administration handlers.
hub/​server/​routes/​product/​index.test.ts Updates publisher lifecycle route coverage.
hub/​server/​routes/​index.ts Registers publishing and publisher administration endpoints.
hub/​server/​repositories/​security-audits/​types.ts Adds workload identifiers to audit contracts.
hub/​server/​repositories/​security-audits/​save-audit/​index.ts Persists workload audit identifiers.
hub/​server/​integrations/​artifact-storage/​types.ts Defines the artifact storage interface.
hub/​server/​integrations/​artifact-storage/​create-local-storage/​index.ts Implements content-addressed local storage.
hub/​server/​integrations/​artifact-storage/​create-local-storage/​index.test.ts Covers storage integrity and symbolic-link rejection.
hub/​server/​domains/​prototype-authorization/​create-service/​types.ts Adds publisher lifecycle service methods.
hub/​server/​domains/​prototype-authorization/​create-service/​index.ts Manages publishers and exposes version metadata.
hub/​server/​domains/​prototype-authorization/​create-service/​index.db.test.ts Covers publisher lifecycle and audit rollback.
hub/​server/​domains/​authentication/​create-runtime/​index.ts Composes publishing with local storage.
hub/​server/​domains/​artifact-publishing/​create-service/​types.ts Defines manifest and publishing contracts.
hub/​server/​domains/​artifact-publishing/​create-service/​index.ts Implements validation, publication, retries, and promotion.
hub/​server/​domains/​artifact-publishing/​create-service/​index.db.test.ts Covers publishing persistence, authorization, and concurrency.
hub/​server/​domains/​artifact-publishing/​create-service/​constants.ts Defines artifact and request size limits.
hub/​server/​create-app/​index.ts Adds publishing parsing and log omission.
hub/​script/​publish-artifact.mjs Provides a local workload producer CLI.
hub/​schema/​workload-publisher.prisma Models prototype-scoped publishers.
hub/​schema/​workload-credential.prisma Models credential digests, expiry, and revocation.
hub/​schema/​security-audit.prisma Adds workload audit fields and indexes.
hub/​schema/​prototype.prisma Relates prototypes to publishing records.
hub/​schema/​prototype-version.prisma Adds artifact metadata and workload ownership.
hub/​schema/​prototype-channel.prisma Models mutable version pointers.
hub/​schema/​migrations/​20261027231000_publisher_lifecycle/​migration.sql Adds credential identification to audits.
hub/​schema/​migrations/​20261027230000_artifact_ingestion/​migration.sql Creates publishing tables and immutability constraints.
hub/​schema/​artifact-publication.prisma Records publisher-scoped idempotency ownership.
hub/​package.json Adds the artifact publishing command.
hub/​docs/​testing-and-operations.md Documents coverage and storage operations.
hub/​docs/​README.md Links the artifact ingestion guide.
hub/​docs/​prototype-authorization.md Updates publishing authority and lifecycle rules.
hub/​docs/​logging.md Documents publishing payload and credential omission.
hub/​docs/​identity.md Explains the separate workload identity boundary.
hub/​docs/​database.md Describes publication metadata and immutable history.
hub/​docs/​artifact-ingestion.md Defines the publishing contract and operational limits.
hub/​docs/​architecture.md Places artifact ingestion within Hub’s architecture.
hub/​.env.example Documents the storage path setting.

Review effort: Max (auto)

Comment thread hub/server/create-app/index.ts Outdated
Comment thread hub/server/domains/authentication/create-runtime/index.ts Outdated
Comment thread hub/server/domains/artifact-publishing/create-service/index.db.test.ts Outdated
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts Outdated
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts Outdated
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts
Comment thread hub/server/domains/prototype-authorization/create-service/index.ts
Comment thread hub/server/integrations/artifact-storage/create-local-storage/index.ts Outdated
Comment thread hub/server/routes/index.ts
@alliscode
alliscode force-pushed the alliscode-artifact-ingestion-layer branch from 066ebdf to 8eeaed5 Compare October 28, 2026 21:16
@alliscode
alliscode requested an auto review from Copilot October 28, 2026 22:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread hub/server/domains/authentication/create-runtime/index.ts Outdated
Comment thread hub/server/create-app/index.ts
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts
Comment thread hub/server/domains/prototype-authorization/create-service/index.ts
Comment thread hub/server/domains/prototype-authorization/create-service/index.ts Outdated
@alliscode
alliscode force-pushed the alliscode-artifact-ingestion-layer branch from 8eeaed5 to 996733f Compare October 29, 2026 11:16
@alliscode
alliscode requested an auto review from Copilot October 29, 2026 11:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread hub/server/domains/artifact-publishing/create-service/index.ts
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts
@alliscode
alliscode force-pushed the alliscode-artifact-ingestion-layer branch from 996733f to 3a9c8d7 Compare October 29, 2026 12:34
@alliscode
alliscode requested an auto review from Copilot October 29, 2026 14:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved NUL validation, retry-key handling, and UUID normalization bugs can cause server errors after artifact storage, retry conflicts, and rejection of valid credentials.

3 open issues
5 resolved since last review

Review effort: Max (auto)

Comment thread hub/script/publish-artifact.mjs
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts Outdated
Comment thread hub/server/domains/artifact-publishing/create-service/index.ts Outdated
@alliscode
alliscode force-pushed the alliscode-artifact-ingestion-layer branch from ee6c60b to ef9d710 Compare October 29, 2026 15:11
@alliscode
alliscode requested an auto review from Copilot and removed request for Copilot October 29, 2026 15:19
@alliscode
alliscode force-pushed the alliscode-artifact-ingestion-layer branch from 5fea929 to fdc0dfc Compare October 29, 2026 16:58
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Ben Thomas and others added 12 commits October 29, 2026 10:27
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@alliscode
alliscode force-pushed the alliscode-artifact-ingestion-layer branch from 18a4aea to be0d403 Compare October 29, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants