Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
187 changes: 187 additions & 0 deletions .github/workflows/github-r2-fallback.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
name: GitHub R2 fallback and typed RPC canary

on:
pull_request:
paths:
- ".github/workflows/github-r2-fallback.yml"
- "scripts/github_r2_fallback.py"
- "docs/github-r2-fallback-canary.md"
workflow_dispatch:
inputs:
zed_cli_ref:
description: Exact 40-character zed-pkg/zed-cli commit
required: false
type: string
zed_interfaces_ref:
description: Exact 40-character zed-pkg/zed-interfaces commit
required: false
type: string
workflow_call:
inputs:
zed_cli_ref:
description: Exact 40-character zed-pkg/zed-cli commit
required: true
type: string
zed_interfaces_ref:
description: Exact 40-character zed-pkg/zed-interfaces commit
required: true
type: string

permissions:
contents: read

concurrency:
group: github-r2-fallback-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref || github.run_id }}
cancel-in-progress: true

defaults:
run:
shell: bash

env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: "1"

jobs:
syntax:
name: Canary script compiles
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- name: Check out harness
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 1
persist-credentials: false
show-progress: false
- name: py_compile
run: python3 -m py_compile scripts/github_r2_fallback.py

canary:
name: Registry-down CDN install
if: github.event_name != 'pull_request'
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- name: Check out independent certification harness
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 1
persist-credentials: false
show-progress: false

- name: Resolve product pins
id: pins
env:
CLI_IN: ${{ inputs.zed_cli_ref || '' }}
INTERFACES_IN: ${{ inputs.zed_interfaces_ref || '' }}
run: |
set -euo pipefail
cli="${CLI_IN:-}"
interfaces="${INTERFACES_IN:-}"
if [[ -z "$cli" ]]; then
echo "zed_cli_ref is required (workflow_call or workflow_dispatch)" >&2
exit 1
fi
if [[ -z "$interfaces" ]]; then
echo "zed_interfaces_ref is required (workflow_call or workflow_dispatch)" >&2
exit 1
fi
[[ "$cli" =~ ^[0-9a-f]{40}$ ]]
[[ "$interfaces" =~ ^[0-9a-f]{40}$ ]]
printf 'cli=%s\n' "$cli" >> "$GITHUB_OUTPUT"
printf 'interfaces=%s\n' "$interfaces" >> "$GITHUB_OUTPUT"

- name: Check out exact zed-cli candidate
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: zed-pkg/zed-cli
ref: ${{ steps.pins.outputs.cli }}
path: product/zed-cli
fetch-depth: 1
persist-credentials: false
show-progress: false

- name: Check out matching zed-interfaces
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: zed-pkg/zed-interfaces
ref: ${{ steps.pins.outputs.interfaces }}
path: product/zed-interfaces
fetch-depth: 1
persist-credentials: false
show-progress: false

- name: Check out zed-clients contract
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: zed-pkg/zed-clients
path: product/zed-clients
fetch-depth: 1
persist-credentials: false
show-progress: false

- name: Check out zed-lib-core contract
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: zed-pkg/zed-lib-core
path: product/zed-lib-core
fetch-depth: 1
persist-credentials: false
show-progress: false

- name: Verify immutable product identity
run: |
set -euo pipefail
test "$(git -C product/zed-cli rev-parse HEAD)" = "${{ steps.pins.outputs.cli }}"
test "$(git -C product/zed-interfaces rev-parse HEAD)" = "${{ steps.pins.outputs.interfaces }}"

- name: Install Rust
uses: dtolnay/rust-toolchain@4be7066ada62dd38de10e7b70166bc74ed198c30
with:
toolchain: stable
components: rustfmt

- name: Install Node.js
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v6.0.0
with:
node-version: "22"

- name: Contract tests (interfaces, clients, lib-core)
run: |
set -euo pipefail
cargo test --manifest-path product/zed-interfaces/src/rust/Cargo.toml --lib zed_api
cargo test --manifest-path product/zed-lib-core/src/rust-orm/Cargo.toml \
--features read-write \
publication::tests::canonical_artifact_key_is_content_addressed
(cd product/zed-clients/clients/typescript && npm ci --ignore-scripts && npm run build && npm test)

- name: Build exact zed CLI
run: |
set -euo pipefail
cargo fmt --all --check --manifest-path product/zed-cli/Cargo.toml
cargo build --release --manifest-path product/zed-cli/Cargo.toml --bin zed
install -Dm755 product/zed-cli/target/release/zed "$RUNNER_TEMP/zed-bin/zed"
"$RUNNER_TEMP/zed-bin/zed" --version

- name: Run registry-down CDN canary
run: |
set -euo pipefail
python3 -m py_compile scripts/github_r2_fallback.py
python3 scripts/github_r2_fallback.py \
--zed "$RUNNER_TEMP/zed-bin/zed" \
--work-root "$RUNNER_TEMP/github-r2-fallback" \
--route-map product/zed-interfaces/route-maps/zed-api.route-map.json \
--interfaces-generated-ts product/zed-interfaces/generated/typescript/zed_api.ts \
--clients-generated-ts product/zed-clients/clients/typescript/src/generated/zed-api.routes.ts

- name: Upload failure diagnostics
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: github-r2-fallback-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/github-r2-fallback
if-no-files-found: warn
retention-days: 14
include-hidden-files: false
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,20 @@ python3 scripts/lifecycle.py \
The work root is disposable. Reusing the same path is safe because the harness
requires a fresh root and GitHub-hosted jobs receive a new runner filesystem.


## GitHub / R2 fallback (registry down, CDN up)

`.github/workflows/github-r2-fallback.yml` is a reusable canary for the
`cdn.zpkg.net` guessable-object contract. `scripts/github_r2_fallback.py`
publishes a disposable package to a `file://` registry, copies the tarball onto
`packages/` and `github/` keys, points `--registry` at a closed loopback port,
and proves `zed install --frozen` restores the packed payload from a loopback
CDN. It also round-trips the typed `get_version` RPC frame over TCP NDJSON
(from `github.com/oresoftware/api-docs`). Pull requests only compile the
script; a full product run needs exact `zed-cli` and `zed-interfaces` commits
via `workflow_call` or `workflow_dispatch`. See
`docs/github-r2-fallback-canary.md`.

## Durable first-install acceptance

`.github/workflows/durable-first-install.yml` is the cross-organization
Expand Down
41 changes: 41 additions & 0 deletions docs/github-r2-fallback-canary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# GitHub / R2 registry-fallback canary

This repository proves that `zed install --frozen` still materializes a
package when the HTTP registry is down, as long as the packed tarball is at a
guessable public CDN key (`packages/{org}/{name}/{version}/…` or
`github/{owner}/{repo}/{tag}/…`). That is the `cdn.zpkg.net` contract.

The canary is credential-free and uses only loopback HTTP:

1. publish a disposable package to a `file://` registry;
2. copy the tarball onto the guessable CDN layout;
3. install once so a lock exists, then uninstall and drop the store;
4. point `--registry` at a closed `127.0.0.1` port;
5. set `ZED_PKG_SOURCE_FALLBACK_ALLOW_LOOPBACK=true` (production loopback stays
hermetic) and `--r2-public-base http://127.0.0.1:<cdn>`;
6. `zed install --frozen` must restore the packed payload from the CDN mock.

It also round-trips the `get_version` JSON call/receipt frame over TCP NDJSON,
using the same key as `zed-interfaces/route-maps/zed-api.route-map.json`
(generated by `github.com/oresoftware/api-docs`). zed-cli itself still speaks
HTTP; the TCP check is the typed RPC wire, not a second CLI transport.

When the TypeScript copies are passed in, the generated `zed-clients` routes
file must match `zed-interfaces/generated/typescript/zed_api.ts` (banner
comments excluded).

## Local run

From a sibling checkout of `zed-pkg`:

```bash
cargo build --release --manifest-path ../zed-pkg/zed-cli/Cargo.toml --bin zed
python3 scripts/github_r2_fallback.py \
--zed ../zed-pkg/zed-cli/target/release/zed \
--work-root /tmp/zed-github-r2-fallback \
--route-map ../zed-pkg/zed-interfaces/route-maps/zed-api.route-map.json \
--interfaces-generated-ts ../zed-pkg/zed-interfaces/generated/typescript/zed_api.ts \
--clients-generated-ts ../zed-pkg/zed-clients/clients/typescript/src/generated/zed-api.routes.ts
```

The work root must not already exist.
Loading
Loading