Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 52 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,53 @@ Do not deploy production from a feature branch or store Cloudflare credentials i

See [CONTRIBUTING.md](CONTRIBUTING.md) for the contribution workflow. The cross-repository operating model is maintained in [`ScientFactory/Scient`](https://github.com/ScientFactory/Scient).

## Desktop update feeds: deploy and verify

The `scientfactory-release-mirror` worker is the only publisher of
`downloads.scientfactory.com/scient-desktop/`. It copies the Stable installers and the complete
Stable and Beta electron-updater sets; see
[docs/architecture/desktop-downloads.md](docs/architecture/desktop-downloads.md). Pages
deployments do not deploy it.

1. After merge, from `main`: `bun run mirror:types:check && bun run mirror:deploy`.
2. Watch the next runs (every ten minutes) with
`bunx wrangler tail scientfactory-release-mirror --config workers/release-mirror/wrangler.jsonc`.
Expect `Stable update feed synchronized` and `Beta update feed synchronized` with `promoted`
naming three manifests each. The first run per release copies up to about 1.8 GB per channel and
hashes it; on the Workers Free plan (50 external subrequests, 10 ms CPU) it cannot finish,
so confirm the account is on Workers Paid. A run cut short resumes from what is already in the
bucket.
3. In the `scientfactory.com` zone, the `Scient download cache policy` ruleset (C1)
applies only to the `downloads.scientfactory.com` hostname:
- `/scient-desktop/updates/*` and `/scient-desktop/stable.json`: **Bypass cache**.
- `/scient-desktop/v*`: **Eligible for cache**, edge and browser TTLs **respect origin**,
with **no-store** for HTTP 300–599. Keep errors and redirects out of immutable caches.
Without such a rule, Cloudflare caches some 404s and does not cache `.AppImage`,
`.blockmap` or `.yml` by default. Files above 512 MB are not cached on non-Enterprise plans.
4. Verify each channel (`stable`/`latest`, then `beta`/`beta`):

```sh
feed=https://downloads.scientfactory.com/scient-desktop/updates/stable
for manifest in latest.yml latest-mac.yml latest-linux.yml; do
curl -fsS -D- "$feed/$manifest?noCache=$RANDOM" -o "/tmp/$manifest" | grep -i '^cache-control' # no-store
grep -E '^(version| - url| size| sha512|releaseName)' "/tmp/$manifest"
done
# For each url: it must resolve under scient-desktop/<tag>/, answer 200 with the listed size,
# and hash to the listed sha512.
url=https://downloads.scientfactory.com/scient-desktop/v0.6.30/Scient-0.6.30-x64.exe
curl -fsSI "$url" | grep -iE '^(content-length|cache-control|cf-cache-status)'
curl -fsS "$url" | openssl dgst -sha512 -binary | base64
curl -fsSI "$url.blockmap" | head -1 # 200
curl -sS -r 0-99 -o /dev/null -w '%{http_code} %{size_download}\n' "$url" # 206 100
curl -sS -o /dev/null -w '%{http_code}\n' https://downloads.scientfactory.com/scient-desktop/v0.0.0/missing.exe # 404, twice: never HIT
```

5. Only then merge the desktop change that makes these feeds the updater's default, and qualify
real updates on each platform from a build that uses them.

To stop publishing, remove the cron trigger and redeploy; the bucket's objects stay valid. Never
delete or rewrite a versioned object: installed clients and caches may still use it.

## Scient Docs publishing

The public `/docs` experience is generated from reviewed Markdown in
Expand Down Expand Up @@ -105,9 +152,13 @@ and sufficient consent for the event's declared privacy level. It rejects nested
properties such as `prompt` or `error`, and any value that looks like a path, link or email
address. `workers/events/src/eventContract.ts` is generated from
`scient-desktop/packages/scient-analytics/src/wireContract.ts`; do not edit it here. The shared
conformance fixtures (revisions 2 to 10) must keep passing. Since revision 10, new desktop events
conformance fixtures (revisions 2 to 11) must keep passing. Since revision 10, new desktop events
and fields need no gateway change, so the desktop and gateway no longer have to ship in order.

After regenerating a shared fixture, apply this repository's formatter before validation:
`bunx --no-install oxfmt workers/events/fixtures/contract-v11.json`, then `bun run check`.
The desktop generator's JSON layout is not the website's formatting contract.

The public desktop endpoint is disabled unless the Cloudflare runtime variable
`DESKTOP_INGESTION_ENABLED` is exactly `true`. It also requires a random
installation-owned deletion token and applies the configured Cloudflare rate
Expand Down
44 changes: 44 additions & 0 deletions docs/architecture/desktop-downloads.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,50 @@ Each run:
A failed run leaves the previous pointer untouched, so the pointer only names installers
that are in the bucket. Older versions stay in the bucket.

## The update feeds

The same worker publishes the desktop app's generic electron-updater feeds:

| Channel | Feed URL (`generic` provider) | Source |
| ------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| Stable | `https://downloads.scientfactory.com/scient-desktop/updates/stable` | `ScientFactory/scient-desktop`, GitHub's latest release |
| Beta | `https://downloads.scientfactory.com/scient-desktop/updates/beta` | `ScientFactory/scient-desktop-beta`, the newest `-beta.` tag in its releases Atom feed |

Each run, per channel (`workers/release-mirror/src/updateFeed.ts`):

1. Lists the channel's feed objects, recording each version, before asking GitHub anything.
2. Reads and validates the channel's handoff (`parseChannelHandoff`): the channel's version
shape, the official source repository, and that channel's distribution. If every channel
manifest already offers this release (`tag` and the manifest's SHA-256 in its metadata),
the run ends here.
3. Downloads the release's channel manifests (`latest.yml`, `latest-mac.yml`,
`latest-linux.yml`, or the `beta*` set) and accepts them only if size and SHA-256 match the
handoff. The macOS manifest must offer ZIPs and DMGs for both x64 and arm64;
Windows must offer the x64 NSIS EXE and Linux the x64 AppImage. Extra Linux
architecture manifests and their listed packages, including debs, are copied too.
Every file must be a bare file name with the handoff's size, and ZIP/EXE payloads
must have differential blockmaps.
4. Copies every payload, blockmap and manifest into `scient-desktop/<tag>/` with immutable
writes. Payloads also match the manifest's SHA-512 while streaming. An existing object
needs a matching SHA-512 verification receipt or is streamed from R2 and hashed before
reuse; a matching handoff SHA-256 alone is insufficient.
5. Rejects a candidate older than any observed platform manifest, then writes
`scient-desktop/updates/<channel>/<manifest>` last with a compare-and-swap on the
observed version. A stale source cannot regress a later promotion. The feed copy changes
file URLs to `../../<tag>/<name>` and adds release name/notes from the Atom entry;
sizes and SHA-512s remain unchanged.

Nothing is redirected: an old version's objects stay at their own paths, and the updater finds
the previous version's blockmap by substituting its version into the new file's path, which
the version directories satisfy for every release mirrored since the feed began. Two runs that
race can leave platforms on different releases for one interval; each platform's manifest is
self-consistent, and the next run moves the rest.

Caching (C1): versioned objects carry `public, max-age=31536000, immutable`; feed manifests and
`stable.json` carry `no-store`, and clients add a random `noCache` query to every check. A
response for a missing object is not one the worker writes, so the zone's cache rules decide
how long a 404 is kept (see the runbook in the README).

Deploy the worker with `bun run mirror:deploy`. The Pages project reads the bucket through
its `DESKTOP_DOWNLOADS` binding in `wrangler.jsonc`.

Expand Down
76 changes: 66 additions & 10 deletions src/lib/release-handoff.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

import type { Release, ReleaseAsset } from "./release-schema";
import {
DESKTOP_BETA_RELEASE_REPOSITORY,
DESKTOP_RELEASE_REPOSITORY,
desktopMirrorUrl,
isOfficialDesktopProvenanceRepository,
Expand Down Expand Up @@ -59,6 +60,7 @@ export function releaseAssetContentType(name: string): string {
if (name.endsWith(".txt")) return "text/plain";
if (name.endsWith(".json")) return "application/json";
if (name.endsWith(".tgz")) return "application/gzip";
if (name.endsWith(".zip")) return "application/zip";
return "application/octet-stream";
}

Expand All @@ -72,8 +74,68 @@ export function requiredInstallerNames(version: string): readonly string[] {
];
}

export function githubReleaseDownloadUrl(tag: string, name: string): string {
return `https://github.com/${DESKTOP_RELEASE_REPOSITORY}/releases/download/${tag}/${encodeURIComponent(name)}`;
export function githubReleaseDownloadUrl(
tag: string,
name: string,
repository: string = DESKTOP_RELEASE_REPOSITORY,
): string {
return `https://github.com/${repository}/releases/download/${tag}/${encodeURIComponent(name)}`;
}

function parseHandoffAssets(value: unknown): readonly ReleaseHandoffAsset[] {
if (!Array.isArray(value) || value.length === 0 || value.length > 128) {
throw new Error("Release handoff is invalid.");
}
const assets = value.map(parseAsset);
if (new Set(assets.map((asset) => asset.name)).size !== assets.length) {
throw new Error("Release handoff contains duplicate assets.");
}
return assets;
}

const BETA_VERSION = /^\d+\.\d+\.\d+-beta\.\d{8}\.[1-9]\d*$/u;

/**
* A handoff for one update channel: Stable's x.y.z or Beta's x.y.z-beta.YYYYMMDD.N, from the
* official source repository, and, when it names its distribution, distributed on that channel
* from that channel's repository. Unlike {@link parseReleaseHandoff} it requires no website
* installers; the update feed checks the electron-updater manifests instead.
*/
export function parseChannelHandoff(value: unknown, channel: "stable" | "beta"): ReleaseHandoff {
const versionShape = channel === "stable" ? /^\d+\.\d+\.\d+$/u : BETA_VERSION;
const repository =
channel === "stable" ? DESKTOP_RELEASE_REPOSITORY : DESKTOP_BETA_RELEASE_REPOSITORY;
if (
!isRecord(value) ||
value.schemaVersion !== 1 ||
value.product !== "Scient" ||
typeof value.version !== "string" ||
!versionShape.test(value.version) ||
value.tag !== `v${value.version}` ||
!isRecord(value.source) ||
typeof value.source.repository !== "string" ||
!isOfficialDesktopProvenanceRepository(value.source.repository) ||
typeof value.source.commit !== "string" ||
!/^[a-f0-9]{40}$/u.test(value.source.commit)
) {
throw new Error(`The ${channel} release handoff is invalid.`);
}
// Older Stable handoffs predate the distribution record; every Beta handoff carries it.
if (value.distribution !== undefined || channel === "beta") {
const distribution = value.distribution;
if (
!isRecord(distribution) ||
distribution.repository !== repository ||
distribution.channel !== channel
) {
throw new Error(`The ${channel} release handoff names another distribution.`);
}
}
return {
version: value.version,
tag: `v${value.version}`,
assets: parseHandoffAssets(value.assets),
};
}

export function parseReleaseHandoff(value: unknown): ReleaseHandoff {
Expand All @@ -90,18 +152,12 @@ export function parseReleaseHandoff(value: unknown): ReleaseHandoff {
typeof value.source.commit !== "string" ||
!/^[a-f0-9]{40}$/u.test(value.source.commit) ||
typeof value.source.tree !== "string" ||
!/^[a-f0-9]{40}$/u.test(value.source.tree) ||
!Array.isArray(value.assets) ||
value.assets.length === 0 ||
value.assets.length > 128
!/^[a-f0-9]{40}$/u.test(value.source.tree)
) {
throw new Error("Release handoff is invalid.");
}

const assets = value.assets.map(parseAsset);
if (new Set(assets.map((asset) => asset.name)).size !== assets.length) {
throw new Error("Release handoff contains duplicate assets.");
}
const assets = parseHandoffAssets(value.assets);

const names = new Set(assets.map((asset) => asset.name));
if (!requiredInstallerNames(value.version).every((name) => names.has(name))) {
Expand Down
19 changes: 19 additions & 0 deletions src/lib/release-source.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
// Layer: Shared marketing and Cloudflare utility

export const DESKTOP_RELEASE_REPOSITORY = "ScientFactory/scient-desktop";
/** Beta builds are published only here, never in the Stable release index. */
export const DESKTOP_BETA_RELEASE_REPOSITORY = "ScientFactory/scient-desktop-beta";
const TRANSITIONAL_DESKTOP_RELEASE_REPOSITORY = "ScientFactory/scient-desktop-next";

const TRUSTED_PROVENANCE_REPOSITORIES = new Set([
Expand All @@ -27,6 +29,23 @@ export function desktopMirrorKey(tag: string, name: string): string {
return `${DESKTOP_MIRROR_PREFIX}${tag}/${name}`;
}

/**
* The desktop app's generic update feeds. Each channel's electron-updater metadata
* (`latest*.yml` for Stable, `beta*.yml` for Beta) is promoted here only after every file it
* names is in that release's immutable `scient-desktop/<tag>/` directory.
*/
export type DesktopUpdateFeedChannel = "stable" | "beta";
export const DESKTOP_UPDATE_FEED_PREFIX = `${DESKTOP_MIRROR_PREFIX}updates/`;

export function desktopUpdateFeedKey(channel: DesktopUpdateFeedChannel, file: string): string {
return `${DESKTOP_UPDATE_FEED_PREFIX}${channel}/${file}`;
}

/** The URL a desktop client configures as its `generic` provider feed. */
export function desktopUpdateFeedUrl(channel: DesktopUpdateFeedChannel): string {
return `${DESKTOP_MIRROR_ORIGIN}/${DESKTOP_UPDATE_FEED_PREFIX}${channel}`;
}

export function desktopMirrorUrl(tag: string, name: string): string {
return `${DESKTOP_MIRROR_ORIGIN}/${DESKTOP_MIRROR_PREFIX}${tag}/${encodeURIComponent(name)}`;
}
Expand Down
Loading