fix(sites): keep serve rule exclusions on publish and warn when the edge never confirms the deploy

.changeset/sites-publish-rules.md

@@ -0,0 +1,5 @@
+---
+"@bunny.net/cli": patch
+---
+
+Sites publishes keep paths excluded from the serve rule, warn about Override Origin URL rules the site still shadows, and warn when the edge never confirms the new deploy

AGENTS.md

@@ -561,6 +561,7 @@ Hard-won, not inferable from the code, and expensive to rediscover.
 - **At the edge, `ctx.request.url` is origin-facing** (`http://<edge-ip>:9000/...`), not the requested host. The client hostname must come from the `CDN-Host`/`Host` headers. Hostname routing on `url.hostname` never fires.
 - **Sites must use the edge rule `OriginStorage` action for deploy routing.** Its parameters are storage-zone ID, storage-zone name, and the origin path prefix (`/deploys/<id>/`). A `Change Origin URL` self-hop leaks that internal prefix into root-relative HTML links; pointing it at the Storage API instead loses linked-origin index handling and its injected `AccessKey` is not reliable at the edge. `OriginStorage` changes only the origin path, so the client URL stays clean while Storage authentication and `index.html` handling remain native. `BlockRequest` runs pre-cache, so it applies to cache hits too.
 - **The pull zone delivery layer rewrites `Cache-Control`** from the zone's cache settings; origin- or script-set values reach browsers only when both `CacheControlMaxAgeOverride` and `CacheControlPublicMaxAgeOverride` are `-1`, and a `no-cache` the CDN respects also stops it storing the object. Sites sidesteps this with a zone-level override plus per-extension browser-cache rules.
+- **A storage-origin edge rule always beats an Override Origin URL rule matching the same request**, regardless of creation order or `OrderIndex` (verified live). Routing a path around a site therefore means excluding it from the rewrite rule's match-none trigger, so `ensureSiteRules` carries extra patterns on that trigger through every publish; everything else in the site's rules is rebuilt in full.
 - **Sites edge rules are identified by their `Description` strings** (constants in `sites/constants.ts`); upserts key on them, so treat them as frozen.
 - **An unchanged, already-live `sites deploy` still converges and purges the site's edge rules.** It is the repair/upgrade path for existing sites, so do not return from the no-op branch before `promoteDeploy()`.
 - **The `/storagezone/regions` endpoint is not reliable.** The region catalog is hand-maintained in the storage constants. The available set is a function of both tier and S3 support: Edge (SSD) zones can only be primaried in DE and the create API silently rewrites any other region rather than erroring, so reject a conflicting `--region` client-side.

packages/cli/README.md

@@ -1093,6 +1093,8 @@ Preconfigure the `sites` block in `bunny.jsonc` (`name`, `build`, `dir`, `spa`)
 
 Every deploy publishes: the files land in an immutable `deploys/<id>/` directory and the rewrite rule is pointed at it, so `deployments publish` rolls back to any earlier deploy by moving that pointer, with no files moving and nothing re-uploaded. The ID is the git short-sha when the tree is clean, a content hash otherwise, or whatever `--deploy-id` supplies (letters, digits, `-`, `_`, `.`; 4-64 chars; case-sensitive): a custom ID never aliases onto another deploy's content, and reusing one for different content asks before replacing (`--force` skips the prompt); a replacement clears the old files first, so nothing stale survives. The live deploy and the rollback target are never replaced in place; deploy those under a new ID. Content is root-served, so absolute asset paths work as-is. Direct `/deploys/<id>/` URLs are blocked at the edge. Site state lives at `_bunny/site.json` inside the storage zone (also blocked at the edge); `.bunny/site.json` is only a local pointer, so a fresh clone can `sites link` and pick up where the last machine left off.
 
+**Routing a path to another origin.** An Override Origin URL edge rule never takes priority over the site's storage origin, whatever the rule order. To send a path such as `/api/*` to another backend, add its pattern (`*/api/*`) to the match-none conditions of the `bunny sites: serve the published deploy` rule, then add your own Override Origin URL rule for that path; deploys keep those exclusions. The pull zone caches responses for 30 days and ignores query strings, so give a dynamic route an Override Cache Time of `0` too. The dashboard currently rejects the serve rule's `/deploys/<id>/` path value, so make the edit through the API. Every publish warns about Override Origin URL rules whose paths the serve rule doesn't exclude, checks that the edge serves the new deploy, and warns when it can't confirm it within about 20 seconds.
+
 | Flag                                   | Commands                                                                                 | Description                                                                                                             |
 | -------------------------------------- | ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
 | `--region`, `--domain`                 | `create`                                                                                 | Main storage region code (default `DE`); custom production domain to attach                                             |

packages/cli/src/commands/sites/api.test.ts

@@ -44,8 +44,7 @@ beforeEach(() => {
   promoteVerification.probe = async (url) => ({
     status: 200,
     deploy:
-      new URL(url).searchParams.get("__bunny_promote")?.replace(/-\d+$/, "") ??
-      null,
+      new URL(url).pathname.match(/^\/__bunny_promote\/(.+)-\d+$/)?.[1] ?? null,
   });
   siteFiles.connect = (zone) =>
     ({ zoneName: zone.Name }) as unknown as ReturnType<
@@ -726,6 +725,41 @@ test("promoteDeploy retargets the rewrite rule, probes the edge, and purges twic
   expect(purges[0]?.params).toEqual({ path: { id: 30 } });
 });
 
+test("promoteDeploy keeps paths a user excluded from the rewrite rule", async () => {
+  const coreClient = fakeCoreClient({ calls: [], storageZones: [ZONE] });
+  await promoteDeploy({ coreClient, state: fakeState(), deployId: "aaaa" });
+
+  const rulesOf = async () =>
+    (await coreClient.GET("/pullzone/{id}", { params: { path: { id: 30 } } }))
+      .data?.EdgeRules ?? [];
+  const rewrite = (await rulesOf()).find(
+    (r) => r.Description === REWRITE_RULE_DESC,
+  ) as EdgeRule;
+  await coreClient.POST("/pullzone/{pullZoneId}/edgerules/addOrUpdate", {
+    params: { path: { pullZoneId: 30 } },
+    body: {
+      ...rewrite,
+      Triggers: [
+        {
+          ...rewrite.Triggers?.[0],
+          PatternMatches: ["*/deploys/*", "*/api/*"],
+        },
+      ],
+    },
+  });
+
+  await promoteDeploy({ coreClient, state: fakeState(), deployId: "bbbb" });
+  const after = (await rulesOf()).filter(
+    (r) => r.Description === REWRITE_RULE_DESC,
+  );
+  expect(after).toHaveLength(1);
+  expect(after[0]?.ActionParameter3).toBe("/deploys/bbbb/");
+  expect(after[0]?.Triggers?.[0]?.PatternMatches).toEqual([
+    "*/deploys/*",
+    "*/api/*",
+  ]);
+});
+
 test("promoteDeploy keeps the storage zone's not-found settings on the live deploy", async () => {
   const coreCalls: Call[] = [];
   const coreClient = fakeCoreClient({

packages/cli/src/commands/sites/api.ts

@@ -380,12 +380,14 @@ async function fetchMiddlewareScriptId(
   return id == null || id === NO_MIDDLEWARE_SCRIPT ? null : id;
 }
 
-// The rules that serve a site, rewrite last so a failed upsert never switches routing early; bodies are always rebuilt in full from these so a hand-edited rule heals on the next upsert.
+const DEPLOYS_PATTERN = `*/${DEPLOYS_DIR}/*`;
+
+// The rules that serve a site, rewrite last so a failed upsert never switches routing early; bodies are rebuilt in full so a hand-edited rule heals, except the rewrite rule's extra exclusions.
 function siteRules(
   storageZone: { Id: number; Name: string },
   deployId: string,
+  exclusions: string[] = [],
 ): Array<EdgeRule & { Description: string }> {
-  const deploysPattern = `*/${DEPLOYS_DIR}/*`;
   return [
     {
       Description: GATE_RULE_DESC,
@@ -395,7 +397,7 @@ function siteRules(
       Triggers: [
         {
           Type: EdgeRuleTriggerType.Url,
-          PatternMatches: [deploysPattern],
+          PatternMatches: [DEPLOYS_PATTERN],
           PatternMatchingType: EdgeRuleMatch.Any,
         },
       ],
@@ -445,33 +447,83 @@ function siteRules(
       Triggers: [
         {
           Type: EdgeRuleTriggerType.Url,
-          PatternMatches: [deploysPattern],
+          PatternMatches: [DEPLOYS_PATTERN, ...exclusions],
           PatternMatchingType: EdgeRuleMatch.None,
         },
       ],
     },
   ];
 }
 
+// Paths a user excluded from the rewrite rule (e.g. `*/api/*` routed to another origin); storage-origin rules win over origin-URL rules whatever the order, so this is the only way to route around the site.
+function rewriteExclusions(rules: EdgeRule[]): string[] {
+  const trigger = rules
+    .find((r) => r.Description === REWRITE_RULE_DESC)
+    ?.Triggers?.find(
+      (t) =>
+        t.Type === EdgeRuleTriggerType.Url &&
+        t.PatternMatchingType === EdgeRuleMatch.None,
+    );
+  return (trigger?.PatternMatches ?? []).filter((p) => p !== DEPLOYS_PATTERN);
+}
+
+/** An Override Origin URL rule whose URL patterns the serve rule still matches, so the site answers instead. */
+export interface ShadowedRule {
+  name: string;
+  patterns: string[];
+}
+
+function shadowedOriginRules(
+  rules: EdgeRule[],
+  exclusions: string[],
+): ShadowedRule[] {
+  const shadowed: ShadowedRule[] = [];
+  for (const rule of rules) {
+    if (rule.Enabled === false) continue;
+    const routesOrigin =
+      rule.ActionType === EdgeRuleAction.OriginUrl ||
+      rule.ExtraActions?.some((a) => a.ActionType === EdgeRuleAction.OriginUrl);
+    if (!routesOrigin) continue;
+    const patterns = (rule.Triggers ?? [])
+      .filter(
+        (t) =>
+          t.Type === EdgeRuleTriggerType.Url &&
+          t.PatternMatchingType !== EdgeRuleMatch.None,
+      )
+      .flatMap((t) => t.PatternMatches ?? [])
+      .filter((p) => !exclusions.includes(p));
+    if (patterns.length > 0) {
+      shadowed.push({
+        name: rule.Description || rule.Guid || "unnamed",
+        patterns,
+      });
+    }
+  }
+  return shadowed;
+}
+
 // Converge the pull zone's rules on `deployId`; create and promote both funnel through here, so a missing or stale rule heals on any run.
 async function ensureSiteRules(opts: {
   coreClient: CoreClient;
   pullZoneId: number;
   storageZone: StorageZoneModel;
   deployId: string;
-}): Promise<void> {
+}): Promise<ShadowedRule[]> {
   const { coreClient, pullZoneId, storageZone, deployId } = opts;
   const { Id: storageZoneId, Name: storageZoneName } = storageZone;
   if (storageZoneId == null || !storageZoneName) {
     throw new UserError("The site's storage zone is missing its ID or name.");
   }
   const existing = await fetchEdgeRules(coreClient, pullZoneId);
+  const exclusions = rewriteExclusions(existing);
   for (const rule of siteRules(
     { Id: storageZoneId, Name: storageZoneName },
     deployId,
+    exclusions,
   )) {
     await upsertEdgeRule(coreClient, pullZoneId, rule, existing);
   }
+  return shadowedOriginRules(existing, exclusions);
 }
 
 interface CreateSiteOptions {
@@ -833,17 +885,21 @@ export const promoteVerification = {
 async function waitForEdgePropagation(
   host: string,
   deployId: string,
-): Promise<void> {
+): Promise<boolean> {
   const start = Date.now();
   const deadline = start + PROPAGATION_DEADLINE_MS;
   let attempt = 0;
+  let confirmed = false;
   while (Date.now() < deadline) {
     try {
-      // A unique query per attempt keeps each probe out of the CDN cache so a stale entry can't mask a propagated rule.
+      // A unique path per attempt keeps each probe out of the CDN cache (the zone ignores query strings) so a stale entry can't mask a propagated rule.
       const { deploy } = await promoteVerification.probe(
-        `https://${host}/?__bunny_promote=${deployId}-${attempt++}`,
+        `https://${host}/__bunny_promote/${deployId}-${attempt++}`,
       );
-      if (deploy === deployId) break;
+      if (deploy === deployId) {
+        confirmed = true;
+        break;
+      }
     } catch {
       // Edge briefly unreachable (DNS/warmup); keep trying until the deadline.
     }
@@ -854,6 +910,7 @@ async function waitForEdgePropagation(
   if (elapsed < SETTLE_FLOOR_MS) {
     await promoteVerification.wait(SETTLE_FLOOR_MS - elapsed);
   }
+  return confirmed;
 }
 
 async function ensureNotFoundSettings(
@@ -874,12 +931,32 @@ async function ensureNotFoundSettings(
   });
 }
 
-// Point production at a deploy: retarget the rewrite rule and purge. The rule propagates async, so purge, wait for the edge to serve it, then purge again so nothing stale survives (the header can confirm on a cached response, which is why the second purge is load-bearing).
+export interface PromoteResult {
+  confirmed: boolean;
+  shadowed: ShadowedRule[];
+}
+
+export function warnAfterPromote(
+  result: PromoteResult,
+  deployId: string,
+): void {
+  for (const { name, patterns } of result.shadowed) {
+    logger.warn(
+      `The edge rule "${name}" routes ${patterns.join(", ")} to another origin, but the site's serve rule still matches it, so the site answers instead. Exclude it from the "${REWRITE_RULE_DESC}" rule.`,
+    );
+  }
+  if (result.confirmed) return;
+  logger.warn(
+    `Couldn't confirm the edge is serving ${deployId}. Check the ${DEPLOY_HEADER} header on the live site; another storage-origin edge rule on the pull zone can keep a different deploy live.`,
+  );
+}
+
+// Point production at a deploy: retarget the rewrite rule and purge; resolves to whether the edge confirmed it and which origin rules the site shadows. The rule propagates async, so purge, wait for the edge to serve it, then purge again so nothing stale survives (the header can confirm on a cached response, which is why the second purge is load-bearing).
 export async function promoteDeploy(opts: {
   coreClient: CoreClient;
   state: RemoteSiteState;
   deployId: string;
-}): Promise<void> {
+}): Promise<PromoteResult> {
   const { coreClient, state, deployId } = opts;
   const purge = () =>
     coreClient.POST("/pullzone/{id}/purgeCache", {
@@ -901,16 +978,17 @@ export async function promoteDeploy(opts: {
   if (!state.current) {
     await applySiteCacheSettings(coreClient, state.pullZoneId);
   }
-  await ensureSiteRules({
+  const shadowed = await ensureSiteRules({
     coreClient,
     pullZoneId: state.pullZoneId,
     storageZone,
     deployId,
   });
   await ensureNotFoundSettings(coreClient, storageZone, state, deployId);
   await purge();
-  await waitForEdgePropagation(host, deployId);
+  const confirmed = await waitForEdgePropagation(host, deployId);
   await purge();
+  return { confirmed, shadowed };
 }
 
 // Refuse to replace version-1 state that changed since it was read. `writeRemoteState`'s own conflict merge can't cover this: it reconciles against the current format, and the file being replaced is the older one, so it would abort as unparseable rather than merge.
@@ -950,6 +1028,7 @@ export interface MigrateResult {
   detachedScriptId: number | null;
   deletedScriptId: number | null;
   scriptError?: string;
+  promoted?: PromoteResult;
 }
 export async function migrateSite(opts: {
   coreClient: CoreClient;
@@ -983,9 +1062,14 @@ export async function migrateSite(opts: {
   });
   await applySiteCacheSettings(coreClient, state.pullZoneId);
 
+  let promoted: PromoteResult | undefined;
   if (state.current) {
     step("Publishing the current deploy...");
-    await promoteDeploy({ coreClient, state, deployId: state.current });
+    promoted = await promoteDeploy({
+      coreClient,
+      state,
+      deployId: state.current,
+    });
   }
 
   // Committed only once every fallible remote step is done: while the file still reads as version 1, a failed run above is fully resumable.
@@ -1012,7 +1096,13 @@ export async function migrateSite(opts: {
     }
   }
 
-  return { state, detachedScriptId, deletedScriptId, scriptError };
+  return {
+    state,
+    detachedScriptId,
+    deletedScriptId,
+    scriptError,
+    promoted,
+  };
 }
 
 interface TeardownResult {

packages/cli/src/commands/sites/deploy.ts

@@ -21,6 +21,7 @@ import {
   fetchSystemHostname,
   promoteDeploy,
   rereadRemoteState,
+  warnAfterPromote,
   writeRemoteState,
 } from "./api.ts";
 import {
@@ -460,17 +461,18 @@ export const sitesDeployCommand = defineCommand<DeployArgs>({
     const production = productionUrl(state, systemHost);
 
     if (skipUpload && alreadyLive) {
-      await withSpinner("Checking routing...", () =>
+      const promoted = await withSpinner("Checking routing...", () =>
         promoteDeploy({ coreClient, state, deployId }),
       );
+      warnAfterPromote(promoted, deployId);
       if (output === "json") {
         logger.log(
           JSON.stringify(
             {
               site: state.name,
               id: deployId,
               unchanged: true,
-              live: true,
+              live: promoted.confirmed,
               production: production ?? null,
             },
             null,
@@ -541,13 +543,18 @@ export const sitesDeployCommand = defineCommand<DeployArgs>({
       etag = await writeRemoteState(connection, state, etag);
     }
 
-    await withSpinner("Publishing to production...", async () => {
-      await promoteDeploy({ coreClient, state, deployId });
-      markCurrent(state, deployId);
-      etag = await writeRemoteState(connection, state, etag, {
-        promotedTo: deployId,
-      });
-    });
+    const promoted = await withSpinner(
+      "Publishing to production...",
+      async () => {
+        const result = await promoteDeploy({ coreClient, state, deployId });
+        markCurrent(state, deployId);
+        etag = await writeRemoteState(connection, state, etag, {
+          promotedTo: deployId,
+        });
+        return result;
+      },
+    );
+    warnAfterPromote(promoted, deployId);
 
     if (output === "json") {
       logger.log(
@@ -559,7 +566,7 @@ export const sitesDeployCommand = defineCommand<DeployArgs>({
             files: files.length,
             bytes: totalBytes,
             unchanged: skipUpload,
-            live: true,
+            live: promoted.confirmed,
             production: production ?? null,
           },
           null,
@@ -570,7 +577,11 @@ export const sitesDeployCommand = defineCommand<DeployArgs>({
     }
 
     if (skipUpload) {
-      logger.success(`Deploy ${deployId} is now live.`);
+      logger.success(
+        promoted.confirmed
+          ? `Deploy ${deployId} is now live.`
+          : `Published deploy ${deployId}.`,
+      );
     } else {
       logger.success(
         `Deployed ${deployId} (${files.length} files, ${formatBytes(totalBytes)}).`,