Skip to content

feat: Support async resource dependencies - #6030

Merged
kof merged 28 commits into
mainfrom
agent/async-resource-dependencies-6029
Sep 22, 2026
Merged

kof merged 28 commits into
mainfrom
agent/async-resource-dependencies-6029

Conversation

@kof

@kof kof commented Aug 6, 2026 •

Copy link
Copy Markdown
Member

Summary

Make Resource<Document> the lazy document-resolution abstraction shared by Content Engine graphs, generated sites, and Builder previews. Expressions use one async computation interface: referenced resource documents resolve first, independent requests run concurrently, and unused resources stay unloaded.

Ref #6029
Ref #6031

Implementation checklist

  • Define one resource contract for synchronous or asynchronous document resolution.

  • Resolve explicit roots lazily with memoization, bounded concurrency, cancellation, and structured errors.

  • Preserve cycle, duplicate, and missing-resource diagnostics.

  • Extract resource dependencies with the existing expression AST.

  • Discover consumed resource roots from the current page tree and page metadata.

  • Generate a lazy request graph for published page data, including independent requests.

  • Resolve dependency documents before evaluating dependent request expressions.

  • Integrate dependency-first loading with Builder preview caching and invalidation.

  • Expose safe Resource bindings in the editor and reject self/indirect cycles.

  • Adapt REST, GraphQL, Assets, date, and sitemap requests through the same Resource contract.

  • Cover laziness, chains, shared dependencies, concurrency, invalidation, cancellation, failures, and cycles with fail/pass tests.

  • Review documentation impact and document dependent Resources in the Data variables guide.

  • Track collection-item and component-parameter-dependent resources as follow-up Support per-render scoped Resource dependencies #6031.

  • Regenerate and review affected fixtures.

  • Resolve action-resource request dependencies when an action is invoked.

  • Reproduce and fix cached form submissions and multi-root dependency-cycle hangs.

  • Document uncached form actions and regenerate affected fixtures.

  • Preserve single and multi-selected content until a cut successfully writes to the clipboard.

  • Preserve boolean expression values in the search-exclusion checkbox.

  • Cancel cut deletion if project data changes while preparing or writing the clipboard; cover content/property edits and project switches for single and multi-selection, and document the retry behavior.

  • Validate pending cuts after the final await, immediately before deletion, and reproduce the queued-update race for single and multi-selection.

Technical direction

The persisted, revisioned DocumentGraph remains the Content Engine artifact. Resource<Document> is the execution abstraction for obtaining a document; its resolver evaluates only the dependency closure reachable from requested roots. Static and remote documents therefore differ by Resource implementation, not by expression value type.

Generated pages always emit a ResourceRequestGraph. Each node owns its output name, dependencies, and request factory. Roots come from resource variables actually referenced by the current page's metadata, props, actions, or expression children. Unused and off-page resources remain unresolved. The loader still accepts legacy request maps for compatibility.

Builder preview uses the same page-root discovery as published generation. Its existing cache remains the lifecycle boundary: cached dependency documents unlock dependent requests, while invalidation removes downstream requests until their inputs resolve again. The separate canvas frame recomputes its local plan from the synchronized cache so rendered expressions receive those documents.

Action requests and their dependencies resolve when a form is submitted. Each submission sends a new action request, bypassing Webstudio's resource cache. Dependency requests still use their configured cache lifetime.

Review simplifications

  • Consolidated page resource-root discovery in the SDK and reused it in Builder.
  • Scoped root and action discovery to the current page tree.
  • Removed the redundant immediate-resource helper; Resource.resolve already accepts Document | Promise<Document>.
  • Moved output names onto graph nodes and removed the duplicate graph-level map.
  • Removed unused planner output and corrected transitive cycle discovery to include only Resource data sources.
  • Regenerated fixtures so independent resources also use lazy graph execution.
  • Reused one abort-aware concurrency limiter for document sessions and Resource resolution.
  • Collapsed duplicate generator action scans and removed redundant generated-resource bookkeeping.
  • Unified legacy request maps and request graphs through one resolution path.
  • Shared the 20-request concurrency limit between SDK, Builder preview, and the server batch boundary.
  • Preserved dependency failure paths and original causes, including the first cancellation reason.
  • Removed invalid-request logging that could expose Resource headers or request bodies.
  • Made editor cycle detection alias-safe by comparing underlying Resource IDs.
  • Memoized page settings values to prevent repeated async validation renders.

Verification

Latest local verification for bbca6395e3:

  • Both queued-update regression cases failed before the fix and passed afterward. Validation now runs synchronously immediately before deletion, after the final await, in both cut paths.
  • Passed: 255 copy/paste and command tests, Builder typecheck, lint, package boundaries, formatting, and diff checks.
  • Reviewed documentation: the existing copy/paste safety description remains accurate. Fixture inputs and outputs are unaffected; no fixture regeneration was needed. Agent evaluations were not run because separate approval is required.
  • CI for the new commit is pending.

Previous local verification for ffbc75b152:

  • Six regression cases failed before the fix and passed afterward: content edits, property edits, and project switches during a pending cut, with both single and multiple selections.
  • Passed: 253 copy/paste and command tests, Builder typecheck, lint, package boundaries, source formatting, and diff checks.
  • An unchanged content-block test file initially failed to import in the browser. Its isolated rerun and the full 253-test rerun passed without code changes.
  • The guard conservatively treats any project-data change as a reason to keep the clipboard copy without deleting content. It shows a retry message. Copy/paste documentation now describes this behavior.
  • Fixture inputs and outputs are unaffected; fixtures were not regenerated. Agent evaluations were not run because separate approval is required. CI for the new commit is pending.

Previous local verification for 3eb580697c:

  • Reproduced both regressions with failing tests before fixing them: premature deletion during cut and false search exclusion appearing checked.
  • Passed: 203 copy/paste and page-settings tests, Builder typecheck, lint, package boundaries, formatting, and diff checks.
  • Cut coverage includes keyboard and programmatic entry points, single and multiple selections, delayed successful writes, rejected writes, and disabled copying.
  • Reviewed documentation impact: these fixes restore existing behavior; no documentation update is needed. Only Builder UI and clipboard files changed; fixture inputs and outputs are unaffected, so fixtures were not regenerated for this commit.
  • CI status for this older commit was recorded as pending at handoff. Agent evaluations were not run; they require separate approval.

Previous local verification for 858b348ad, rebased onto cf606ca4c:

  • Regression tests failed before each fix and passed afterward: repeated submissions in both generated route templates, plus three multi-root/branching cycle cases.
  • Passed: 590 SDK tests, 1,129 CLI tests, 72 focused Builder tests, SDK/CLI/Builder typechecks, lint, package boundaries, and fixture link/sync/build.
  • Initial concurrent runs hit timeouts. The complete CLI suite passed with one worker; the focused Builder suites passed without file parallelism.
  • Documentation and generated fixture changes reviewed. Agent evaluations were not run; they require separate approval.
  • CI for 858b348ad passed: Main (including all six E2E shards), Visual Regression, Lint PR, and Check submodules.

Earlier verification (before this update):

TDD fail/pass evidence includes resolver creation, AST dependency discovery, published graph reachability, dependency-gated request generation, Builder dependency-first planning, Resource editor cycle filtering, page-tree root scoping, off-page action isolation, transitive Resource-only cycle discovery, abort/timeout precedence, server batch limits, and the rebased SSG fixture consuming its lazy Resource.

  • pnpm checks passed after rebasing onto ce84ddcf12; it covered workspace tests, typechecks, lint, package boundaries, generated API/docs validation, and fixture link/sync/build.
  • Earlier focused checks passed: Builder typecheck, 12 page-settings/pages unit tests, lint, formatting, and git diff --check.
  • Local generated-resource E2E passed for HTTP, GraphQL, and current-date resources (3 tests).
  • The page-actions and Marketplace page-settings persistence E2Es both pass locally.
  • CI for 6b8b08a70 passed: Main workflow (including all six Builder E2E shards), Visual Regression, Lint PR, and Check submodules.
  • Agent evaluations were not run because they require separate explicit approval.

Current behavior and limitations

  • A failed dependency blocks its dependents and reports the resource path and cause.
  • One published resolution call is the memoization boundary; Builder maps dependency readiness onto its versioned request-cache lifecycle.
  • Scoped collection-item and component-parameter resources remain in Support per-render scoped Resource dependencies #6031.
  • Builder skips cyclic dependency paths without blocking independent resources.

@kof
kof marked this pull request as ready for review August 7, 2026 17:57
@kof
kof force-pushed the agent/async-resource-dependencies-6029 branch 3 times, most recently from 205916b to 3d74b34 Compare August 13, 2026 19:36
@kof kof changed the title experimental: Support async resource dependencies feat: Support async resource dependencies Aug 13, 2026
@kof
kof force-pushed the agent/async-resource-dependencies-6029 branch 2 times, most recently from 2982506 to 57496a9 Compare August 31, 2026 16:44
@kof
kof force-pushed the agent/async-resource-dependencies-6029 branch from db469e4 to c710522 Compare September 18, 2026 15:47
@kof
kof force-pushed the agent/async-resource-dependencies-6029 branch from 2287ffa to 858b348 Compare September 22, 2026 17:21
@kof
kof merged commit 3d0a065 into main Sep 22, 2026
34 checks passed
@kof
kof deleted the agent/async-resource-dependencies-6029 branch September 22, 2026 19:04
kof added a commit that referenced this pull request Sep 23, 2026
New resources can fail to show data in the **New variable** dialog
because a page-plan refresh cancels their preview request before it
finishes.

This change keeps explicitly requested previews alive until the dialog
releases them. Changing inputs, changing resource type, or closing the
dialog cancels obsolete work and prevents old data from showing. Unused
page resources remain lazy; a preview still makes an on-demand request
when **Load data** is clicked. The dialog stays usable while unrelated
page resources are loading.

Follow-up to #6030.

### Checklist
- [x] Reproduce the cancellation with a regression test that fails
before the fix.
- [x] Keep dialog previews independent of page resource discovery, using
the existing loader, cache, and request versions.
- [x] Clear old previews on input changes and release requests on dialog
close.
- [x] Cover new Current date, Sitemap, HTTP, and existing unbound
resources, plus repeated loads and stale responses.
- [x] Keep tests for obsolete page-request cancellation and async
resource dependencies passing.
- [x] Test the unsaved dialog with an unrelated page request pending and
a page-plan recalculation.
- [x] Clear the old preview when switching resource types or starting an
asynchronous body edit.
- [x] Review documentation impact: no update needed; this restores the
documented preview behavior.
- [ ] Confirm GitHub CI passes.

### Verification
- The dialog loading, type-switch, and body-edit regression tests failed
before their fixes and passed afterward.
- 82 focused tests and the settings-panel suite (588 tests) passed.
- Builder typecheck and repository lint passed.
- With preview retention temporarily disabled, all nine new loader
regression cases failed as expected.

This branch was successfully deployed

1 active deployment
development — bbca6395 Deployed Sep 22, 2026 by kof via builder-e2e (shard-2) #21557
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant