Skip to content

ci: run Playwright against the latest Cloud release and its AI image - #664

Merged
appflowy merged 5 commits into
mainfrom
ci/pin-standalone-ai-image
Oct 23, 2026
Merged

appflowy merged 5 commits into
mainfrom
ci/pin-standalone-ai-image

Conversation

@appflowy

@appflowy appflowy commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Every AI chat Playwright spec has failed since 2026-10-22 ~15:00 UTC: auth-chat (chat specs), page (document-sidebar-refresh "creating AI chat", e.g. #663's job) and account-space-user (ai-enabled-flag).

Cause

  • Cloud-Premium #1441 merged the AI service into Cloud, and Cloud now creates the AI schema. Starting with Cloud 0.20.1 (published 2026-10-22 14:50 UTC), appflowyinc/appflowy_ai_premium:latest-amd64 is built from Cloud-Premium.
  • This workflow pinned Cloud to the pre-#1441 custom build ci-web-row-list-9563cf767ef1 but pulled AI as latest-amd64. The new AI crash-looped (Restarting (1)):
    Error: AI database schema is missing (af_ai_registry_bootstrap_state); AppFlowy Cloud applies the shared migrations at startup, so start or upgrade appflowy_cloud against this database first
    
  • Cloud then rejected chat page creation (Connect … http://ai:5001/health → "AI service is temporarily unavailable"), so ai-chat-container never rendered.

Changes

  • Follow the latest Cloud release. Cloud, Worker, Search and AI all pull latest-amd64, and no version is written into the workflow. Each test job reads the source revision behind appflowy_cloud_premium:latest-amd64 (Resolve latest Cloud release) and checks out Cloud-Premium at that revision, so compose files, deploy.env and the seed snapshot match the images. The image-revision check now covers AI too and fails clearly if latest is mixed, for example while a release is still publishing. Today this resolves to 0.20.2 (54fe4d0a42c5).
  • On failure, also save the ai container's logs (test-results/ai-service.log, with the last 80 lines in the job log) and docker compose ps -a. The error above came from this diagnostics step.
  • database-duplicate-cloud: the newer seed snapshot appends Database 1 (Copy) below the visible sidebar, where the spec's forced click hit the footer's Trash button. The spec now opens the copy with openCopiedPage, which falls back to the copy's route.

Known failure until the next Cloud release

list-view-live-flutter-parity › "List Properties shows every supported field type" needs Cloud-Premium #1372 (new List views pick visible fields in stored field order). The old custom build carried it, but no release does yet. Once #1372 ships, this workflow picks it up without changes here.

The same commits are on #661.

🤖 Generated with Claude Code

appflowy and others added 2 commits October 23, 2026 11:29
AI chat specs fail with Cloud reporting only "AI service is temporarily
unavailable" because http://ai:5001 is unreachable; the AI container has
already exited, but its logs were never collected. Save them, plus the
container exit states, before the stack is torn down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cloud-Premium #1441 moved the AI service into Cloud, and from Cloud 0.20.1
the published appflowy_ai_premium:latest-amd64 expects Cloud's shared
migrations to have created its schema. CI pins Cloud to a pre-#1441
revision, so the new AI image exits with "AI database schema is missing
(af_ai_registry_bootstrap_state)", Cloud rejects AI chat creation as
"AI service is temporarily unavailable", and every AI chat spec fails.

Pin the last standalone AI release (0.13.5), which applies its own
migrations, until CLOUD_REVISION moves past #1441.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

The workflow now uses the last standalone AI release instead of a floating image that is incompatible with the pinned pre-migration Cloud revision, preventing AI service crash loops. Failure handling also records container state and AI-specific logs so schema or startup errors are visible during review.

File-Level Changes

Change Details Files
Pin the Playwright test stack to a standalone AI image compatible with the pinned Cloud revision.
  • Replace the floating AI image tag with version 0.13.5-amd64.
  • Document the schema/migration compatibility requirement and when the pin should move to a Cloud-built AI image.
.github/workflows/playwright-test.yml
Improve failure diagnostics for AI service startup and backend test failures.
  • Capture the AI container status and timestamped logs in a dedicated artifact.
  • Print the last 80 AI log lines directly in the job output.
.github/workflows/playwright-test.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Cloud-Premium #1441 moved the AI service into Cloud, which now owns the AI
schema, so the AI image must come from the same revision as Cloud. Move the
backend from the custom ci-web-row-list-9563cf767ef1 build to the 0.20.2
release, take the AI image from CLOUD_VERSION instead of a separate tag,
and verify its revision with the other Rust images.

0.20.2 includes atomic row creation (Cloud-Premium #1375); the List
field-order repair (#1372) that the custom build carried is not released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@appflowy appflowy changed the title ci: pin the AI image to the last standalone release ci: run Playwright against Cloud 0.20.2 and its AI image Oct 23, 2026
appflowy and others added 2 commits October 23, 2026 12:40
With Cloud 0.20.2's larger seeded workspace, "Database 1 (Copy)" is
appended below the visible sidebar, so the spec's forced click landed on
the sidebar footer's Trash button instead. Use openCopiedPage, which
falls back to the copy's own route when the sidebar click misses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pull latest-amd64 for Cloud, Worker, Search and AI, and resolve the
source revision behind that tag at the start of each test job, so the
Cloud-Premium checkout (compose files, deploy.env, seed snapshot)
always matches the images. The image-revision check still rejects a
mixed set, for example while a release is being published.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@appflowy appflowy changed the title ci: run Playwright against Cloud 0.20.2 and its AI image ci: run Playwright against the latest Cloud release and its AI image Oct 23, 2026
@appflowy
appflowy merged commit f930154 into main Oct 23, 2026
20 of 23 checks passed
@appflowy
appflowy deleted the ci/pin-standalone-ai-image branch October 23, 2026 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant