Epic: DeployedFlow aliases, queue routing, and queue-centric workers

[jumski]

Outcome

Separate logical flow behavior, deployment metadata, queue identity, and worker subscriptions without breaking the current one-flow/one-queue experience.

The completed design provides:

• stable aliases for immutable concrete flow versions;
• a typed DeployedFlow boundary for alias and routing metadata;
• immutable per-task queue snapshots;
• typed static step-to-queue routing;
• queue-centric workers that dispatch several concrete flows.

A downstream integration spike must then show that this removes meaningful project-owned queue and worker plumbing without weakening domain safety.

Why an epic, not a milestone

These issues form one ordered architecture change, but the integration spike still decides whether the feature set is valuable enough for a release commitment.

Do not create a release milestone until the spike records an adopt decision.

Ownership model

Flow
  DAG, handlers, conditions, and execution semantics

DeployedFlow
  stable alias and immutable routing for one concrete version

Postgres flow definition
  concrete slug, alias membership, resolved step queues

Run
  resolved concrete flow slug

Task
  concrete flow slug and immutable queue snapshot

Worker
  one queue subscription and a registry of concrete flows

Defaults preserve current behavior:

alias = flow.slug
step queue = flow.slug
task queue = flow.slug
worker queue = flow.slug

Ordered subissues

Land these in order:

• Remove ControlPlane (pgflow edge function) — startup compilation makes it redundant #647 — Remove the ControlPlane and unsupported alternate compilation path.
• Add stable flow aliases through DeployedFlow startup deployment #648 — Add DeployedFlow and stable alias-first dispatch.
• Persist queue identity on flow steps and tasks #650 — Persist queue identity on steps and tasks; include the requeue_stalled_tasks() uses flow timeout instead of effective step timeout #621 stalled-timeout fix.
• Add typed static queue routing to DeployedFlow #651 — Add typed, immutable static queue routing to DeployedFlow.
• Make flow workers queue-centric with multi-flow dispatch #652 — Make flow workers queue-centric with multi-flow dispatch and drain safety.
• Document the worker-function enable fence for production deployments #654 — Correct the production deployment guide's worker-function enable fence.
• community-threads#27 — Measure the refactor in a real two-stage pipeline.

Architecture issues #647 through #652 land in order. #654 may land independently. Do not implement the SQL foundation, public routing API, and worker registry in one pull request.

Cross-issue invariants

Deployment versions

flow_slug identifies one immutable production deployment version. Shape or routing changes require a new concrete slug.

flow_alias selects the concrete version for new runs. Alias rollback restores that version's original shape and routing.

Queue identity

PGMQ message identity is:

(queue_name, message_id)

Every send, claim, visibility update, retry, archive, skip, recovery, and deletion path must use the queue snapshot stored on the task.

Worker dispatch

One worker polls one queue. It may dispatch several concrete flows from a complete registry.

Every live consumer of one queue must support every (flow_slug, step_slug) pair that can appear there. Keep old concrete handlers until no started old-version run can reach the queue and no executable or recoverable old-version task remains.

Production rollout

For a version that spans several queue worker functions:

1. Record and disable the affected pgflow.worker_functions rows.
2. Deprecate their live workers and wait until all have stopped.
3. Deploy every affected function with both old and new versions while the functions remain disabled.
4. Re-enable only the previously enabled functions.

The first new worker may activate the alias before another queue worker starts. Durable tasks wait for that worker. Do not add explicit normal activation, cross-worker readiness coordination, rejected-worker message metadata, or custom PGMQ filtering.

Configuration classes

Keep these independent:

FlowShape
  DAG and execution semantics

FlowRouting
  immutable placement for one concrete version

RuntimeOptions
  mutable timeout and retry tuning

Do not put physical queue names on .step().

Delivery rules

• Worker startup is the only supported deployment owner after Remove ControlPlane (pgflow edge function) — startup compilation makes it redundant #647.
• Alias activation occurs only after complete shape and routing persistence succeeds.
• Worker readiness does not gate activation; the disable/deprecate/deploy/enable fence prevents incompatible consumers while durable queues hold work during startup recovery.
• Queue provisioning occurs while steps compile, never while runs schedule tasks.
• Flow deletion never drops a potentially shared queue.
• Routing changes use a new concrete version; mutable route cutover is deferred.

Type-quality requirements

The final API must:

• preserve the exact wrapped Flow type;
• autocomplete route step keys from flow steps;
• preserve queue-name literals for worker configuration;
• reject unknown route step keys and unsupported worker queue names;
• preserve condition, dependency, input, output, context, and environment inference.

Do not add queue or alias generic parameters to the fluent Flow chain.

Validation gate

The integration spike must preserve domain tables and transition guards while replacing only queue and worker mechanics.

It must report:

• orchestration code removed and retained;
• direct PGMQ references removed and retained;
• success, retry, crash, ambiguity, and concurrency behavior;
• retry or rate-limit gaps;
• type and autocomplete quality;
• an adopt, revise, or stop recommendation.

An adopt result may justify a release milestone and production hardening. A revise result creates only the missing scoped capabilities. A stop result leaves the features independently usable without forcing the integration.

Acceptance criteria

• All ordered pgflow subissues land in sequence.
• Plain EdgeWorker.start(flow) remains source compatible.
• Alias starts persist and return the resolved concrete slug.
• Shape and routing remain distinct deployment checks.
• Every queue-backed task stores immutable queue identity.
• All SQL queue operations use (queue_name, message_id).
• Named routes compile through DeployedFlow, not .step().
• One queue worker safely dispatches several concrete versions.
• Unsupported-flow work causes no partial claim and remains recoverable.
• Production rollout uses the worker-function enable fence and deploys every affected registry with old and new versions.
• Old and new versions coexist during drain, including versions with started runs that may create later tasks.
• The external integration spike records an explicit recommendation.

Out of scope

• Manual approval or external task completion.
• Mutable routing for an existing concrete version.
• Multi-queue polling inside one worker.
• Automatic queue deletion.
• Runtime input schemas or alias compatibility checks.
• Release scheduling before the integration spike.