From 08cdb3b6696ae78a5f26f8ade7f75a0561b7e143 Mon Sep 17 00:00:00 2001 From: M2Night Date: Fri, 30 Oct 2026 14:22:17 +0800 Subject: [PATCH] Self-hosting docs: a Models page with Drama 3, and a shorter overview The self-hosting pages assumed one model. Drama 3 now ships self-hosted too. A new Models page puts the models side by side: the request header and the GPUs each takes. Drama 3 is described in broad strokes, since its deployment guide carries the sizing. The other pages keep what applies to every model. The pages are also cut to an overview. Detail is left to the deployment guide each customer downloads: - The introduction has three sections: Platforms, Billing and Supported APIs. - The Releases page, the troubleshooting table, the architecture component list and the step-by-step dashboard walkthrough are gone. - The Kubernetes and All-in-One pages link to the introduction's billing table instead of repeating it. Registry & license follows the Self Host page. A reader test fixed undefined terms, inconsistent names, and a token rotation that left a gap. Part of FISH-1723 Co-Authored-By: Claude Opus 5.5 (1M context) --- developer-guide/self-hosting/air-gapped.mdx | 52 ++---- developer-guide/self-hosting/all-in-one.mdx | 98 ++++------- .../self-hosting/enterprise-releases.mdx | 44 ----- developer-guide/self-hosting/introduction.mdx | 154 +++++++----------- developer-guide/self-hosting/kubernetes.mdx | 110 +++++-------- developer-guide/self-hosting/models.mdx | 32 ++++ developer-guide/self-hosting/operations.mdx | 88 +++------- .../self-hosting/registry-access.mdx | 111 ++++--------- developer-guide/self-hosting/requirements.mdx | 15 +- docs.json | 8 +- 10 files changed, 246 insertions(+), 466 deletions(-) delete mode 100644 developer-guide/self-hosting/enterprise-releases.mdx create mode 100644 developer-guide/self-hosting/models.mdx diff --git a/developer-guide/self-hosting/air-gapped.mdx b/developer-guide/self-hosting/air-gapped.mdx index a2274959..22e62234 100644 --- a/developer-guide/self-hosting/air-gapped.mdx +++ b/developer-guide/self-hosting/air-gapped.mdx @@ -4,19 +4,17 @@ description: "Running Fish Audio Enterprise on a network that cannot reach Fish icon: "shield-halved" --- -Both offline modes make no outbound calls at runtime. Neither one, however, -installs itself out of thin air: a Helm install pulls container images and the chart -from a registry, and the appliance needs its image on the host. An air-gapped -deployment is about getting those artifacts across the boundary. The license bundle -has to cross it too, and unlike an image it expires — plan a recurring transfer, not -a one-off. The reference voices you use have to cross too, copied in as archives: -voice sync cannot reach Fish Audio from there. +Both offline modes need no outbound access at runtime, but installing them still +takes artifacts from Fish Audio: the images and, on Helm, the chart. An air-gapped +deployment is about getting those across the boundary, together with the license +bundle and the reference voices you use. The license expires, so plan a recurring +transfer, not a one-off; voices are copied in as archives, since voice sync cannot +reach Fish Audio from there. Fish Audio does this with you as part of an air-gapped delivery, and the step-by-step - procedure is in the deployment runbook for the version you install. See - [Releases](/developer-guide/self-hosting/enterprise-releases). This page is what to - expect and what to plan for. + procedure is in the deployment guide for the version you install. This page is + what to expect and what to plan for. ## Which one to choose @@ -25,41 +23,27 @@ voice sync cannot reach Fish Audio from there. gap: one image, moved to the disconnected host as a file and loaded there. Beyond the image, the license bundle — and, in the external-weights modes, the weight fileset — has to be carried across. Its online counterpart cannot run air -gapped: it authorises every request against Fish Audio. +gapped: it authorizes every request against Fish Audio. **The Helm chart** is more work, because the release is many images rather than one. Every image has to be mirrored into a registry the cluster can reach and the release pointed at it, component by component. Plan for the whole set, and expect to do it with your account team rather than alone. -**What you give up either way:** images are pinned to the exact content Fish Audio -published, so a deployment cannot quietly run anything else. Copying an image into -another registry drops that pin, so check what you mirrored while both sides are still -there to compare. - ## Offline usage accounting -An offline deployment cannot call a billing service, so it records usage locally -instead, in a signed ledger on persistent storage. It is durable data rather than -cache: include it in your backups, and agree a reconciliation cadence with your account -team. Usage is settled from the ledger files themselves, not from totals compiled off -them. - -Because there is no service to validate a token against, any non-empty bearer token is -accepted and recorded as the billing identity. Use a stable, distinct token per tenant. +Offline billing records usage in a signed ledger on persistent storage. It is durable +data, not cache: back it up, and agree with your account team how often to send it for +reconciliation. ## Proving there is no egress -Regulated deployments usually need evidence rather than a configuration review, and the -runbook carries the exercise. Two things are worth knowing before you plan it. - -It answers two questions, and they need different methods: whether the deployment -*needs* the internet, and whether it *calls out* when allowed to. The second is what a -telemetry or data-residency review actually asks. - -It is also a cluster-level exercise rather than a namespace one. Confirm early that -whoever runs your cluster can take part, because a namespace-scoped account cannot -complete it. +Regulated deployments usually need evidence rather than a configuration review, and +your deployment guide carries the exercise. It shows both that the deployment works +without the internet and that, with voice sync off, it makes no calls when the internet +is available. On Helm +it is a cluster-level exercise, so involve whoever runs your cluster early: a +namespace-scoped account cannot complete it. ## Next steps diff --git a/developer-guide/self-hosting/all-in-one.mdx b/developer-guide/self-hosting/all-in-one.mdx index 5bde7059..6dabe8e4 100644 --- a/developer-guide/self-hosting/all-in-one.mdx +++ b/developer-guide/self-hosting/all-in-one.mdx @@ -1,34 +1,29 @@ --- title: "All-in-One container" -description: "The single-container appliance: what it is, what it cannot do, and what running it involves" +description: "Run a model's whole speech stack in one Docker container on a single machine" icon: "box" --- -The All-in-One image packages the whole speech stack (edge API, model API layer, -inference router and worker, vocoder, text normalizer, and Redis) into one container, -with the model weights baked into the image, except in the external-weights modes. It runs as a turnkey single-node appliance, no Kubernetes. +The All-in-One image packages the whole speech stack into one container and runs as a +turnkey single-node appliance, no Kubernetes. The model weights are baked into the +image, except in the external-weights modes, which mount them from your own +storage at run time. -The two billing modes, **Offline All-in-One** and **Online All-in-One**, differ only in how -usage is accounted for. The offline mode records usage to a local -ledger and needs no network once the image is on the host. The online mode authorises -each request against Fish Audio and charges the key that made it, so it needs outbound -access and a key with credit. The accounting is compiled in, so an image is one or the -other; the container logs which at startup. +Each image is built for one [billing mode](/developer-guide/self-hosting/introduction#billing), +online or offline; the container logs which at startup. This page covers what the appliance is and what to plan for. The commands, tuning - options, and troubleshooting are in the **All-in-One guide**, which ships in the - deployment guide download for the image version you run. See - [Releases](/developer-guide/self-hosting/enterprise-releases). + options, and troubleshooting are in the **All-in-One guide**, which you download from + **Developer → Self Host** with the image version you run. ## What it cannot do -The appliance runs one inference worker and one vocoder — a GPU each; the -Single-GPU special case shares one H100-class 80 GB card, with lower throughput. -It does not -autoscale, does not shard across more GPUs or nodes, and does not ship the forced -aligner, so it returns no word or segment timings. For elastic or higher-throughput +The appliance runs one inference worker and one vocoder, on the cards its model +takes — see [Models](/developer-guide/self-hosting/models). It does not +autoscale or add workers on more GPUs or nodes, and it does not include timestamp +alignment, so it returns no word or segment timings. For elastic or higher-throughput deployments, use the [Kubernetes chart](/developer-guide/self-hosting/kubernetes), which scales replicas across all GPUs and nodes. @@ -38,61 +33,40 @@ One `docker run` on a host that meets the [requirements](/developer-guide/self-hosting/requirements). It needs: -- **Two GPUs** — one for the inference worker, one for the vocoder. The Single-GPU - mode is a special case: one H100-class 80 GB card shared by both, with lower - throughput. +- **The GPUs its model takes** — usually two; a Single-GPU mode, where the model offers + one, shares one card at lower throughput. - **One exposed port** for the API. -- **One persistent volume.** Compile caches, the vocoder's built engine, reference - voice archives, and — in the offline mode — the usage ledger all live there. Model - weights are in the image, not on the volume; the external-weights modes mount - them at run time instead — the delivered guide covers that mount. -- **The license bundle.** A host directory holding the certificate and key pair from - **Developer → Self Host**, mounted read-only. Without it the API never comes up; once - the certificate expires the container starts but refuses requests. Renewing is - replacing those files in place; only a bundle that carries a new key needs the - container restarted. +- **One persistent volume.** Caches, reference voice archives, and — in the offline + mode — the [usage ledger](/developer-guide/self-hosting/air-gapped#offline-usage-accounting) + all live there; the model weights do not. +- **The license bundle** from **Developer → Self Host**, mounted read-only. See + [The license](/developer-guide/self-hosting/registry-access#the-license). Everything inside the container runs as a non-root user, so a reused volume or a host bind mount has to be writable by it. -**Plan for a slow first start.** The worker compiles its inference graphs and the -vocoder builds its engine before either serves, and the health endpoint verifies the -speech backend end to end rather than reporting immediate liveness. Both artifacts are -cached on the volume, so later starts take minutes. The vocoder engine is specific to -the GPU model, so moving to different cards rebuilds it once. +**Plan for a slow first start.** The worker and the vocoder prepare for your GPU type +before either serves, and the health endpoint verifies the speech backend end to end +rather than reporting immediate liveness. Every start takes minutes; the first can take +much longer. -## Usage accounting and tenancy +To upgrade, pull or load the new version's image and recreate the container against +the same volume. -The billing mode determines how usage is accounted for and what the bearer token has to be. +## Tokens and tenancy -**Offline.** Usage goes to a local, signed, append-only ledger on the volume. Two -consequences worth designing around: - -- **Any non-empty bearer token is accepted**; a missing or empty one is rejected. The - appliance has nothing to validate a token against. -- **The token is recorded verbatim as the billing identity.** Use a stable, distinct - token per tenant. Two tenants sharing a token are indistinguishable in the ledger. - -Backups and reconciliation: [Offline usage accounting](/developer-guide/self-hosting/air-gapped#offline-usage-accounting). - -**Online.** Each request is authorised against Fish Audio and charged to the key that -made it. The token must be a [Fish Audio API key](/developer-guide/getting-started/api-key) -with credit — not the deploy token used to pull the image — and anything else is -rejected. No ledger is written; usage appears in your Fish Audio account instead. - -Authorisation happens on every request, not only at startup, and the appliance has no -local fallback. Treat outbound reachability as a production dependency. +Offline, any non-empty bearer token is accepted and recorded as the billing identity: +give each tenant a stable token of its own, and control access in front of the +appliance. Online, each request must carry a +[Fish Audio API key](/developer-guide/getting-started/api-key) with credit, not the +deploy token. It is authorized against Fish Audio with no local fallback, so treat +outbound reachability as a production dependency. ## Capacity -The single worker admits a bounded number of in-flight requests; beyond that, requests -queue and time-to-first-audio climbs. The ceiling is set by the worker's key-value -cache VRAM, so larger cards support a higher cap. It is adjustable at launch without -rebuilding the image. Validate latency and error rate at any new value before -committing to it. - -Reference-id requests resolve only from local archives on the volume, one zip per -voice, placed by you or by the optional voice sync. +The worker admits a bounded number of requests at once; beyond that, requests queue and +time-to-first-audio climbs. Your deployment guide gives the cap for your model; validate +latency and error rate before changing it. ## Next steps diff --git a/developer-guide/self-hosting/enterprise-releases.mdx b/developer-guide/self-hosting/enterprise-releases.mdx deleted file mode 100644 index 20a60c6a..00000000 --- a/developer-guide/self-hosting/enterprise-releases.mdx +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: "Releases" -description: "How Fish Audio Enterprise versions are identified and how to move between them" -icon: "code-branch" ---- - -Fish Audio publishes a version for each mode once it has been tested. -**Developer → Self Host** lists the versions available to your team and fills the -one you pick into the install commands on that page. - -## Version identifiers - -| Platform | Identified by | Applied with | -| ----------- | --------------- | ------------------------------------ | -| Helm | A chart version | `--version` on `helm pull`/`install` | -| All-in-One | An image tag | The tag in the image reference | - -Both Helm profiles are one chart, so they share a version. Which profile you deploy is -decided by the values file, not by the version. Each All-in-One mode — the -combination of billing, GPU topology, and how weights are delivered — is released -and tagged independently; **Developer → Self Host** shows the ones your team -is granted. - -## Upgrading - -A deployment stays on its version until you change it. Take the new version from -**Developer → Self Host**, then follow the upgrade -procedure in the deployment runbook for the Helm platform, or pull the new tag and -recreate the container against the same volume for -[All-in-One](/developer-guide/self-hosting/all-in-one). Air-gapped deployments -mirror the new version first. See -[Air-gapped deployments](/developer-guide/self-hosting/air-gapped). - - - Talk to your account team before skipping versions, and before upgrading a - production deployment, so the change can be checked against how yours is - configured. - - -## Next step - -Return to [Kubernetes deployment](/developer-guide/self-hosting/kubernetes) or -[All-in-One container](/developer-guide/self-hosting/all-in-one) to apply the -version you picked. diff --git a/developer-guide/self-hosting/introduction.mdx b/developer-guide/self-hosting/introduction.mdx index ca37d477..d7b4bf22 100644 --- a/developer-guide/self-hosting/introduction.mdx +++ b/developer-guide/self-hosting/introduction.mdx @@ -6,8 +6,8 @@ icon: "server" Fish Audio Enterprise can be deployed into infrastructure you control: your own cloud account, an on-premise data center, or a network with no internet access at -all. Fish Audio delivers container images and a Helm chart through a private -registry; you own the cluster, the network boundary, and the data. +all. Fish Audio delivers each model's container images and Helm chart through a +private registry; you own the cluster, the network boundary, and the data. The self-hosted stack runs the same speech engine as the hosted API, so requests, audio formats, and voice behavior match what you already build against. @@ -18,90 +18,51 @@ audio formats, and voice behavior match what you already build against. | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | | Colocation | Run inference in the same region, VPC, or rack as your application and remove public-internet round trips from time-to-first-audio. | | Single-tenant isolation | Dedicated GPUs and queues. Capacity is not shared with other tenants, and you decide when the deployment is upgraded. | -| Security posture | Voice traffic never leaves your network. The offline modes make no outbound calls at runtime and run on disconnected networks. | +| Security posture | Voice traffic never leaves your network. The offline modes need no outbound access at runtime and run on disconnected networks. | | Data sovereignty | Input text, generated audio, and reference voices stay inside your boundary and under your own retention policy. | -## Platforms and modes - -The same engine ships on two platforms. Your enterprise agreement determines -which platforms — and which modes of each — your team is granted. - -| | Helm (Kubernetes) | All-in-One (Docker) | -| -------- | ------------------------------------------------------------------------------ | ------------------------------------------------------ | -| Shape | The chart on your cluster: replicas across GPUs and nodes, rolling upgrades. | One container on a single machine; the smallest footprint. | -| Best for | Production clusters and elastic capacity. | Evaluation, fixed-size appliances, strict air gaps. | - -A **mode** is a combination of the axes below; on Helm only billing is a mode — -the other axes are cluster configuration. **Developer → Self Host** shows the -modes your team can install. - -**Online** billing authorizes and bills each request against Fish Audio and -needs outbound access; **Offline** records usage to a local signed ledger and -runs air-gapped. - -| Axis | All-in-One | Helm | -| -------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -| Billing | Online or Offline. | The same two, as deployment profiles of one chart — a values file selects the profile. | -| Topology | **Dual-GPU** — worker and vocoder on a card each. **Single-GPU** — a special case: both share one H100-class 80 GB card, with lower throughput. | Not a mode — each component takes a whole GPU; you add replicas to grow. | -| Weights | **Baked** into the image, or **external** — mounted at run time. | In-cluster by default; the online profile can instead pull from S3-compatible storage — see [Kubernetes deployment](/developer-guide/self-hosting/kubernetes). | - -The offline modes are air-gap capable — the Helm profile after mirroring images -and the chart into your own registry. - -Choose the platform and mode that fit your constraints, then agree them with -your Fish Audio account team — your agreement determines what **Developer → -Self Host** offers you. Combinations beyond this standard set can be discussed -there too. - -Both Helm profiles share one deployment procedure and differ only in a few values. -See [Kubernetes deployment](/developer-guide/self-hosting/kubernetes) for the -Helm platform and [All-in-One container](/developer-guide/self-hosting/all-in-one) -for the single-container platform. - -## What is included - -| Capability | Online Helm | Offline Helm | All-in-One (all modes) | -| ----------------------------------------- | ---------------------------- | ------------------------ | ------------------------ | -| Text to speech over `POST /v1/tts` | Included | Included | Included | -| WebSocket streaming | Included | Included | Included | -| Reference-voice requests (`reference_id`) | From local archives | From local archives | From local archives | -| Word and segment timestamps | Optional, needs an extra GPU | Optional, needs an extra GPU | Not included | -| Horizontal scaling and autoscaling | Included | Included | Not included | -| Prometheus metrics | Included | Included | Edge API only, at `/metrics` | - - - Timestamp alignment is served by a separate forced-aligner component. It is - disabled by default and is not built into the All-in-One image, so - `/v1/tts/stream/with-timestamp` returns audio without alignment data there. - Offline Helm needs its model staged on shared storage first; the deployment - runbook has the step. Contact Fish Audio if your deployment needs timestamps. - - -Reference voices are resolved from local archives, which you stage yourself or the -optional voice sync downloads: your team's own fish.audio voices and Fish Audio's -official voices. A self-hosted deployment never reads from the hosted voice library -at request time. Products other than text to -speech are hosted-only unless your agreement says otherwise. - -## Architecture at a glance - -The table describes the Helm deployment; the All-in-One runs the same components -inside one container. - -| Component | Role | -| ------------------ | ---------------------------------------------------------------------------------------------- | -| Edge API | Entry point. Validates requests, applies product logic, and routes work to the model services. | -| Model API layer | Coordinates normalization, inference, and audio decoding. | -| Inference router | Distributes work across the GPU workers. | -| Inference workers | GPU-backed generation. | -| Vocoder | GPU-backed audio decoding. | -| Text normalizer | Text normalization ahead of inference. | -| Redis | Runtime state and cache. Installed by the chart into the release namespace. | -| Shared storage | Model cache, reference archives, and the usage ledger. Mounted on every node. | - -Fish Audio delivers the deployment at the Kubernetes service level. Ingress, DNS, -TLS, external load balancing, and network allowlists are yours to choose and -configure. +## Platforms + +Each model ships on two platforms. Your agreement decides which ones your team gets, +and in which modes; **Developer → Self Host** shows them. + +| | Helm (Kubernetes) | All-in-One (Docker) | +| ---------- | ----------------------------------------------------------------- | ---------------------------------------------------------------- | +| What it is | The model's chart, installed on your cluster | One container on a single machine | +| Best for | Production clusters and elastic capacity | Evaluation, fixed-size appliances, strict air gaps | +| Scaling | Replicas across GPUs and nodes, with optional autoscaling | None: one worker and one vocoder | +| Metrics | Prometheus metrics from the speech services | API metrics only, at `/metrics` | +| Timestamps | Optional, where available | Not included | +| Guide | [Kubernetes deployment](/developer-guide/self-hosting/kubernetes) | [All-in-One container](/developer-guide/self-hosting/all-in-one) | + +The GPUs each model needs are on [Models](/developer-guide/self-hosting/models). On +either platform, ingress, DNS, TLS, external load balancing, and network allowlists +are yours to choose and configure. + +## Billing + +Both platforms come in either billing mode: + +| | Online | Offline | +| ------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | +| Usage | Authorized and billed per request against Fish Audio | Recorded in a signed ledger on your side and reconciled with Fish Audio | +| Network needed at runtime | Outbound HTTPS to Fish Audio | None, so it can run air-gapped — see [Air-gapped deployments](/developer-guide/self-hosting/air-gapped) | +| Request token | A [Fish Audio API key](/developer-guide/getting-started/api-key) with credit | Any stable, non-empty token, one per tenant, recorded as the billing identity and not checked: control access in front of the deployment | + +## Supported APIs + +A self-hosted deployment serves: + +- **Text to speech** over [HTTP](/api-reference/endpoint/openapi-v1/text-to-speech) + and [WebSocket streaming](/api-reference/endpoint/websocket/tts-live). +- **Reference voices.** A request that names a voice by `reference_id` uses that + voice's archive on the deployment. Your team's fish.audio voices and Fish Audio's + official voices get there by hand, or over HTTPS through the optional voice sync; + the hosted voice library is never read at request time. +- **[Word and segment timestamps](/api-reference/endpoint/openapi-v1/text-to-speech-stream-with-timestamps)**, + optional. They need one more GPU; contact Fish Audio to enable them. + +Other Fish Audio products are hosted-only unless your agreement says otherwise. ## Getting access @@ -112,13 +73,12 @@ configure. target, expected concurrency, and whether you need air-gapped operation. - Once your agreement is in place, Fish Audio grants your team the artifacts - it is entitled to: the Helm chart, the component images, and the All-in-One - image where applicable. + Once your agreement is in place, Fish Audio grants your team the models and + modes it is entitled to, and the charts and images that go with them. Sign in to fish.audio and open **Developer → Self Host** to see the - platforms and modes your team is granted and to create the token that + models, platforms, and modes your team is granted and to create the token that authenticates against the registry. See [Registry & license](/developer-guide/self-hosting/registry-access). @@ -127,6 +87,13 @@ configure. ## Next steps + + The models you can self-host, and where their requirements differ. + - Install, upgrade, roll back, and validate the Helm release. + What the Helm install involves and what to arrange first. - Monitoring signals, scaling, backups, and troubleshooting. - - - How versions are published, and which one to deploy. + Monitoring signals, scaling, backups, and upgrades. diff --git a/developer-guide/self-hosting/kubernetes.mdx b/developer-guide/self-hosting/kubernetes.mdx index af996b88..d1dbd0ef 100644 --- a/developer-guide/self-hosting/kubernetes.mdx +++ b/developer-guide/self-hosting/kubernetes.mdx @@ -1,89 +1,55 @@ --- title: "Kubernetes deployment" -description: "What the Helm delivery installs, what you decide before installing, and what to expect" +description: "Install a model's speech stack on your own Kubernetes cluster with its Helm chart" icon: "dharmachakra" --- -Fish Audio Enterprise ships as a Helm chart that installs the whole speech stack into -a cluster you run. Both Helm profiles use the same chart and the same procedure; -they differ only in how usage is accounted. +Each model ships as its own Helm chart, which installs the whole speech stack into a +cluster you run, with an online or an offline billing profile. This page covers what the deployment involves and what you need to decide. The - commands, values, and troubleshooting are in the **deployment runbook**, which ships - in the deployment guide download for the chart version you install. See - [Releases](/developer-guide/self-hosting/enterprise-releases). The runbook is - versioned with the chart; this page is not, so follow the runbook when they differ. + commands, values, and troubleshooting are in the **deployment runbook**, which you + download from **Developer → Self Host** with the chart version you install. The + runbook is versioned with the chart; this page is not, so follow the runbook when + they differ. -## Choose a billing profile - -| | Offline | Online | -| ---------------- | ------------------------------------------------ | --------------------------------------------------- | -| Usage accounting | Local signed ledger on shared storage | Validated and billed against the Fish Audio service | -| Runtime egress | None | The billing endpoint on 443 — plus the model-source endpoint, if you use one | -| Model assets | Served in-cluster by the bundled model warehouse | In-cluster by default; can instead pull from S3-compatible storage | - -The bearer-token rules follow the profile: online requests must carry a -[Fish Audio API key](/developer-guide/getting-started/api-key) with credit, while -offline accepts any stable non-empty token and records it verbatim as the billing -identity — the same semantics as the -[All-in-One's usage accounting](/developer-guide/self-hosting/all-in-one#usage-accounting-and-tenancy). - -Both profiles serve model weights from inside the cluster by default. The online -profile can point the workers at S3-compatible storage instead — Fish Audio's -hosted bucket or your own; the deployment runbook's "Model source" section covers -the how. The offline profile stays in-cluster, since it runs with no egress. -"Offline" means runtime-offline: installation still pulls -images and the chart from a registry. For a cluster with no network access at all, see -[Air-gapped deployments](/developer-guide/self-hosting/air-gapped). - -## What gets installed - -The chart deploys the edge API, the model API layer, an inference router and worker, -a vocoder, a text normalizer, the model warehouse that serves the weights when -you use the default in-cluster model source, and Redis. -Everything lands in one namespace, in the single-worker shape the delivery was sized -against. - -Ingress and TLS are off by default. Put your own ingress controller or load balancer -in front of the edge API service, or enable the chart's if you want Kubernetes to -manage that layer. - -## What you decide before installing - -| Decision | Notes | -| --------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| Namespace | `fish-audio` is the supported default. A different one only needs Helm's `--namespace` argument. | -| Shared storage path | Mounted at the same path on **every** node. See [Requirements](/developer-guide/self-hosting/requirements#helm-kubernetes). | -| GPU scheduling | Tolerations and node selectors, if your GPU nodes are tainted or you run more than one GPU model. | -| Metrics | The chart exposes Prometheus annotations and creates no ServiceMonitors, so an annotation-scraping Prometheus works as-is and kube-prometheus-stack needs a scrape config. | -| Reference voices | Requests carrying a `reference_id` resolve from a local archive, never from the network. One zip per voice on shared storage, staged by you or by the optional voice sync before that traffic starts. | -| Replica counts | Talk to your account team first. The shipped shape is what the delivery was sized and validated against. | - -Three Kubernetes Secrets have to exist before the install: registry credentials, the -license bundle, and one shared between the in-cluster model store and the workers that -read from it (with an external model source, that secret instead carries the real -access keys for the storage — the runbook covers it). The license bundle is the certificate and key pair downloaded from -**Developer → Self Host**; the services that enforce it stay down until the Secret -exists, and refuse requests once the certificate lapses — renewal is described in -[Operations](/developer-guide/self-hosting/operations#license-renewal). The model-store -secret authenticates nothing outside the cluster, so the value is yours to choose. For -production, prefer External Secrets, Sealed Secrets, or your cloud secret manager over -plain Secrets. +## Billing profile + +The [billing modes](/developer-guide/self-hosting/introduction#billing) are two profiles +of the chart. The online profile can also pull the model weights from S3-compatible +storage, Fish Audio's or your own, instead of the in-cluster model warehouse. Installing +either profile still pulls images and the chart from a registry; for a cluster with no +network access at all, see [Air-gapped deployments](/developer-guide/self-hosting/air-gapped). + +## Before installing + +The chart installs everything into one namespace. Arrange the rest yourself: + +- **Shared storage**, mounted at the same path on every node. See + [Requirements](/developer-guide/self-hosting/requirements#helm-kubernetes). +- **GPU scheduling**: tolerations and node selectors, if your GPU nodes are tainted or + you run more than one GPU type. +- **Ingress and TLS**, off by default: put your own ingress or load balancer in front of + the API service, or turn on the chart's. +- **Secrets** for registry credentials, the license bundle, and model storage, + preferably from your secret manager. The services that check the license stay down + until its Secret exists. +- **Metrics**: the chart exposes Prometheus scrape annotations and creates no + ServiceMonitors. +- **Reference voices**, staged on shared storage by you or the optional voice sync + before traffic that uses them starts. +- **Replica counts**: talk to your account team before changing them. ## What to expect -Image references are pinned in the chart, so you select a version with a Helm argument -rather than by editing tags. - -The first install is slow: the inference worker compiles its GPU graphs and the -vocoder builds its inference engine before either becomes ready, which takes far longer -than any later rollout. Both artifacts are cached on shared storage, so subsequent -starts are quick, as long as that storage persists. +The first install is slow: the inference worker and the vocoder fetch their weights and +prepare for your GPU type before either becomes ready. What they download or compile stays +on shared storage, so later starts are quicker, as long as that storage persists. -Upgrades, rollback, and uninstall are all standard Helm operations. The runbook covers -the order to do them in and what to check at each step. +Upgrades, rollback, and uninstall are standard Helm operations; the runbook covers the +order and the checks. ## Next steps diff --git a/developer-guide/self-hosting/models.mdx b/developer-guide/self-hosting/models.mdx new file mode 100644 index 00000000..69a87eac --- /dev/null +++ b/developer-guide/self-hosting/models.mdx @@ -0,0 +1,32 @@ +--- +title: "Models" +description: "The models you can self-host, and where their requirements differ" +icon: "cubes" +--- + +Each model is delivered on its own: its own Helm chart, All-in-One images, versions, +and deployment guide. Your agreement grants each model separately, and **Developer → +Self Host** lists the ones your team can install. The rest of this section applies to +every model; this page covers where they differ. + +## At a glance + +| Model | Request header | GPU types | GPUs per deployment | +| -------- | ----------------- | ------------------------ | ----------------------------------------------------------------- | +| S2.1 Pro | `model: s2.1-pro` | H100, H200, or B300 | Two; one in the All-in-One's Single-GPU mode, at lower throughput | +| Drama 3 | `model: drama-3` | H200 or B300 recommended | At least two or three, depending on the GPU | + +## S2.1 Pro + +- **The first start compiles for your GPU.** The result is cached on shared storage on + Helm, and on the volume on the All-in-One, so later starts are quicker. Moving to a + different GPU type compiles once more. +- **Timestamps offline.** A Helm install without internet access copies the timestamp + model onto shared storage before turning timestamps on; the deployment runbook has the + step. + +## Drama 3 + +[Drama 3](/features/text-to-speech/drama-3) is a much larger model than S2.1 Pro and +needs more resources, GPUs above all; your deployment guide has the sizing. For a GPU +other than H200 or B300, check with Fish Audio before you plan around it. diff --git a/developer-guide/self-hosting/operations.mdx b/developer-guide/self-hosting/operations.mdx index 47ff4535..178bf0bb 100644 --- a/developer-guide/self-hosting/operations.mdx +++ b/developer-guide/self-hosting/operations.mdx @@ -1,11 +1,12 @@ --- title: "Operations" -description: "Monitoring, scaling, backups, and troubleshooting for a self-hosted deployment" +description: "Monitoring, scaling, backups, and upgrades for a self-hosted Helm deployment" icon: "chart-line" --- -Day-2 guidance for the Helm platform. For the single-container platform, see -[All-in-One container](/developer-guide/self-hosting/all-in-one). +Running the Helm platform once it is live. For the single-container platform, see +[All-in-One container](/developer-guide/self-hosting/all-in-one). Commands and +troubleshooting are in your deployment guide. ## Ownership @@ -21,37 +22,23 @@ Agree an escalation path before production traffic starts. ## Monitoring signals -Prometheus is the metrics baseline. Watch: +Prometheus is the metrics baseline. Watch request success, latency, and +time-to-first-audio; pod readiness and restarts, Redis included; CPU and memory against +requests and limits, and GPU use; shared storage; image pull failures; and the license's +time to expiry. The services export it as a metric and log warnings well before the +license lapses. -- Pod readiness and restart counts for every service in the release, including Redis. -- Request success rate, error rate, latency, and time-to-first-audio. -- Queue depth, where the application exposes it. -- GPU utilization and GPU memory. -- CPU and memory usage against the configured requests and limits. -- Shared storage usage and latency. -- Redis availability and latency. -- License validity and time to expiry, exported as metrics; the services also log - warnings well before expiry. -- Image pull failures and Kubernetes events in the release namespace. - -Centralize application logs and retain Kubernetes events long enough to -investigate a bad rollout. Redact secrets before sharing any logs outside your -environment. +Centralize logs, keep Kubernetes events long enough to investigate a bad rollout, and +redact secrets before sharing logs outside your environment. ## Scaling -Capacity is changed through the release configuration: API replicas, GPU worker -replicas, CPU and memory requests, GPU resource requests, and per-worker concurrency. -All of it takes effect on an upgrade. Scaling on load instead of by hand is included -and off by default; while it is on, the replica count is the autoscaler's rather than -the file's. Your deployment guide covers turning it on. - -Add a GPU replica only when a GPU is actually free: a pod that requests one on a full -cluster stays `Pending` indefinitely. - -Validate every scale change with the smoke test and a benchmark run at your -expected concurrency, and remember that the first start of a new GPU worker pays -the compile cost before it becomes ready. +Capacity is set in the release configuration (replicas, resource requests, and +per-worker concurrency) and applied with `helm upgrade`. Autoscaling on load is included and +off by default; your deployment guide covers turning it on. Add a GPU replica only when +a GPU is actually free: a pod that requests one on a full cluster stays `Pending`. +Validate every change at your expected concurrency, and allow minutes for a new GPU +worker to become ready. ## Backup and retention @@ -68,51 +55,18 @@ Back up: Compile and model caches on shared storage do not need backing up. Losing them costs a slow first start, nothing more. -## License renewal - -Update the license Secret in place with the renewed bundle; no restart is needed. -Kubernetes propagates the change to the running pods, and each service re-verifies on -its own short cycle, so expect up to a couple of minutes of mixed responses before -it settles. One exception: a bundle -that carries a new key rather than a re-signed certificate. The offline profile's edge API -reads the ledger-signing key only at startup, so restart that deployment after -replacing a key. - -Pods with an expired certificate keep running but refuse requests, then recover once -a renewal propagates. - ## Upgrades -Commands are in the deployment runbook. Before running them, capture the release -history so you know the revision to roll back to, and keep that revision available -until the new one is accepted. - -## Troubleshooting - -The failures a Kubernetes deployment actually produces, and where each one usually -comes from. The commands to diagnose each, and the fixes, are in the troubleshooting -guide that ships with your deployment guide. - -| Symptom | Likely causes | -| ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `ImagePullBackOff` or `ErrImagePull` | Missing pull secret, a deploy token that was rotated or deleted, a tag that is not in your mirror, or blocked egress to the registry. | -| Pods stay `Pending` | Not enough CPU, memory, or GPU capacity; GPU node taints without matching tolerations; a node selector that matches nothing. | -| Pods stuck `ContainerCreating` | A Secret the pod mounts does not exist yet — most often the license bundle. The pods recover on their own once it is created. | -| GPU not available, or `nvidia-smi` fails in a pod | Driver missing or unhealthy, device plugin or GPU Operator not ready, container runtime not configured for NVIDIA, or an unsupported GPU for the driver stack. | -| hostPath volume errors, or missing model and cache paths | Shared storage is not mounted on that node, a newly added node never got the mount, the mount target is unreachable, or permissions prevent writes. | -| Requests fail after reaching the edge API, with Redis errors in its logs | The Redis master or replica is not ready, the service is missing, or a network policy blocks it. | -| `Reference not found` | The archive was never staged, the filename does not exactly match the reference id, the zip is malformed, or the reference root was changed without mounting the new path into every edge API pod. | -| Metrics missing from Prometheus | Scrape annotations disabled in values, Prometheus not configured for annotation-based discovery, a port mismatch, or a network policy blocking scrapes. | -| `CrashLoopBackOff` | A missing environment variable, a license file missing or unreadable inside the Secret, model assets unreachable, or a path the container cannot write. | -| Requests refused with a license error | The certificate expired, or a renewal has not finished propagating — see [License renewal](#license-renewal). | -| Time-to-first-audio high, GPU utilization low | Concurrency above what the current replicas can serve, cold caches after a restart, storage latency, or CPU saturation on the API nodes. | +Take the new version from **Developer → Self Host** and follow its deployment runbook, +which covers rollback. Talk to your account team before skipping versions or upgrading +production. ## Escalating to Fish Audio Include: - Kubernetes version, cloud provider, region, node types, and GPU type. -- Chart version and the values file with secrets removed. +- The model, its chart version, and the values files with secrets removed. - Pod status, relevant events, and logs with secrets redacted. - The exact command that failed, with its output. - Timestamp and time zone. diff --git a/developer-guide/self-hosting/registry-access.mdx b/developer-guide/self-hosting/registry-access.mdx index b055404a..990d0e23 100644 --- a/developer-guide/self-hosting/registry-access.mdx +++ b/developer-guide/self-hosting/registry-access.mdx @@ -10,100 +10,49 @@ dashboard. ## The Self Host page -Sign in to fish.audio and open **Developer → Self Host**. The page opens with -**Before you start** — the platforms and modes your team is granted, with the -prerequisites it checks. If it is empty, or reports that self-host deployment -is not enabled, contact your account manager. Then five steps: - -1. **Deploy token** — create the credential the install commands authenticate with. -2. **Sign in** — one registry login with that token. -3. **License** — download the certificate and key bundle the deployment needs - before it will serve, at first install and at every renewal. Some contracts - bind it to specific GPUs — have the card UUIDs ready when you request it. -4. **Install** — pick your mode and version; the download commands are built for - your team, with the registry host, artifact references, and the version you pick - already filled in. The same step downloads the **deployment guide** for that - version — the runbook or guide the other pages here refer to. See - [Releases](/developer-guide/self-hosting/enterprise-releases) for how versions work. -5. **Test voices** — get your team's voices and Fish Audio's official voices onto the - deployment, by voice sync or by hand, then make a request with one to confirm it - works. - - - The registry host, the artifact references, and the versions available to you are - specific to your team and are shown only in the dashboard. Copy them from the Self - Host page. - +Sign in to fish.audio and open **Developer → Self Host**. It takes your team from a +deploy token and the license to the model you pick, the environment it needs, and +install commands for the version you pick, with the registry host and artifact +references filled in. The install step also downloads that version's **deployment +guide**: the Helm deployment runbook or the All-in-One guide that the other pages here +refer to. Each +model has its own versions, and a deployment stays on its version until you install +another. If the page says self-hosting is not enabled, or nothing has been granted to +your team yet, contact your account manager. ## Prerequisites - Self-hosting enabled for your team under an enterprise agreement. - A fish.audio account that is a member of that team. -- Docker, and Helm 3.8 or newer for the OCI chart commands. - -## Create a deploy token - - - - On **Developer → Self Host**, select **Create Deploy Token**. - - - Use a name that identifies the consumer, such as `prod-cluster` or `ci-mirror`. The - name appears in the token list alongside the creation date and last-used time. - - - The token value is shown once, at creation. Store it in your secret manager before - closing the dialog. If you lose it, rotate the token to issue a new one. - - +- Docker for the All-in-One, and for mirroring images into an air-gapped cluster; + Helm 3.8 or newer for the Kubernetes chart. -A team can hold up to five deploy tokens at a time. Tokens carry the grants of the team -that owns them, not of the person who created them, and follow those grants as they -change, so there is no token to recreate when your entitlement is updated. +## Deploy tokens -Authenticate with your account email as the username and the deploy token as the -password. Docker, Helm, and the cluster's image pull secret all use the same pair; the -exact commands are on the Self Host page and in the deployment runbook. +Create one token per consumer, such as a production cluster, a staging cluster, or a CI +mirror, and name it after that consumer. The value is shown once: store it in your +secret manager, not in values files or version control. A team holds up to five tokens, +and they carry the team's grants rather than their creator's. -## Managing tokens +Docker, Helm, and the cluster's image pull secret all sign in with your account email +and the token. -| Action | Effect | -| --------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -| Rotate | Issues a new token value and invalidates the old one immediately. Update every consumer before rotating, or new pods will fail to pull. | -| Delete | Revokes the token immediately. Any deployment still using it stops pulling images. Running pods keep running until they are rescheduled. | -| Last used | Shows when the token last authenticated, which identifies tokens that are safe to retire. | - -Recommended practice: - -- Issue one token per consumer (production cluster, staging cluster, CI mirror) so a - single revocation never takes down more than one of them. -- Store tokens in your secret manager, not in values files or version control. -- Rotate on your normal credential schedule and whenever someone with access to a token - leaves the team. +Rotating or deleting a token revokes the old value at once. Running pods keep running, +but new pulls fail until every consumer has a valid token. To switch with no gap, +create a second token, move every consumer to it, then delete the old one. The token +list shows when each token was last used, so you can retire the ones nothing uses. ## The license -The license is the product permit: every platform and billing mode needs one to -run, and it is separate from how usage is billed. Fish Audio issues the -license against your contract, and the dashboard's License step is -where you download the bundle — a certificate (`license.crt`) and key -(`license.key`) that the deployment mounts, as its guide's License section -shows. Some contracts bind the license to specific GPU UUIDs; the dashboard -shows the binding. - -The two failure modes are worth knowing before they happen. **Missing**, the -services never come up. **Expired**, they start but refuse every request with a -license error (403) until the renewed bundle is in place. +The license is the product permit: every platform and billing mode needs one. Download +the bundle, `license.crt` and `license.key`, from the License step on the Self Host +page; some contracts bind it to specific GPUs, so have their UUIDs ready. Without it, the services never come up. +Once it expires, they refuse requests (403) until a renewed one is in place. The +dashboard shows the current expiry. -**Validity and renewal.** The certificate carries a validity window bounded by -your contract term; the dashboard shows the current expiry. Renewals are driven -by Fish Audio: before the expiry, settle the period's invoice — and, on offline -billing, send the usage ledger for reconciliation — and a renewed license is -issued. A renewal replaces only `license.crt`, in place, and the running -services reload it without a restart; only a bundle that carries a new key -needs one. The Helm-side mechanics are in -[License renewal](/developer-guide/self-hosting/operations#license-renewal); -the All-in-One's are in its deployment guide. +Before the expiry, settle the period's invoice and, on offline billing, send the usage +ledger; Fish Audio then issues the renewed bundle. Your deployment guide shows how to +apply it. ## Next step diff --git a/developer-guide/self-hosting/requirements.mdx b/developer-guide/self-hosting/requirements.mdx index 72f369d3..8948cc84 100644 --- a/developer-guide/self-hosting/requirements.mdx +++ b/developer-guide/self-hosting/requirements.mdx @@ -13,9 +13,8 @@ team and measure on your own hardware before you commit to a capacity plan. | Requirement | Detail | | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Hosts | Linux x86-64 on every machine, CPU-only Kubernetes nodes included. | -| GPU type | NVIDIA H100, H200, or B300. Contact Fish Audio before standardizing on any other GPU. | -| GPU count | **One**: Single-GPU mode, All-in-One only, with lower throughput.
**Two**: recommended. One for the inference worker, one for the vocoder.
**Three**: adds timestamp alignment, where available.
**More**: scales throughput on Kubernetes; size it with your account team. | -| GPU memory | The worker pre-allocates most of its card for the KV cache, so a near-full card is expected, not a fault. | +| GPU type | NVIDIA. The types, and the GPU count, depend on the model: see [Models](/developer-guide/self-hosting/models). Contact Fish Audio before standardizing on a GPU not listed there. | +| GPU count | **Two** in most deployments: one for the inference worker, one for the vocoder.
**One more**: timestamp alignment, where available.
**More**: scales throughput on Kubernetes; size it with your account team. | | MIG | Disable MIG unless the configuration has been validated with Fish Audio. | | NVIDIA driver | Must support CUDA 13.x and your card's compute capability. | @@ -28,12 +27,14 @@ team and measure on your own hardware before you commit to a capacity plan. | Voice sync | Optional. When turned on, the deployment downloads voices from Fish Audio over HTTPS. | | Inbound | Your own ingress, load balancer, or private endpoint, with DNS and TLS in place before production traffic. | -## Helm (Kubernetes) +## Platform-specific requirements + +### Helm (Kubernetes) | Requirement | Detail | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Control plane | 3 CPU-only nodes, roughly 4 vCPU and 16 GB RAM each, for the control plane or, on managed Kubernetes, the system node group. | -| CPU workloads | Roughly 28 vCPU and 125 GiB of memory requests for the non-GPU services. Provide a CPU node group, or leave GPU nodes schedulable for them. | +| CPU workloads | Roughly 28 vCPU and 125 GiB of memory requests for the non-GPU services, beyond the control plane. Provide a CPU node group, or leave GPU nodes schedulable for them. | | Shared storage | At least 1 TB of EFS, NFS, or equivalent, mounted at the same path on every node (`/mnt/shared` by default). The chart mounts it with hostPath and creates no PersistentVolumeClaims. | | Node-local disk | Redis keeps its data on node-local ephemeral storage. Leave room for it on the nodes that host it, plus headroom for rewrites. | | Kubernetes | A currently supported minor version and a maintained Helm release. | @@ -47,13 +48,13 @@ team and measure on your own hardware before you commit to a capacity plan. before installing. -## All-in-One (Docker) +### All-in-One (Docker) | Requirement | Detail | | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | Docker | Docker Engine 24 or newer, with the NVIDIA Container Toolkit installed and its `nvidia` runtime registered. | | CPU and RAM | 32 vCPU and 128 GiB minimum; 48 to 64 vCPU and 192 GiB recommended. | -| Disk | 100 GB free to pull the image, or 120 GB to load it from a file. Plus a persistent volume for caches, voices, and the usage ledger; local disk is enough. | +| Disk | For S2.1 Pro, 100 GB free to pull the image, or 120 GB to load it from a file; Drama 3 needs more. Plus a persistent volume for caches, voices, and the usage ledger; local disk is enough. | ## Next step diff --git a/docs.json b/docs.json index 30ae881f..639c945b 100644 --- a/docs.json +++ b/docs.json @@ -258,13 +258,13 @@ "group": "Self-Hosting", "pages": [ "developer-guide/self-hosting/introduction", + "developer-guide/self-hosting/models", "developer-guide/self-hosting/requirements", "developer-guide/self-hosting/registry-access", "developer-guide/self-hosting/kubernetes", "developer-guide/self-hosting/all-in-one", "developer-guide/self-hosting/air-gapped", - "developer-guide/self-hosting/operations", - "developer-guide/self-hosting/enterprise-releases" + "developer-guide/self-hosting/operations" ] }, { @@ -724,6 +724,10 @@ "source": "/self-host", "destination": "/developer-guide/self-hosting/introduction" }, + { + "source": "/developer-guide/self-hosting/enterprise-releases", + "destination": "/developer-guide/self-hosting/registry-access" + }, { "source": "/developer-guide/self-hosting/local-setup", "destination": "/developer-guide/self-hosting/introduction"