From 0b6705003638d15d53ffe169db294ff9fb8dffd7 Mon Sep 17 00:00:00 2001 From: M2Night Date: Sat, 31 Oct 2026 10:27:50 +0800 Subject: [PATCH] Self-hosting docs: the All-in-One scales out by adding containers, and has a timestamps build The pages said the All-in-One does not scale at all, which led readers to rule it out for production. A container never scales itself, but customers can run more of them behind their own load balancer. The Kubernetes chart remains the way to scale workers and vocoders separately. Two timestamp statements were out of date. The All-in-One now has a separate build with timestamp alignment, where available. S2.1 Pro's timestamp service carries its model in its own image, so an offline Helm install no longer copies that model onto shared storage. Part of FISH-1723 Co-Authored-By: Claude Opus 5.5 (1M context) --- developer-guide/self-hosting/all-in-one.mdx | 9 ++++----- developer-guide/self-hosting/introduction.mdx | 16 ++++++++-------- developer-guide/self-hosting/models.mdx | 3 --- developer-guide/self-hosting/requirements.mdx | 2 +- 4 files changed, 13 insertions(+), 17 deletions(-) diff --git a/developer-guide/self-hosting/all-in-one.mdx b/developer-guide/self-hosting/all-in-one.mdx index 6dabe8e..3387b83 100644 --- a/developer-guide/self-hosting/all-in-one.mdx +++ b/developer-guide/self-hosting/all-in-one.mdx @@ -21,11 +21,10 @@ online or offline; the container logs which at startup. ## What it cannot do The appliance runs one inference worker and one vocoder, on the cards its model -takes — see [Models](/developer-guide/self-hosting/models). It does not -autoscale or add workers on more GPUs or nodes, and it does not include timestamp -alignment, so it returns no word or segment timings. For elastic or higher-throughput -deployments, use the [Kubernetes chart](/developer-guide/self-hosting/kubernetes), which -scales replicas across all GPUs and nodes. +takes — see [Models](/developer-guide/self-hosting/models). A container does not scale +itself: for more capacity, run more containers behind your own load balancer. To scale +workers and vocoders separately, use the +[Kubernetes chart](/developer-guide/self-hosting/kubernetes). ## What running it involves diff --git a/developer-guide/self-hosting/introduction.mdx b/developer-guide/self-hosting/introduction.mdx index d7b4bf2..92f218c 100644 --- a/developer-guide/self-hosting/introduction.mdx +++ b/developer-guide/self-hosting/introduction.mdx @@ -26,14 +26,14 @@ audio formats, and voice behavior match what you already build against. Each model ships on two platforms. Your agreement decides which ones your team gets, and in which modes; **Developer → Self Host** shows them. -| | Helm (Kubernetes) | All-in-One (Docker) | -| ---------- | ----------------------------------------------------------------- | ---------------------------------------------------------------- | -| What it is | The model's chart, installed on your cluster | One container on a single machine | -| Best for | Production clusters and elastic capacity | Evaluation, fixed-size appliances, strict air gaps | -| Scaling | Replicas across GPUs and nodes, with optional autoscaling | None: one worker and one vocoder | -| Metrics | Prometheus metrics from the speech services | API metrics only, at `/metrics` | -| Timestamps | Optional, where available | Not included | -| Guide | [Kubernetes deployment](/developer-guide/self-hosting/kubernetes) | [All-in-One container](/developer-guide/self-hosting/all-in-one) | +| | Helm (Kubernetes) | All-in-One (Docker) | +| ---------- | ----------------------------------------------------------------- | -------------------------------------------------------------------------------- | +| What it is | The model's chart, installed on your cluster | One container on a single machine | +| Best for | Production clusters and elastic capacity | Evaluation, fixed-size appliances, strict air gaps | +| Scaling | Replicas across GPUs and nodes, with optional autoscaling | Add containers behind your own load balancer; each is one worker and one vocoder | +| Metrics | Prometheus metrics from the speech services | API metrics only, at `/metrics` | +| Timestamps | Optional, where available | A separate build, where available | +| Guide | [Kubernetes deployment](/developer-guide/self-hosting/kubernetes) | [All-in-One container](/developer-guide/self-hosting/all-in-one) | The GPUs each model needs are on [Models](/developer-guide/self-hosting/models). On either platform, ingress, DNS, TLS, external load balancing, and network allowlists diff --git a/developer-guide/self-hosting/models.mdx b/developer-guide/self-hosting/models.mdx index 69a87ea..9110db1 100644 --- a/developer-guide/self-hosting/models.mdx +++ b/developer-guide/self-hosting/models.mdx @@ -21,9 +21,6 @@ every model; this page covers where they differ. - **The first start compiles for your GPU.** The result is cached on shared storage on Helm, and on the volume on the All-in-One, so later starts are quicker. Moving to a different GPU type compiles once more. -- **Timestamps offline.** A Helm install without internet access copies the timestamp - model onto shared storage before turning timestamps on; the deployment runbook has the - step. ## Drama 3 diff --git a/developer-guide/self-hosting/requirements.mdx b/developer-guide/self-hosting/requirements.mdx index 8948cc8..a4dcbf5 100644 --- a/developer-guide/self-hosting/requirements.mdx +++ b/developer-guide/self-hosting/requirements.mdx @@ -14,7 +14,7 @@ team and measure on your own hardware before you commit to a capacity plan. | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Hosts | Linux x86-64 on every machine, CPU-only Kubernetes nodes included. | | GPU type | NVIDIA. The types, and the GPU count, depend on the model: see [Models](/developer-guide/self-hosting/models). Contact Fish Audio before standardizing on a GPU not listed there. | -| GPU count | **Two** in most deployments: one for the inference worker, one for the vocoder.
**One more**: timestamp alignment, where available.
**More**: scales throughput on Kubernetes; size it with your account team. | +| GPU count | **Two** in most deployments: one for the inference worker, one for the vocoder.
**One more**: timestamp alignment, where available.
**More**: adds throughput as more replicas or containers; size it with your account team. | | MIG | Disable MIG unless the configuration has been validated with Fish Audio. | | NVIDIA driver | Must support CUDA 13.x and your card's compute capability. |