diff --git a/aigw/product/ai-gateway/circuit-breaker.mdx b/aigw/product/ai-gateway/circuit-breaker.mdx index b723695b..380846c9 100644 --- a/aigw/product/ai-gateway/circuit-breaker.mdx +++ b/aigw/product/ai-gateway/circuit-breaker.mdx @@ -76,3 +76,15 @@ flowchart TD - Unhealthy targets removed from routing - If all targets are OPEN, circuit breaker is bypassed + +## Where Circuit Breakers Fit + +Circuit breakers protect your setup from unhealthy targets. They watch for failures and open when a target stops responding reliably, so the gateway can route around it until it recovers. Circuit breakers are failure-based. They open when failures cross your configured thresholds, and they don't limit how many requests run in parallel. + +If your goal is to shape traffic or manage load, for example handling bursts of parallel requests, these work well alongside circuit breakers in a config: + +- [Rate limits](/aigw/product/ai-gateway/virtual-keys/rate-limits) to cap requests or tokens on a provider or integration +- [Load balancing](/aigw/product/ai-gateway/load-balancing) to spread traffic across targets by weight +- [Fallbacks](/aigw/product/ai-gateway/fallbacks) to route to a backup target when a request fails + +Each handles its own part. For example, if a provider returns `429` under heavy load and `429` is in `failure_status_codes`, the circuit breaker counts those responses as failures and can open the circuit, while your fallback or load balancing strategy decides where traffic goes next. diff --git a/product/ai-gateway/circuit-breaker.mdx b/product/ai-gateway/circuit-breaker.mdx index c24dfd0b..9c39ed34 100644 --- a/product/ai-gateway/circuit-breaker.mdx +++ b/product/ai-gateway/circuit-breaker.mdx @@ -81,6 +81,17 @@ flowchart TD - Unhealthy targets removed from routing - If all targets are OPEN, circuit breaker is bypassed +## Where Circuit Breakers Fit + +Circuit breakers protect your setup from unhealthy targets. They watch for failures and open when a target stops responding reliably, so the gateway can route around it until it recovers. Circuit breakers are failure-based. They open when failures cross your configured thresholds, and they don't limit how many requests run in parallel. + +If your goal is to shape traffic or manage load, for example handling bursts of parallel requests, these work well alongside circuit breakers in a config: + +- [Rate limits](/product/ai-gateway/virtual-keys/rate-limits) to cap requests or tokens on a provider or integration +- [Load balancing](/product/ai-gateway/load-balancing) to spread traffic across targets by weight +- [Fallbacks](/product/ai-gateway/fallbacks) to route to a backup target when a request fails + +Each handles its own part. For example, if a provider returns `429` under heavy load and `429` is in `failure_status_codes`, the circuit breaker counts those responses as failures and can open the circuit, while your fallback or load balancing strategy decides where traffic goes next. import PrismaAirsCta from "/snippets/prisma-airs-cta.mdx";