Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions aigw/product/ai-gateway/circuit-breaker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,15 @@ flowchart TD

- Unhealthy targets removed from routing
- If all targets are OPEN, circuit breaker is bypassed

## Where Circuit Breakers Fit

Circuit breakers protect your setup from unhealthy targets. They watch for failures and open when a target stops responding reliably, so the gateway can route around it until it recovers. Circuit breakers are failure-based. They open when failures cross your configured thresholds, and they don't limit how many requests run in parallel.

If your goal is to shape traffic or manage load, for example handling bursts of parallel requests, these work well alongside circuit breakers in a config:

- [Rate limits](/aigw/product/ai-gateway/virtual-keys/rate-limits) to cap requests or tokens on a provider or integration
- [Load balancing](/aigw/product/ai-gateway/load-balancing) to spread traffic across targets by weight
- [Fallbacks](/aigw/product/ai-gateway/fallbacks) to route to a backup target when a request fails

Each handles its own part. For example, if a provider returns `429` under heavy load and `429` is in `failure_status_codes`, the circuit breaker counts those responses as failures and can open the circuit, while your fallback or load balancing strategy decides where traffic goes next.
11 changes: 11 additions & 0 deletions product/ai-gateway/circuit-breaker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,17 @@ flowchart TD
- Unhealthy targets removed from routing
- If all targets are OPEN, circuit breaker is bypassed

## Where Circuit Breakers Fit

Circuit breakers protect your setup from unhealthy targets. They watch for failures and open when a target stops responding reliably, so the gateway can route around it until it recovers. Circuit breakers are failure-based. They open when failures cross your configured thresholds, and they don't limit how many requests run in parallel.

If your goal is to shape traffic or manage load, for example handling bursts of parallel requests, these work well alongside circuit breakers in a config:

- [Rate limits](/product/ai-gateway/virtual-keys/rate-limits) to cap requests or tokens on a provider or integration
- [Load balancing](/product/ai-gateway/load-balancing) to spread traffic across targets by weight
- [Fallbacks](/product/ai-gateway/fallbacks) to route to a backup target when a request fails

Each handles its own part. For example, if a provider returns `429` under heavy load and `429` is in `failure_status_codes`, the circuit breaker counts those responses as failures and can open the circuit, while your fallback or load balancing strategy decides where traffic goes next.

import PrismaAirsCta from "/snippets/prisma-airs-cta.mdx";

Expand Down
Loading