Skip to content
DeepintShield DeepintShield

Circuit Breaker

Overview

Section titled “Overview”

Circuit breaking protects your traffic from unhealthy provider API keys. When a key starts failing - repeated errors, timeouts, or rate limits - DeepIntShield temporarily removes it from the rotation so new requests are routed to healthy keys instead. The key is automatically reconsidered once it recovers.

This is part of DeepIntShield’s LLM Key Load Balancer, which selects among the multiple keys you configure for a provider and continuously tracks each key’s health.

How it behaves

Section titled “How it behaves”
  • Detection: Each key’s error rate, latency, and success rate are tracked in real time.
  • Open the circuit: A key that crosses the failure threshold is taken out of rotation. Requests that would have used it are routed to other healthy keys.
  • Recovery: The key is periodically retried, and once it succeeds again it returns to normal weighted rotation.

From the caller’s perspective this is transparent - requests keep succeeding on healthy keys with no code changes.

Enabling it

Section titled “Enabling it”

Circuit breaking is enabled with the LLM Key Load Balancer. Turn on load_balancer_enabled in your configuration (or the Load Balancing toggle in the dashboard), then configure multiple keys per provider so there is something to fail over to.

{
  "client": {
    "load_balancer_enabled": true
  }
}

See Load Balance for configuring multiple weighted keys per provider, and Adaptive Load Balancing for the full health-monitoring and recovery behavior.

Section titled “Related features”