IM

ISSA MDOE

Full Stack Software Engineer

Loading0%
2025-12-28 12 min read

Building Resilient Systems: The Circuit Breaker Pattern

#System Design#Microservices#Reliability
👨🏽‍💻
Issa Ally Mdoe
Full Stack Engineer

In a distributed microservices environment, failure is not an exception; it is the rule. Network glitches happen, databases overload, and dependencies go down.

If Service A calls Service B, and Service B hangs, Service A will eventually run out of threads waiting for B, crashing itself. This causes a Cascading Failure, taking down your entire platform.

Enter the Circuit Breaker

The Circuit Breaker pattern prevents an application from performing an operation that is likely to fail. It wraps a protected function call and monitors for failures.

The 3 States

  1. CLOSED (Normal): Requests pass through. We count errors. If the error rate exceeds a threshold (e.g., 50%), the breaker TRIPS.
  2. OPEN (Failing): The breaker prevents requests from executing and immediately returns a fallback error. This gives the failing service time to recover.
  3. HALF-OPEN (Testing): After a timeout, we let a *few* requests through. If they succeed, we reset to CLOSED. If they fail, we go back to OPEN.

Implementation in Node.js

We can use libraries like opossum.

javascript
const CircuitBreaker = require('opossum');

async function riskyServiceCall() { // Call unreliable API return axios.get('http://unreliable-api.com'); }

const options = { timeout: 3000, // If request takes > 3s, fail errorThresholdPercentage: 50, // Trip if 50% fail resetTimeout: 10000 // Wait 10s before trying again };

const breaker = new CircuitBreaker(riskyServiceCall, options);

breaker.fallback(() => ({ body: "Service unavailable, try later." }));

// Use it const response = await breaker.fire(); `

Use Cases

  • External APIs: Payment gateways, Maps API, etc.
  • Database Queries: Protecting against DB overload.
  • Microservice Inter-communication.

Best Practices

  1. Fail Fast: Don't wait 30 seconds for a timeout. If the breaker is open, fail instantly.
  2. Fallbacks: Always provide a degraded experience. If the "Recommendations" service is down, show "Popular Items" from cache instead of a 500 error.
  3. Monitoring: Log every state change. Knowing when breakers trip gives you early warning of system instability.

Adding circuit breakers makes your system strictly more robust and is a hallmark of senior engineering architecture.


Enjoyed this article?
Share it with your network