Service A Is Calling Service B Synchronously, but Service B Is Down — How Will You Handle This Scenario?
In microservices architecture, services frequently communicate using synchronous REST API calls.
Example:
Service A → REST Call → Service B
If Service B goes down:
- Requests start failing
- Threads become blocked
- Timeouts increase
- Application performance degrades
- Cascading failures may occur
This is one of the most common production issues in distributed systems.
Real-Time Production Example
Banking Application Flow
Transaction Service
↓
Payment Service
↓
Fraud Detection Service
Suppose:
- Transaction Service = Service A
- Payment Service = Service B
If Payment Service goes down:
- Fund transfers fail
- Threads remain waiting
- API timeout increases
- Entire banking application becomes slow
Production Problems That Can Happen
| Problem | Impact |
|---|---|
| Thread Blocking | Requests wait indefinitely |
| Timeouts | User receives errors |
| Retry Storms | System overload increases |
| Cascading Failure | Entire application becomes unavailable |
| Resource Exhaustion | CPU and memory spike |
Production-Level Solutions
- Timeout Configuration
- Circuit Breaker
- Fallback Mechanism
- Retry Mechanism
- Bulkhead Pattern
- Asynchronous Communication
- Caching
- Graceful Degradation
Step 1: Configure Timeout
Never allow Service A to wait forever for Service B.
Problem Without Timeout
Service A
↓ waits infinitely
Service B Down
Threads become blocked.
Spring WebClient Timeout Example
@Bean
public WebClient webClient() {
return WebClient.builder()
.baseUrl("http://service-b")
.build();
}
Timeout Configuration
public Mono<String> callServiceB() {
return webClient.get()
.uri("/payment")
.retrieve()
.bodyToMono(String.class)
.timeout(Duration.ofSeconds(3));
}
Now Service A stops waiting after 3 seconds.
Step 2: Implement Circuit Breaker
Circuit Breaker prevents cascading failures.
What Happens Without Circuit Breaker?
Service B Down
↓
Service A keeps calling
↓
Threads exhausted
↓
CPU increases
↓
Entire system becomes slow
How Circuit Breaker Works
Failures Increase
↓
Circuit Opens
↓
Calls Temporarily Blocked
↓
Fallback Response Returned
Resilience4j Dependency
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-spring-boot3</artifactId>
</dependency>
Circuit Breaker Example
@CircuitBreaker(
name = "paymentService",
fallbackMethod = "fallbackResponse")
public String processPayment() {
return paymentClient.call();
}
Fallback Method
public String fallbackResponse(Exception ex) {
return "Payment Service Temporarily Unavailable";
}
Benefits
- Prevents thread exhaustion
- Reduces unnecessary calls
- Protects the system
- Improves application stability
Step 3: Implement Retry Mechanism
Sometimes failures are temporary.
Examples:
- Temporary network issue
- Short DB outage
- Container restart
Retry Example
@Retry(name = "paymentRetry")
public String callService() {
return paymentClient.call();
}
Retry Configuration
resilience4j:
retry:
instances:
paymentRetry:
max-attempts: 3
wait-duration: 2s
Important Production Note
Retries must be limited.
Otherwise:
Thousands of retries
↓
System overload
↓
Retry Storm
Step 4: Use Bulkhead Pattern
Bulkhead isolates failures.
Real Example
Suppose:
- Loan Service becomes slow
- UPI payments should still work
Bulkhead prevents one service from consuming all threads/resources.
Bulkhead Example
@Bulkhead(
name = "paymentBulkhead",
type = Bulkhead.Type.THREADPOOL)
public String callPaymentService() {
return paymentService.call();
}
Step 5: Graceful Degradation
Some features can temporarily degrade instead of failing completely.
Example
If recommendation service is down:
Show products without recommendations
Instead of crashing the entire application.
Banking Example
If reward points service fails:
- Transaction should still continue
- Reward points can update later
Step 6: Use Cache as Fallback
Previously cached data can be returned temporarily.
Redis Cache Example
@Cacheable("customerCache")
public Customer getCustomer(Long id) {
return repository.findById(id).get();
}
Fallback Using Cache
If Service B Down
↓
Return Cached Response
Step 7: Move to Asynchronous Communication
Long synchronous chains increase system dependency.
Bad Design
Order Service
↓
Payment Service
↓
Notification Service
Every service waits for another service.
Better Design Using Kafka
Order Created
↓
Publish Kafka Event
↓
Payment Service Consumes
Now services become loosely coupled.
Benefits of Async Communication
- Improved scalability
- Reduced cascading failures
- Better fault tolerance
- Improved system resilience
Step 8: Monitor the Failure
Use observability tools:
- Grafana
- Prometheus
- Zipkin
- ELK Stack
- Jaeger
Production Metrics to Monitor
- Error rate
- Timeout count
- Circuit breaker open state
- Retry count
- CPU usage
- Thread pool usage
Production Incident Example
Issue
An e-commerce payment gateway became unavailable during a festival sale.
Without Resilience Patterns
- Threads exhausted
- CPU reached 100%
- Entire application crashed
Fixes Applied
- Added timeout
- Implemented circuit breaker
- Configured retries
- Introduced Kafka async flow
- Added fallback responses
- Implemented bulkhead isolation
Final Result
Before: Entire application unavailable After: Only payment feature degraded Other services continued working
Important Production-Level Techniques
| Technique | Purpose |
|---|---|
| Timeout | Avoid infinite waiting |
| Circuit Breaker | Prevent cascading failures |
| Retry | Handle temporary failures |
| Bulkhead | Isolate failures |
| Fallback | Provide backup response |
| Cache | Return cached data |
| Kafka | Asynchronous communication |
| Monitoring | Track failures and latency |
Final Interview Answer
If Service A is calling Service B synchronously and Service B is down, I would first configure proper timeout settings to avoid infinite waiting. Then I would implement Circuit Breaker using Resilience4j to prevent cascading failures and protect system resources. I would also add fallback mechanisms, retries with limits, and bulkhead isolation to improve resilience. For critical business flows, I would move from synchronous communication to asynchronous communication using Kafka or RabbitMQ. Additionally, I would monitor failures using observability tools like Grafana and Prometheus to quickly detect and recover from issues in production environments.