How Will You Implement Rate Limiting in Microservices?
Rate limiting is a technique used to control how many requests a client can send to a system within a specific time period.
Main Goal
Protect Services From Excessive Traffic And Prevent System Overload
Why Rate Limiting Is Important?
- Prevent server overload
- Protect against DDoS attacks
- Avoid abuse by clients
- Ensure fair resource usage
- Prevent cascading failures
- Reduce infrastructure cost
- Improve system stability
Real Production Example
Suppose:
Payment Service
normally handles:
1000 requests/second
Suddenly One Client Sends
100,000 requests/second
Without Rate Limiting
- CPU becomes 100%
- Memory increases
- Threads exhausted
- Database overloaded
- Entire application slows down
With Rate Limiting
Only Allowed Requests Processed Extra Requests Rejected
Rate Limiting Techniques
- Token Bucket Algorithm
- Leaky Bucket Algorithm
- Fixed Window Counter
- Sliding Window Counter
- Sliding Log Algorithm
- API Gateway Rate Limiting
- Redis-Based Distributed Rate Limiting
- User-Based Rate Limiting
- IP-Based Rate Limiting
- Tenant-Based Rate Limiting
- Kubernetes Ingress Rate Limiting
- Service Mesh Rate Limiting
1. Token Bucket Algorithm (Most Popular)
Widely used in production systems.
Concept
Bucket Contains Tokens Each Request Consumes One Token
Example
Bucket Capacity = 10 Tokens
Flow
Request Arrives
↓
Token Available?
↓
YES → Process Request
NO → Reject Request
Tokens Refill Gradually
1 Token Every Second
Benefits
- Supports burst traffic
- Flexible
- Widely adopted
Production Example
User Allowed: 100 Requests/Minute
If User Sends
101st Request
Response
HTTP 429 Too Many Requests
2. Leaky Bucket Algorithm
Requests processed at fixed rate.
Concept
Bucket Leaks Requests Slowly
Flow
Incoming Requests Stored
↓
Processed At Constant Speed
Benefits
- Smooth traffic flow
- Prevents sudden spikes
Problem
- Burst traffic may get dropped
3. Fixed Window Counter
Simple implementation.
Example
Limit: 100 Requests Per Minute
Flow
Counter Reset Every Minute
Problem
Boundary issue.
Example
100 Requests At 12:00:59 100 Requests At 12:01:00
Actual Requests
200 Requests In 2 Seconds
4. Sliding Window Counter
Improves fixed window problem.
Concept
Uses Rolling Time Window
Benefits
- More accurate
- Smoother rate limiting
Production Usage
Frequently used in API gateways.
5. API Gateway Rate Limiting
Most common production approach.
Architecture
Client ↓ API Gateway ↓ Microservices
Why API Gateway?
- Centralized control
- Easy management
- Protect all services
- Reduce duplicate logic
Popular API Gateways
- :contentReference[oaicite:0]{index=0}
- :contentReference[oaicite:1]{index=1}
- :contentReference[oaicite:2]{index=2}
- :contentReference[oaicite:3]{index=3}
NGINX Example
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
Meaning
Allow 10 Requests Per Second
6. Redis-Based Distributed Rate Limiting
Important in distributed microservices.
Problem
Single instance counters fail in distributed systems.
Example
3 API Gateway Instances
Without Shared Storage
Each Instance Maintains Separate Counter
Result
Incorrect Rate Limiting
Solution
Centralized Redis Counter
Architecture
Clients ↓ API Gateway Instances ↓ Redis Shared Counter
Benefits
- Distributed consistency
- Fast performance
- Scalable
Popular Tool
- :contentReference[oaicite:4]{index=4}
Redis Flow
Request Arrives
↓
Increment Redis Counter
↓
Check Limit
↓
Allow Or Reject
7. User-Based Rate Limiting
Different users get different limits.
Example
| User Type | Limit |
|---|---|
| Free User | 100 requests/day |
| Premium User | 10,000 requests/day |
Benefits
- Business flexibility
- Fair resource usage
8. IP-Based Rate Limiting
Common for public APIs.
Example
Limit: 50 Requests Per Minute Per IP
Benefits
- Simple protection
- Blocks abusive traffic
Problem
- NAT/shared IP limitations
9. Tenant-Based Rate Limiting
Common in SaaS platforms.
Example
Company A: 1 Million Requests/day Company B: 10,000 Requests/day
Benefits
- Subscription-based control
- Enterprise scalability
10. Kubernetes Ingress Rate Limiting
Rate limiting at ingress level.
NGINX Ingress Example
nginx.ingress.kubernetes.io/limit-rps: "10"
Meaning
10 Requests Per Second
Benefits
- Cluster-level protection
- Simple configuration
11. Service Mesh Rate Limiting
Modern microservices often use service mesh.
Popular Tools
- :contentReference[oaicite:5]{index=5}
- :contentReference[oaicite:6]{index=6}
Benefits
- Fine-grained control
- Traffic management
- Observability
12. Spring Boot Rate Limiting Example
Production Java implementation.
Using Bucket4j
- :contentReference[oaicite:7]{index=7}
Maven Dependency
<dependency> <groupId>com.bucket4j</groupId> <artifactId>bucket4j-core</artifactId> </dependency>
Java Example
Bucket bucket = Bucket.builder()
.addLimit(Bandwidth.simple(100,
Duration.ofMinutes(1)))
.build();
if(bucket.tryConsume(1)) {
return "Allowed";
}
else {
return "Too Many Requests";
}
Meaning
Allow 100 Requests Per Minute
13. HTTP Response for Rate Limiting
Standard response:
HTTP 429 Too Many Requests
Useful Headers
X-RateLimit-Limit: 100 X-RateLimit-Remaining: 20 Retry-After: 60
Benefits
- Client awareness
- Better API usage
14. Rate Limiting + Circuit Breaker
Both work together for protection.
Rate Limiting
Controls Incoming Traffic
Circuit Breaker
Prevents Dependency Failures
Popular Tool
- :contentReference[oaicite:8]{index=8}
15. Monitoring Rate Limiting
Monitor rejected traffic.
Monitor
- Rejected requests
- Traffic spikes
- Latency
- CPU usage
- API abuse patterns
Monitoring Tools
- :contentReference[oaicite:9]{index=9}
- :contentReference[oaicite:10]{index=10}
16. Real Production Example
Scenario
Public payment API exposed externally.
Problem
One Client Sending Massive Requests
Impact
- CPU spike
- Database overload
- Payment latency increased
Solution Applied
- API Gateway rate limiting
- Redis distributed counters
- User-based quotas
- IP throttling
- Kubernetes auto scaling
Final Result
- Stable system
- Controlled traffic
- Protected services
- Improved availability
Production Best Practices
| Practice | Purpose |
|---|---|
| API Gateway Limiting | Centralized protection |
| Redis Counters | Distributed consistency |
| Token Bucket | Flexible rate limiting |
| User Quotas | Business control |
| Monitoring | Detect abuse |
| Circuit Breakers | Prevent cascading failures |
| Auto Scaling | Handle traffic spikes |
| HTTP 429 | Standard client handling |
Final Interview Answer
To implement rate limiting in microservices, I would typically use centralized rate limiting at the API Gateway layer using tools such as :contentReference[oaicite:11]{index=11}, :contentReference[oaicite:12]{index=12}, or :contentReference[oaicite:13]{index=13}. The most common production algorithm is the Token Bucket algorithm, where requests consume tokens from a bucket and excess requests are rejected with HTTP 429 responses. In distributed environments, I would use :contentReference[oaicite:14]{index=14} to maintain centralized counters across multiple gateway instances to ensure consistent rate limiting. Depending on business requirements, rate limits can be applied per user, IP address, API key, or tenant. I would also combine rate limiting with circuit breakers using tools like :contentReference[oaicite:15]{index=15} to prevent cascading failures. Additionally, I would monitor rejected requests, traffic spikes, and service metrics using :contentReference[oaicite:16]{index=16} and :contentReference[oaicite:17]{index=17}. The overall goal is to protect microservices from overload, ensure fair usage, improve system stability, and maintain high availability in production environments.