Multiple Microservices Are Updating the Same Data Simultaneously — How Will You Avoid Data Conflicts?
In microservices architecture, multiple services may try to update the same data at the same time.
This can create:
- Race conditions
- Dirty writes
- Lost updates
- Duplicate transactions
- Data inconsistency
Handling concurrent updates properly is very important in banking, e-commerce, stock trading, inventory management, and payment systems.
Real-Time Banking Example
Scenario
Suppose:
- Customer account balance = ₹10,000
- Two transactions happen simultaneously
Request 1
ATM Withdrawal = ₹7000
Request 2
UPI Payment = ₹5000
Problem Without Proper Concurrency Handling
Initial Balance = 10000 Thread 1 Reads Balance = 10000 Thread 2 Reads Balance = 10000 Thread 1 Updates = 3000 Thread 2 Updates = 5000
Final Incorrect Balance
Balance = 5000
This is wrong because:
10000 - 7000 - 5000 = -2000
This problem is called:
Lost Update Problem
Common Production Problems
| Problem | Description |
|---|---|
| Race Condition | Multiple services update same data simultaneously |
| Dirty Write | One transaction overwrites another transaction |
| Lost Update | One update gets overwritten |
| Duplicate Processing | Same request processed multiple times |
| Stale Data | Old data used for update |
Production-Level Solutions
- Optimistic Locking
- Pessimistic Locking
- Distributed Locking
- Idempotency
- Event Ordering
- Single Writer Principle
- Kafka Partitioning
- Database Transactions
- Versioning
- CQRS Pattern
Most Common Solution — Optimistic Locking
Optimistic locking assumes conflicts are rare.
It is widely used in high-scale production systems.
How Optimistic Locking Works
A version field is added to the table.
Table Example
Account Table ID | BALANCE | VERSION 1 | 10000 | 1
JPA Entity Example
@Entity
public class Account {
@Id
private Long id;
private Double balance;
@Version
private Integer version;
}
Update Flow
Transaction 1 Reads: Balance = 10000 Version = 1 Transaction 2 Reads: Balance = 10000 Version = 1
Transaction 1 Updates Successfully
Balance = 3000 Version = 2
Transaction 2 Tries Update
But database version already changed.
Update fails with:
OptimisticLockException
Benefits
- Prevents lost updates
- Better performance
- No long database locking
- Highly scalable
Retry Logic Example
@Retryable(
value = OptimisticLockException.class,
maxAttempts = 3
)
public void updateBalance() {
accountRepository.save(account);
}
Pessimistic Locking
Pessimistic locking locks the row immediately.
How It Works
Transaction 1 Locks Row
↓
Transaction 2 Waits
JPA Pessimistic Lock Example
@Lock(LockModeType.PESSIMISTIC_WRITE)
@Query("SELECT a FROM Account a WHERE a.id=:id")
Account findAccount(Long id);
Advantages
- Strong consistency
- No concurrent modifications
Disadvantages
- Reduced performance
- Deadlock possibility
- Poor scalability
When to Use Pessimistic Locking?
- Banking transactions
- Stock trading
- Wallet balance updates
- Critical financial operations
Distributed Locking
In distributed systems, multiple service instances may update same data.
Production Example
Payment Service Pod 1 Payment Service Pod 2 Payment Service Pod 3
All pods try updating same account simultaneously.
Solution — Redis Distributed Lock
SETNX account-lock true
Only one service instance gets the lock.
Redisson Example
RLock lock = redissonClient.getLock("account-lock");
lock.lock();
try {
updateBalance();
} finally {
lock.unlock();
}
Benefits
- Prevents concurrent updates
- Works across multiple servers
- Suitable for distributed systems
Idempotency
Sometimes duplicate requests occur due to retries.
Real Banking Example
Payment API Timeout
↓
Client Retries Same Request
Without idempotency:
Money Deducted Twice
Production Solution
Use unique transaction ID.
if(transactionAlreadyProcessed(transactionId)) {
return;
}
Kafka Partitioning
Kafka guarantees ordering within a partition.
Example
Use Account ID as Kafka key.
kafkaTemplate.send(
"transaction-topic",
accountId,
transaction
);
Benefit
All updates for same account go to same partition.
This prevents out-of-order updates.
Single Writer Principle
Only one service should own and update specific data.
Bad Design
Order Service Updates Inventory Inventory Service Updates Inventory
Conflict possibility increases.
Better Design
Only Inventory Service Updates Inventory
Other services send requests/events only.
CQRS Pattern
Separate read and write operations.
Example
Write Database → Updates Read Database → Queries
Improves scalability and reduces conflicts.
Event Sourcing
Instead of updating current state directly:
- Store all events
- Rebuild state from events
Banking Example
Account Created Money Deposited Money Withdrawn UPI Payment Completed
Current balance derived from events.
Database Isolation Levels
Isolation levels help control concurrency problems.
Common Isolation Levels
| Isolation Level | Behavior |
|---|---|
| READ UNCOMMITTED | Allows dirty reads |
| READ COMMITTED | Prevents dirty reads |
| REPEATABLE READ | Prevents non-repeatable reads |
| SERIALIZABLE | Highest consistency |
Production Recommendation
- READ COMMITTED → Commonly used
- SERIALIZABLE → Critical banking operations
Real Production Incident
Issue
A wallet application faced duplicate balance deductions during payment retries.
Root Cause
- No idempotency
- Concurrent balance updates
- No optimistic locking
Fixes Applied
- Added version column
- Implemented optimistic locking
- Added Redis distributed lock
- Introduced idempotency key
- Used Kafka partitioning by wallet ID
Final Result
Before: Duplicate deductions occurred After: No data conflicts Consistent balance updates
Production Best Practices
| Technique | Purpose |
|---|---|
| Optimistic Locking | Prevent lost updates |
| Pessimistic Locking | Strong consistency |
| Distributed Lock | Cross-instance locking |
| Idempotency | Avoid duplicate processing |
| Kafka Partitioning | Maintain event ordering |
| Single Writer Principle | Avoid update conflicts |
| CQRS | Separate reads and writes |
| Versioning | Detect concurrent modifications |
Final Interview Answer
If multiple microservices update the same data simultaneously, I would avoid data conflicts using techniques like optimistic locking, pessimistic locking, distributed locking, idempotency, and Kafka partitioning. In most production systems, optimistic locking with a version column is preferred because it prevents lost updates without heavy locking overhead. For critical financial operations, I may use pessimistic locking or Redis distributed locks to ensure strong consistency. I would also enforce the Single Writer Principle, implement idempotency to avoid duplicate processing, and use Kafka partitioning to maintain event ordering in distributed systems.