Order Service Created Order Successfully but Payment Service Failed — How Will You Maintain Data Consistency?

In microservices architecture, distributed transactions are one of the biggest challenges.

Suppose:

• Order Service creates the order successfully
• Payment Service fails

Now the system becomes inconsistent.

---

Real-Time E-Commerce Production Scenario

Flow

Customer Places Order
        ↓
Order Service Creates Order
        ↓
Payment Service Processes Payment
        ↓
Inventory Service Reserves Product
        ↓
Notification Service Sends SMS

---

Problem Scenario

Suppose:

• Order Service saved order successfully
• Payment Service failed due to bank timeout

Now System Problem

Order Status = CREATED
Payment Status = FAILED

Customer sees order created, but payment failed.

This is called:

Distributed Data Inconsistency

---

Why Traditional Database Transactions Do Not Work?

In monolithic applications:

Single Database
Single Transaction
ROLLBACK possible

But in microservices:

• Each service has its own database
• Separate deployments
• Separate transactions

So distributed rollback becomes difficult.

---

Production-Level Solutions

• Saga Pattern
• Compensation Transaction
• Outbox Pattern
• Event-Driven Architecture
• Idempotency
• Retry Mechanism
• Dead Letter Queue

---

Most Popular Solution — Saga Pattern

Saga Pattern is the industry standard solution for distributed transaction management.

---

How Saga Pattern Works

Step-by-Step Flow

Step 1:
Order Service Creates Order

Status = PENDING
        ↓

Step 2:
Publish OrderCreated Event to Kafka
        ↓

Step 3:
Payment Service Consumes Event
        ↓

Step 4:
Payment Success
        ↓

Update Order Status = CONFIRMED

---

Failure Scenario

Order Created
       ↓
Payment Failed
       ↓
Compensation Transaction Triggered
       ↓
Order Cancelled

---

Final Consistency Achieved

Order Status = CANCELLED
Payment Status = FAILED

System becomes consistent again.

---

Real Production Example

Suppose Flipkart or Amazon order placement:

Customer Orders iPhone
       ↓
Order Service Saves Order
       ↓
Payment Gateway Timeout Occurred

Now:

• Order should not remain CONFIRMED
• Inventory should not remain reserved
• Customer should not receive success message

Compensation events must rollback business operations.

---

Production Architecture

Order Service
     ↓
Kafka Topic
     ↓
Payment Service
     ↓
Inventory Service
     ↓
Notification Service

---

Step 1: Order Service Creates Pending Order

Order Entity

@Entity
public class OrderEntity {

    @Id
    private Long orderId;

    private String status;

}

---

Save Order as PENDING

public OrderResponse createOrder(OrderRequest request) {

    OrderEntity order = new OrderEntity();

    order.setStatus("PENDING");

    repository.save(order);

    kafkaTemplate.send(
        "order-created-topic",
        order.getOrderId()
    );

    return new OrderResponse("ORDER_CREATED");
}

---

Why Use PENDING Status?

Never mark order as SUCCESS immediately.

Because:

• Payment not completed yet
• Inventory not reserved yet
• Other services may fail

---

Step 2: Payment Service Consumes Event

@KafkaListener(topics = "order-created-topic")
public void processPayment(Long orderId) {

    boolean paymentSuccess = bankAPI.process();

    if(paymentSuccess) {

        kafkaTemplate.send(
            "payment-success-topic",
            orderId
        );

    } else {

        kafkaTemplate.send(
            "payment-failed-topic",
            orderId
        );
    }
}

---

Step 3: Order Service Handles Payment Result

Payment Success Consumer

@KafkaListener(topics = "payment-success-topic")
public void paymentSuccess(Long orderId) {

    OrderEntity order =
        repository.findById(orderId).get();

    order.setStatus("CONFIRMED");

    repository.save(order);
}

---

Payment Failed Consumer

@KafkaListener(topics = "payment-failed-topic")
public void paymentFailed(Long orderId) {

    OrderEntity order =
        repository.findById(orderId).get();

    order.setStatus("CANCELLED");

    repository.save(order);
}

---

Final State

Scenario	Order Status	Payment Status
Payment Success	CONFIRMED	SUCCESS
Payment Failed	CANCELLED	FAILED

---

What Is Compensation Transaction?

Compensation transaction means undoing previously completed business actions.

---

Example

Step 1:
Inventory Reserved
       ↓

Step 2:
Payment Failed
       ↓

Step 3:
Release Inventory

This rollback logic is called compensation transaction.

---

Inventory Compensation Example

@KafkaListener(topics = "payment-failed-topic")
public void releaseInventory(Long orderId) {

    inventoryRepository.release(orderId);
}

---

What Happens If Kafka Message Fails?

In production:

• Kafka broker may fail
• Consumer may crash
• Network issue may occur

This can create inconsistencies.

---

Production Solution — Outbox Pattern

Outbox Pattern ensures database update and Kafka event publishing happen reliably.

---

Problem Without Outbox

Order Saved in DB
      ↓
Kafka Publish Failed

Now other services never receive event.

---

Outbox Table Example

CREATE TABLE outbox_events (

    id BIGINT PRIMARY KEY,

    event_type VARCHAR(100),

    payload TEXT,

    status VARCHAR(20)
);

---

Order + Outbox Save in Same Transaction

@Transactional
public void createOrder() {

    orderRepository.save(order);

    outboxRepository.save(event);
}

Now both are committed together.

---

Outbox Publisher Job

@Scheduled(fixedDelay = 5000)
public void publishEvents() {

    List<OutboxEvent> events =
        repository.findPending();

    for(OutboxEvent event : events) {

        kafkaTemplate.send(
            event.getTopic(),
            event.getPayload()
        );

        event.setStatus("COMPLETED");
    }
}

---

Why Outbox Pattern Is Important?

It prevents:

• Message loss
• Partial transactions
• Inconsistent systems

---

Idempotency Handling

Sometimes Kafka retries same message multiple times.

Without idempotency:

Payment Deducted Twice

Very dangerous in banking systems.

---

Production Solution

Store unique transaction ID.

if(transactionAlreadyProcessed(transactionId)) {

    return;
}

---

Retry Mechanism

Temporary failures should be retried.

Spring Kafka Retry Example

@RetryableTopic(
attempts = "3",
backoff = @Backoff(delay = 2000)
)
@KafkaListener(topics = "payment-topic")
public void consume(String message) {

    process(message);
}

---

Dead Letter Queue (DLQ)

If retries fail:

• Message goes to DLQ
• Manual investigation happens

---

DLQ Example

payment-topic.DLT

---

Real Banking Example

Suppose:

• Money debited from sender account
• Receiver account credit failed

Compensation transaction:

Refund sender account

This maintains consistency.

---

Production Best Practices

Technique	Purpose
Saga Pattern	Distributed transaction management
Compensation Transaction	Rollback business actions
Outbox Pattern	Reliable event publishing
Kafka	Async communication
Retry	Handle temporary failures
DLQ	Handle failed messages
Idempotency	Avoid duplicate processing
PENDING Status	Avoid premature success

---

Final Interview Answer

If Order Service creates the order successfully but Payment Service fails, I would maintain data consistency using Saga Pattern with compensation transactions. Initially, the order is created with PENDING status and an event is published through Kafka. If payment succeeds, the order status is updated to CONFIRMED. If payment fails, a compensation transaction updates the order status to CANCELLED and releases reserved inventory. To ensure reliable event publishing, I would use the Outbox Pattern. Additionally, I would implement retries, dead letter queues, and idempotency to handle failures safely in production systems.