Microservices & Distributed Systems Interview Questions¶

Architecture patterns, scalability, resilience, messaging, and distributed computing principles.

1. When should you use microservices vs monolith?¶

View Answer

Monolith: Single Unified Application

All features in one codebase, one deployment unit.

Advantages:

✅ Simple to build initially
✅ Easier debugging (single process)
✅ Single database (ACID transactions)
✅ Better performance (in-process calls)

Disadvantages:

❌ Hard to scale (scale entire app)
❌ Tight coupling (changes affect entire system)
❌ Technology lock-in (one language/framework)
❌ Deployment risk (one bug breaks everything)

Microservices: Distributed Services

Multiple independent services, each with own responsibility.

Advantages:

✅ Independent scaling (scale what you need)
✅ Loose coupling (change one without affecting others)
✅ Technology freedom (different stacks per service)
✅ Faster deployment (smaller units)
✅ Fault isolation (one service down ≠ entire system down)

Disadvantages:

❌ Complex distributed system (network issues, latency)
❌ Harder debugging (logs across multiple services)
❌ Data consistency challenges (eventual consistency)
❌ Operational overhead (monitoring, deployment, versioning)
❌ Network latency (inter-service calls)

Choose Monolith When

MVP/startup (speed to market)
Small team (coordination overhead)
Simple, well-defined domain
ACID transactions critical
Low scalability requirements

Choose Microservices When

Large team (independent development)
Scalability required (different services scale differently)
Technology diversity needed
Independent deployment important
Fault isolation critical

Recommendation

Start with monolith, migrate to microservices when you outgrow it. Most companies shouldn't start with microservices.

📚 Learn more: Microservices Principles

2. What's the difference between synchronous and asynchronous communication?¶

View Answer

Synchronous Communication

Caller waits for response (blocking).

Examples: REST API calls, gRPC, direct method calls

Client calls Service A
Client blocks, waiting for response
Service A processes and returns result
Client receives response

Advantages:

✅ Simple to implement
✅ Immediate feedback
✅ Error handling straightforward

Disadvantages:

❌ Tight coupling (client waits for server)
❌ Cascading failures (if Service A slow, client slow)
❌ Less resilient (network issues block caller)
❌ Poor scalability (threads waiting)

Asynchronous Communication

Caller doesn't wait for response (fire-and-forget or callback).

Examples: Message queues (Kafka, RabbitMQ), event-driven, webhooks

Client sends message to Queue
Client continues immediately
Service A consumes message when ready
Service A processes and publishes result event
Client receives event via callback/subscription

Advantages:

✅ Loose coupling (caller doesn't wait)
✅ Better resilience (queue buffers load)
✅ Scalability (no thread blocking)
✅ Can handle spikes (queue absorbs load)

Disadvantages:

❌ Complex (eventual consistency, ordering issues)
❌ Debugging harder (distributed flow)
❌ Guaranteeing delivery complex
❌ Error handling less obvious

Comparison Table

Aspect	Synchronous	Asynchronous
Latency	Immediate but may be slow	Delayed but responsive
Coupling	Tight	Loose
Complexity	Simple	Complex
Resilience	Poor (cascading failures)	Good (buffering, retry)
Best For	Request-response (GET, queries)	Long-running operations, events

📚 Learn more: Messaging & Events

3. Explain the saga pattern for distributed transactions¶

View Answer

The Problem: Distributed Transactions

ACID transactions don't work across multiple databases/services. Need alternative pattern.

Example: Money Transfer

Service A: Debit account from_account
Service B: Credit account to_account
If Service B fails, Service A's debit is stuck!

The Saga Pattern: Two Approaches

Orchestration (Centralized Coordinator)

One service orchestrates the saga steps.

Saga Orchestrator
  ↓ Step 1: Debit Account A
  ← Success
  ↓ Step 2: Credit Account B
  ← Failure → Compensate (refund A)

Code Example:

@Service
public class TransferSaga {
    public void transfer(String fromId, String toId, BigDecimal amount) {
        try {
            Step 1: accountService.debit(fromId, amount);
            Step 2: accountService.credit(toId, amount);
        } catch (Exception e) {
            // Compensate: reverse the debit
            accountService.credit(fromId, amount);
        }
    }
}

Choreography (Event-Driven)

Each service listens for events and publishes new events.

AccountService: "TransferInitiated" event
  → (publishes) "MoneyDebited"
    → LedgerService listens
    → (publishes) "MoneyCredited"
      → If fails: (publishes) "TransferFailed"
        → AccountService listens and refunds

Comparison

Aspect	Orchestration	Choreography
Coordination	Centralized (Saga Orchestrator)	Decentralized (event listeners)
Complexity	Simpler to understand	Complex (distributed logic)
Coupling	Tight (orchestrator knows all steps)	Loose (services listen to events)
Testing	Easier (centralized logic)	Harder (distributed)

Compensating Transactions (Rollback)

For each forward step, define a compensating transaction:

Debit: Compensate with Credit
Create Order: Compensate with Cancel Order
Reserve Inventory: Compensate with Release Inventory

📚 Learn more: Microservices Patterns

4. How does the circuit breaker pattern work?¶

View Answer

The Problem

If a service is down, repeatedly calling it wastes resources and causes timeouts.

Normal:  Request → Service → Success
Broken:  Request → Service (timeout) → Retry → Timeout → ...

Circuit Breaker Solution

Three states: CLOSED → OPEN → HALF_OPEN → CLOSED

┌─────────────┐
│   CLOSED    │ ← All requests pass through
├─────────────┤
│ Failure count exceeds threshold
├─────────────┤
│    OPEN     │ ← Requests fail immediately (no call to service)
├─────────────┤
│ After timeout
├─────────────┤
│ HALF_OPEN   │ ← Allow single test request
├─────────────┤
│ If success: Back to CLOSED
│ If failure: Back to OPEN

Implementation (Spring Cloud)

@Service
public class PaymentService {

    @CircuitBreaker(
        name = "paymentService",
        fallback = "paymentFallback"
    )
    public Payment processPayment(Order order) {
        return paymentGateway.charge(order);
    }

    // Fallback when circuit is OPEN
    public Payment paymentFallback(Order order, Exception e) {
        return new Payment("PENDING", "Service temporarily unavailable");
    }
}

States Explained

State	Behavior	Requests Pass?
CLOSED	Normal operation, count failures	✅ Yes
OPEN	Reject all requests immediately	❌ No (return fallback)
HALF_OPEN	Allow single test request	⚠️ One only

Benefits

Fail fast: Don't wait for timeouts
Resource protection: Stop wasting threads/connections
Time to recover: Gives failing service time to recover
Cascading failure prevention: Stop propagating errors

📚 Learn more: Circuit Breaker Patterns

5. What's eventual consistency and when is it acceptable?¶

View Answer

Eventual Consistency Definition

Data across services will eventually be consistent, but not immediately.

Short-term: Different services may have different views of data Long-term: Data will synchronize

Time 0: Account A has $100
Time 0: Transfer $50 initiated
Time 1: Service A shows $50 (debited)
Time 1: Service B still shows old $0 (not credited yet)
Time 2: Service B syncs and shows $50 (eventually consistent)

Strong Consistency vs Eventual Consistency

Property	Strong Consistency	Eventual Consistency
Data Sync	Immediate	After some delay
Implementation	Distributed transactions, locks	Event-driven, message queue
Performance	Slower (must wait for sync)	Faster (no waiting)
Availability	Lower (may be unavailable during sync)	Higher (always available)

When Eventual Consistency is Acceptable

✅ Social Media: Like count doesn't need to be perfect immediately
✅ Analytics: Reports don't need real-time accuracy
✅ Caching: Cache stale data is acceptable
✅ Read-heavy: Most users just reading, not writing

When Strong Consistency is Required

❌ Banking: Money transfer must be atomic
❌ Inventory: Cannot oversell
❌ Payments: Charge must debit account correctly

Strategies for Eventual Consistency

Event-driven: Publish events, consumers update independently
Message queues: Async updates via Kafka, RabbitMQ
CQRS: Separate read/write models, reconcile later
Compensating transactions: Fix inconsistencies with sagas

📚 Learn more: System Design Fundamentals

6. Explain the database per service pattern and its challenges¶

View Answer

Pattern: Database Per Service

Each microservice has its own private database. No shared database.

UserService      OrderService      PaymentService
    ↓                  ↓                  ↓
  UserDB          OrderDB            PaymentDB
(Private)       (Private)           (Private)

Advantages

✅ Loose coupling: Database schema changes don't affect other services
✅ Technology choice: Each service can use optimal DB (SQL, NoSQL, etc.)
✅ Scalability: Scale each database independently
✅ Data privacy: Services can't accidentally access other service's data

Challenges

1. Distributed Transactions

Cannot use ACID transactions across databases.

Solution: Saga pattern (orchestration or choreography)

2. Data Consistency

Services may have inconsistent views of data.

User updates: UserService DB updated
But OrderService doesn't know yet

Solution: Event-driven architecture, eventual consistency

3. Complex Queries Across Services

Cannot JOIN tables across databases.

-- IMPOSSIBLE: User and Order in different DBs
SELECT u.name, COUNT(o.id)
FROM users u
JOIN orders o ON u.id = o.user_id;

Solution: CQRS (materialized views), API calls to aggregate data

4. Reporting & Analytics

Difficult to aggregate data from multiple services.

Solution: Data warehouse, event streaming, analytics DB

Database Selection Per Service

UserService → PostgreSQL (relational)
OrderService → MongoDB (documents)
PaymentService → Redis (key-value cache)
AnalyticsService → Elasticsearch (search)

Recommendation

Start monolithic (shared DB), migrate to database-per-service as you scale. The operational overhead is significant.

📚 Learn more: Microservices Patterns

7. What's a consumer group and how do offsets work in Kafka?¶

View Answer

Kafka Topic Structure

Topic: order-events
├─ Partition 0: [msg0] [msg1] [msg2] ...
├─ Partition 1: [msg0] [msg1] [msg2] ...
└─ Partition 2: [msg0] [msg1] [msg2] ...

Each partition is an immutable log of messages in order.

Offset: Message Position

Offset = unique sequential identifier for each message within a partition.

Partition 0:
  Offset 0: {"order": "123", "amount": "$100"}
  Offset 1: {"order": "124", "amount": "$200"}
  Offset 2: {"order": "125", "amount": "$150"}
  ↑
  Consumer reads from here (offset tracking)

Consumer Group: Multiple Consumers

Multiple consumers reading same topic in parallel.

Topic: orders (3 partitions)

Consumer Group A:
  Consumer 1 → Partition 0
  Consumer 2 → Partition 1
  Consumer 3 → Partition 2
  (Each reads independently, parallel processing)

Consumer Group B:
  Consumer 1 → All 3 partitions
  (Different group, reads entire topic)

How It Works

Kafka stores: Consumer Group → Topic → Partition → Offset

Consumer Group A:
  Topic=orders, Partition=0, Offset=42 (last message read)
  Topic=orders, Partition=1, Offset=150
  Topic=orders, Partition=2, Offset=88

Offset Commit Strategies

Strategy	When	Semantics
Auto-commit	After polling	At-most-once (may lose messages)
Manual commit after processing	After business logic	At-least-once (may duplicate)
Transactional	With database transaction	Exactly-once (best)

Example: Consumer Group

// Consumer 1: Process Partition 0
while (true) {
    ConsumerRecords<String, String> records = consumer.poll(100);
    for (ConsumerRecord record : records) {
        // Process message
        processOrder(record.value());
    }
    consumer.commitSync();  // Update offset
}

Rebalancing

If a consumer dies, its partitions reassigned to others.

Before: C1→P0, C2→P1, C3→P2
C2 crashes
After:  C1→P0+P1, C3→P2 (rebalance)

📚 Learn more: Kafka & Messaging

8. How do you handle distributed transactions?¶

View Answer

The Problem: Distributed Transactions

Traditional ACID transactions don't work when data is spread across multiple services/databases.

Service A: Debit $50
Service B: Credit $50
If B fails, A's change is stuck!

Solutions

1. Saga Pattern (Primary Approach)

Orchestrate multi-step transactions with compensating transactions.

See Question 3 for detailed explanation.

2. Event Sourcing

Store all state changes as immutable events.

Events Log:
  - TransferInitiated(from=A, to=B, amount=50)
  - MoneyDebited(account=A, amount=50)
  - MoneyCredited(account=B, amount=50)

Replay events to rebuild state

3. CQRS (Command Query Responsibility Segregation)

Separate read and write models.

Write:  Update primary database
        Publish event
Read:   Update materialized views
        Eventually consistent

4. Two-Phase Commit (❌ Avoid)

Coordinator says "prepare to commit", then "commit".

Theoretical but rarely used in practice:

❌ Blocking (locks held during coordination)
❌ Slow (requires multiple round trips)
❌ Coordinator failure = deadlock

Practical Approach

Scenario	Solution
Money transfer	Saga with compensating transactions
Inventory + Order	Event-driven with eventual consistency
Audit trail needed	Event Sourcing
Complex queries needed	CQRS with materialized views

Guarantees

At-most-once: Messages may be lost
At-least-once: Messages may be duplicated
Exactly-once: Perfect, but expensive (use for critical operations)

📚 Learn more: Distributed Patterns

Additional Topics¶

See Sample Topics for more areas to explore: - How do you define service boundaries? - What's service discovery and why do you need it? - How do you monitor and trace requests across services? - Explain the bulkhead pattern - When should you use RabbitMQ vs Kafka? - How do you handle network failures in distributed systems?