Replication Patterns for High Availability and Read Scaling

Replication is the backbone of resilient database architectures, enabling both high availability and read scaling without requiring application rewrites. By maintaining synchronized copies of data across multiple nodes, replication ensures that a single server failure doesn't become a production outage, while simultaneously distributing read traffic to reduce load on the primary. The key challenge lies in choosing the right pattern for your workload, understanding the consistency trade-offs, and implementing failover procedures that don't introduce more downtime than they prevent.

Why Replication Matters for HA and Read Scaling

High availability (HA) requires that your database remains accessible even when individual components fail. Replication achieves this by maintaining one or more standby copies that can take over if the primary becomes unavailable. Read scaling addresses a different problem: when a single server cannot handle the volume of read queries, replicas distribute the load horizontally. These two goals often overlap—the same replicas that provide HA can also serve read traffic—but they impose different requirements on replication topology and consistency guarantees. Without replication, a single server failure causes complete downtime, and read-heavy workloads quickly saturate CPU and I/O capacity.

Leader-Follower Replication: Setup and Failover

Leader-follower (also called primary-replica or master-slave) is the most common pattern. One node accepts all writes and propagates changes to one or more followers. Followers typically serve read-only queries and can be promoted to leader if the primary fails. Setup involves configuring the leader to stream its write-ahead log (WAL) to followers, which replay the changes. For PostgreSQL, this uses streaming replication:

-- On the leader (postgresql.conf)
wal_level = replica
max_wal_senders = 5

-- On the follower (recovery.conf or postgresql.auto.conf)
primary_conninfo = 'host=leader_ip port=5432 user=replicator password=secret'
standby_mode = on

For MySQL, the equivalent uses binary log replication:

-- On the leader (my.cnf)
server-id = 1
log_bin = mysql-bin

-- On the follower
CHANGE MASTER TO
  MASTER_HOST='leader_ip',
  MASTER_USER='replicator',
  MASTER_PASSWORD='secret',
  MASTER_LOG_FILE='mysql-bin.000001',
  MASTER_LOG_POS=154;
START SLAVE;

Failover in this pattern requires promoting a follower to leader, either manually or automatically. The promoted node must have applied all committed transactions from the old leader to avoid data loss. In asynchronous replication, some transactions may still be in flight—a trade-off between performance and durability.

Multi-Leader and Peer-to-Peer Replication Patterns

Multi-leader replication allows multiple nodes to accept writes, each propagating changes to the others. This pattern is useful for geographically distributed deployments where latency to a single leader is unacceptable, or for applications that must remain writable during network partitions. However, it introduces write-write conflicts that must be resolved—typically through last-writer-wins (LWW), conflict-free replicated data types (CRDTs), or application-level merge logic. Peer-to-peer replication, common in Cassandra and Riak, extends this concept to a fully distributed ring where every node can accept reads and writes. The trade-off is increased complexity in conflict resolution and eventual consistency. Use multi-leader only when single-leader latency or availability constraints force it; the operational overhead is significantly higher.

Read Replicas: Scaling Reads Without Downtime

Read replicas are followers explicitly dedicated to serving read queries, offloading the leader's workload. They can be added or removed without affecting write availability. The primary consideration is routing: applications must direct read queries to replicas and write queries to the leader. This is typically handled by a connection pooler (PgBouncer, ProxySQL) or an application-level read/write splitter. For example, with a PostgreSQL connection string, you might configure a load balancer that routes to replicas:

# Application configuration (pseudo-code)
read_pool = [
  "postgresql://user:pass@replica1:5432/db",
  "postgresql://user:pass@replica2:5432/db"
]
write_pool = "postgresql://user:pass@leader:5432/db"

Adding a read replica typically involves taking a base backup from the leader, configuring replication, and waiting for it to catch up. The leader remains fully online during this process, so there is no downtime. However, replicas will lag behind the leader by some amount—the key challenge for read scaling.

Handling Replication Lag and Consistency Guarantees

Replication lag is the delay between a write being committed on the leader and appearing on a follower. In asynchronous replication, this can range from milliseconds to seconds under load. Applications that read from replicas may see stale data, which can cause issues for workflows like "write then immediately read your own update." Mitigations include: using synchronous replication for critical transactions (at the cost of write latency), routing read-after-write queries to the leader, or implementing session-level consistency where a replica is pinned to a user session. A common rule of thumb:

If your application cannot tolerate stale reads, route all writes and reads to the leader—replicas are for read-heavy, eventually-consistent workloads. Never assume a replica is up to date unless you explicitly verify replication lag.

For stronger guarantees, synchronous replication ensures that a write is committed on at least one replica before acknowledging the client. This reduces lag to near-zero but increases write latency and reduces availability if the replica fails. Most production systems use asynchronous replication with application-level awareness of lag.

Failover Strategies: Automated vs. Manual Promotion

Manual failover gives an operator full control: they can verify the leader is truly down, check replication lag, and promote a follower only when safe. The downside is downtime that lasts minutes to hours while the operator diagnoses and acts. Automated failover (using tools like Patroni, Repmgr, or Orchestrator) detects leader failure and promotes a follower within seconds. The trade-off is the risk of split-brain—two nodes both believing they are the leader—if the old leader comes back online. To avoid this, automated systems use a distributed consensus store (etcd, ZooKeeper) or a STONITH (shoot the other node in the head) mechanism. For most production systems, automated failover with a consensus quorum is preferred, but it requires careful testing. A manual failover procedure for PostgreSQL might look like:

-- On the follower to promote
SELECT pg_promote();

-- Update application connection strings to point to new leader
-- Reconfigure old leader as a follower once it recovers

Choosing the Right Pattern for Your Engine and Workload

The optimal replication pattern depends on your database engine and workload characteristics. For PostgreSQL and MySQL, leader-follower with asynchronous replication is the default and works well for most OLTP workloads. Use synchronous replication only when you need zero data loss and can tolerate higher write latency. For Cassandra or CockroachDB, peer-to-peer replication is built-in and handles multi-region deployments natively. For read-heavy workloads with a single writer, leader-follower with multiple read replicas is simplest. For multi-region writes, consider multi-leader only if you have a conflict resolution strategy. Key questions to ask: Can your application tolerate stale reads? How much write latency is acceptable? What is your recovery time objective (RTO)? The answers will guide whether you need synchronous replication, automated failover, or a fully distributed topology.

The short version

Replication is essential for both high availability and read scaling. Leader-follower is the simplest and most widely supported pattern; use it unless your workload demands multi-region writes. Read replicas scale reads but introduce lag—route critical reads to the leader. Automated failover reduces downtime but requires split-brain prevention. Choose based on your engine's native capabilities and your tolerance for consistency trade-offs. Test your failover procedure regularly; a pattern you never verify is a pattern that will fail in production.