Source URL: https://cloud.google.com/blog/products/databases/memorystore-cross-region-replication-and-single-shard-clusters/
Source: Cloud Blog
Title: Achieve global scale and greater flexibility with new Memorystore enhancements
Feedly Summary: Many Google Cloud customers need to build multi-region or globally distributed architectures with sub-millisecond latencies at scale — and with high availability. Memorystore for Redis Cluster and Valkey is Google Cloud’s fully managed, in-memory data store that delivers extreme performance, sub-millisecond latency, and high scalability for enterprise applications. Today, we are excited to announce the preview of two new features: cross-region replication and single-shard clusters that help customers achieve these goals.
Let’s dive into these two new features and how they establish Memorystore as a top choice for a key-value data store.
Cross-region replication
Since the launch of Memorystore for Redis Cluster, cross-region replication has been one of our customers’ most requested features, and it’s now available to all Memorystore for Redis Cluster users. Cross-region replication clusters offer several advantages for customers, including:
Enhanced business continuity with high resilience to regional failures
Ability to serve a global user base with low-latency reads in different regions
Lower cost for serving reads across regions
These capabilities are available out-of-the-box in all Google Cloud regions without any development work on your part.
How does cross-region replication work?Cross-region replication for Memorystore for Redis Cluster uses an active-passive architecture, where the primary cluster handles reads and writes while secondary cluster(s) are read-only. Each cluster gets its own independent endpoints for application access. The initial release supports up to two secondary clusters.
Our cross-region replication strategy prioritizes efficiency and resilience through:
Using the primary cluster as the source: All data replication originates from the primary cluster. This means secondary clusters only receive data from the primary, not from each other. This “hub-and-spoke" model prevents cascading failures — if one secondary cluster goes down, the others remain unaffected.
Optimized secondary replication: Within each secondary cluster, the primary replica replicates data to all its HA replicas. This minimizes the amount of cross-region traffic required to keep the replicas up-to-date and avoids extra network hops. The reduction of cross region traffic also ensures that secondary replicas are kept up to date in a cost optimal way without any additional charges for network replication to replicas within the secondary clusters.
Setting up cross-region replication When creating a new cluster, you can designate it as a secondary cluster to set up the cross-region replication relationship with the primary cluster. Memorystore for Redis Cluster automatically copies and applies the necessary settings to the secondary cluster. Most properties are kept in sync across the clusters by the underlying systems; this means if the primary cluster is scaled or relevant properties on the cluster are modified, Memorystore ensures that all secondary clusters also scale in sync with the primary. While most cluster properties are kept in sync by this mechanism, certain properties such as maintenance window slots may vary across the cluster and can be overridden as desired.
code_block
The diagram depicts a primary cluster in the us-east1 region with secondary clusters in us-west1 and asia-east1. In this configuration, the replication direction is always from us-east1 to the other regions. Cross-region replication allows for varying replica numbers on the primary and secondary clusters (maximum of two replicas), thereby enabling disaster recovery redundancy and global reads while providing the ability to optimize infrastructure costs as per the desired read throughput on secondary clusters.
Performing a switchoverSwitchovers allow you to switch the primary cluster to one of the secondary clusters, and switch the older primary cluster and other secondary clusters to replicate from the new primary cluster automatically. The following single command takes care of the switchover and performs the orchestration of changes across all the clusters in the cross-region replication group.
code_block
<ListValue: [StructValue([(‘code’, ‘gcloud redis clusters switchover asiaeast-cluster-id \\\r\n–project=projectid –region=asia-east1’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3ef2d1b6fa30>)])]>
Cross-region replication after switchover
Cross-region replication provides robust disaster recovery. Even if the primary cluster is down due to a regional outage, the switchover can be completed and the old primary cluster will reconcile its state as a secondary cluster automatically once it recovers from the outage. There is no manual intervention required, and you can even detach the primary cluster during an outage to move all traffic to the secondary cluster.
You can return the cluster configuration to the original cross-region replication configuration by performing another switchover back to the original primary (us-east1). You can also use this mechanism to regularly test disaster recovery scenarios, a best practice.
code_block
<ListValue: [StructValue([(‘code’, ‘gcloud redis clusters switchover useast-cluster-id –project=projectid –region=us-east1’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3ef2d1b6ffa0>)])]>
Cross-region replication pricingThere is no additional charge to enable cross-region replication for the Memorystore for Redis Cluster. You only pay for the actual infrastructure cost of the primary and secondary clusters, as one would incur for any regular cluster, and any inter-region networking charges you may incur for the cross-region data replication.
Single-shard clusters
While cross-region replication meets the needs of the customers who need a multi-region setup, at the same time we have customers who have expressed interest in standardizing on Memorystore for Redis Cluster and Valkey to support all of their workloads, regardless of size.. The current minimum cluster size would sometimes make it cost-prohibitive for them to adopt Memorystore for Redis cluster and Valkey. Single-shard clusters on Memorystore for Redis Cluster and Memorystore for Valkey provide cost-effective high availability for smaller workloads.
Until now, Memorystore for Redis Cluster and Valkey supported a minimum shard count of three and didn’t support a shard count of four due to lack of high availability of the open-source failover mechanism. This resulted in needing to overprovision instances for smaller workloads, leading to less efficient resource utilization.
Memorystore addresses the high availability gap in this failover mechanism through voting replicas and auxiliary nodes (as applicable) to reliably achieve failover quorum. These auxiliary nodes are internal nodes which host no data, have no client access, and are provided at no extra charge to help maintain quorum. Auxiliary nodes are used in scenarios (e.g., single shard with one replica) where there aren’t enough primary or replica nodes to ensure high availability of the failover quorum during network partitions or zone outages.
Provisioning single-shard clustersYou can now create both Memorystore for Redis Cluster and Memorystore for Valkey instances with a single shard (or two / four shards) across any node type, and subsequently dynamically scale out and back as before to right-size the instances. You can use the gcloud commands below to create these clusters:
code_block
<ListValue: [StructValue([(‘code’, ‘gcloud redis clusters create my-cluster-id \\\r\n–project=my-project –region=asia-east1 \\\r\n–network=projects/my-project/global/networks/default –replica-count=1 \\\r\n–node-type=redis-highmem-medium \\\r\n–shard-count=1\r\n\r\ngcloud beta memorystore instances create my-cluster-id \\\r\n–project=my-project \\\r\n–location=asia-east1 \\ –psc-auto-connections=network=projects/my-project/global/networks/default,projectId=my-project \\\r\n–-replica-count=1 \\\r\n–node-type=redis-highmem-medium \\\r\n–shard-count=1’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3ef2d1b6fd00>)])]>
At general availability, this will be possible while maintaining a multi-zone 99.99% SLA (or a single-zone 99.9% SLA) with replicas.
Low shard counts pricingThe new cluster sizes are priced exactly the same as the existing per-node cost, so you get the flexibility of smaller clusters with no additional premiums.
Get started today
Both cross-region replication and single-shard clusters are now available in Memorystore for Redis Cluster in all regions. Single-shard clusters are also available in Memorystore for Valkey Cluster. Get started with Memorystore.
AI Summary and Description: Yes
Summary: Google Cloud introduces two significant features for its Memorystore for Redis Cluster: cross-region replication and single-shard clusters. These enhancements not only address the needs for high availability and low latency across global architectures but also promote cost-efficiency for smaller workloads, catering specifically to a wide range of enterprise applications.
Detailed Description:
The text discusses new functionalities in Google Cloud’s Memorystore for Redis, emphasizing its utility for businesses requiring high scalability and low latency in their global operations. The two main features highlighted — cross-region replication and single-shard clusters — are designed to improve disaster recovery, availability, and cost management.
– **Cross-Region Replication:**
– This feature has been highly requested and is designed to enhance business continuity and reduce latency for global user bases.
– Key advantages include:
– **Business Continuity:** High resilience against regional failures allows for uninterrupted service.
– **Low-Latency Reads:** By serving a global user base from the nearest secondary cluster, the feature ensures quick data access.
– **Cost Efficiency:** It reduces the costs associated with serving reads across regions.
– **Architecture and Operation:**
– Utilizes an active-passive cluster design where the primary cluster manages writes and reads, while secondary clusters are read-only.
– An optimized hub-and-spoke model for data replication helps in preventing cascading failures and maintains cost-efficient data transfer.
– **Disaster Recovery Mechanisms:**
– Enables easy switchover processes to maintain operations during outages.
– Facilitates regular testing of disaster recovery scenarios, promoting better preparedness.
– **Single-Shard Clusters:**
– Allows for smaller workloads to utilize Memorystore without the burden of overprovisioning, thus improving resource utilization.
– Introduces a failover mechanism supported by voting replicas and auxiliary nodes, solving the previous challenges associated with the minimum shard size which was cost-prohibitive for some customers.
– Scalable design options enable the creation and dynamic resizing of clusters to better meet specific workload needs.
– **Pricing Model:**
– Both features are priced similarly to existing cluster node costs without additional premiums for the smaller single-shard clusters, making them financially accessible.
These enhancements not only solidify Google Cloud’s Memorystore as a competitive choice for enterprise-grade applications but also align with the increasing demand for robust, reliable infrastructure with stringent performance requirements. By addressing both high availability for large-scale operations and affordability for smaller applications, Google Cloud solidifies its commitment to catering to a diverse range of enterprise needs.