Source URL: https://cloud.google.com/blog/products/databases/scann-for-alloydb-index-is-ga/
Source: Cloud Blog
Title: AlloyDB supercharges PostgreSQL vector search with accuracy, speed, and 1B+ scale
Feedly Summary: In our 20 years of experience integrating AI into real-world applications an important theme emerges: the key to building enterprise gen AI applications is having a trustworthy, scalable data foundation that supports the scale and performance needs of the largest workloads. When you’re building a gen AI or search application, you need high-quality results in real time, grounded in your application’s latest data. And you need solutions that are easy for developers to build and manage, and low-cost to run at scale.
Today, we’re announcing the general availability of the ScaNN for AlloyDB index to help enterprises build scalable,performant, and accurate gen AI and search applications. AlloyDB for PostgreSQL is our fully managed PostgreSQL-compatible service for demanding enterprise workloads, including 4x faster transactional workloads and up to 100x faster analytical queries than standard PostgreSQL. Now, the ScaNN index brings 12 years of Google research — the same technology used at Google to power some of our billion+ user services — to AlloyDB for PostgreSQL to supercharge vector database workloads as well.
With the ScaNN index, AlloyDB combines the performance of a vector-optimized database, delivering accurate results in record time and scaling to support more than a billion vectors — with the query flexibility, enterprise features, and multi-workload capabilities of a relational database.
The ScaNN for AlloyDB index
The ScaNN for AlloyDB index is the first PostgreSQL-compatible index that can scale to support more than a billion vectors while maintaining state-of-the-art query performance across the whole spectrum, delivering high performance at <25ms p95 latency at 1B vectors with 95% recall. It provides up to 4x faster vector queries than the HNSW index in standard PostgreSQL. Moreover, the ScaNN index in AlloyDB offers important benefits for real-world workloads, notably fast index build times — to improve developer productivity and reduce expensive bottlenecks for large-scale workloads. It also has efficient memory usage — typically 3-4x less memory than the HNSW index in standard PostgreSQL — to help larger workloads run on smaller machines and improve performance for hybrid workloads. Finally, AlloyDB supports indices that are much larger than what can fit in main memory, allowing customers to right-size for their specific workloads. Together, these improvements vastly expand the set of vector database workloads that can efficiently run on AlloyDB while reducing the need for specialized tools.
At the same time, AlloyDB offers all the benefits of a commercial-grade relational database. Full PostgreSQL compatibility gives developers a simple and flexible query language that can be used to query vectors and scalar data together, supporting everything from simpler filters to complex joins to hybrid searches on a single database. AlloyDB supports full transactional consistency and high write throughput. In fact, the ScaNN index in AlloyDB has up to 10x higher write throughput than the HNSW index in standard PostgreSQL, so customers with very high update rates can rest assured that writes will be performant and readily available for querying. Finally, AlloyDB has a customer-friendly 99.99% availability SLA that includes maintenance, and critical enterprise capabilities around high availability, disaster recovery, data protection, security, and more.
The new ScaNN index for AlloyDB is fully PostgreSQL- and pgvector-compatible, making this technology easily accessible to the professional developers that now use the most popular open source database, PostgreSQL.
Next steps
To learn more about the ScaNN for AlloyDB index, check out our overview of the ScaNN for AlloyDB index, or read our ScaNN for AlloyDB whitepaper for a deep dive into Google’s ScaNN algorithm and how we’ve implemented it in PostgreSQL and AlloyDB.
ScaNN for AlloyDB is now generally available in AlloyDB in Google Cloud. Follow our quickstart guide to creating an AlloyDB instance, then follow our documentation to get started with easy and fast vector queries. You can also now try AlloyDB for free with our 30-day free trials.
AI Summary and Description: Yes
Summary: The text discusses the launch of the ScaNN for AlloyDB index, a significant advancement designed to support scalable, performant generative AI applications and search capabilities within enterprise contexts. By incorporating over a decade of Google research, this technology enhances AlloyDB for PostgreSQL, facilitating the handling of billions of vectors with low latency and high recall.
Detailed Description:
The provided text announces the general availability of the ScaNN (Scalable Nearest Neighbors) index for AlloyDB, emphasizing its significance for enterprises developing generative AI and search applications. Key points and insights are as follows:
- **Core Offering**:
- ScaNN for AlloyDB is designed to enhance the performance of generative AI applications and search functionalities, built on a solid data foundation necessary for supporting large-scale workloads.
- **Performance Capabilities**:
- **Latency and Recall**: Achieves <25ms p95 latency at 1 billion vectors with a recall rate of 95%.
- **Speed**: Offers up to 4x faster vector queries compared to standard PostgreSQL's HNSW index.
- **Transactional Throughput**: ScaNN in AlloyDB has a 10x higher write throughput than the HNSW index, crucial for high update rate use cases.
- **Resource Efficiency**:
- **Memory Usage**: Typically consumes 3-4x less memory than competing indices, making it efficient for larger workloads on smaller machines.
- **Index Build Times**: Features accelerated index build times, enhancing developer productivity while avoiding costly bottlenecks.
- **Compatibility and Integration**:
- Fully compatible with PostgreSQL and pgvector, making it easily accessible to developers using PostgreSQL, allowing for flexible query capabilities that support both vector and scalar data.
- Supports complex queries and hybrid searching capabilities on a single database, streamlining the development process.
- **Enterprise Features**:
- Offers features vital for enterprise operations such as high availability, disaster recovery, and security, backed by a service-level agreement (SLA) of 99.99% availability.
- **Accessibility**:
- Available on Google Cloud, offering a quickstart guide for creating an AlloyDB instance and easy integration for users, including a 30-day free trial for new users to explore the capabilities.
These advancements position AlloyDB with the ScaNN index as a robust framework for enterprises looking to leverage generative AI in their workflows while maintaining high performance, efficiency, and security within their data architectures. This is particularly relevant for professionals in AI, cloud, and infrastructure security as it highlights the need for scalable solutions that combine performance with effective data management and compliance.