Zilliz Announces Key Contributions to Milvus 2.1, the Leading Open-Source Vector Database for Structured and Unstructured Data

Popular with enterprise users, Milvus handles billion-fold, high-dimensional vector representations of unstructured data and simplifies deploying AI and MLOps at scale

Zilliz, whose founders created the Milvus open-source project, announced major contributions to the Milvus 2.1 release. The added functionality further bridges the gap between data pools, removes data silos, and offers performance and availability enhancements to address developers’ most common concerns. Milvus is one of the world’s most advanced vector databases, capable of managing massive quantities of both structured and unstructured data to accelerate the development of next-generation data fabric.

Latest Aithority Insights: AiThority.com to Attend The Character of AI – A Technology Ethics Conference (Virtual)

“When it comes to unstructured data, solutions offered by industry incumbents tend to be add-on capabilities or tools in a legacy database management system, whereas Milvus is designed around unstructured data from day one and is now offering more built-in capabilities to unlock more powerful and integrated data processing.”

A graduated-stage open-source project under the LF AI & Data Foundation, Milvus is built for scalable similarity search and used by a wide array of enterprises across industries. It embraces distributed architecture and can easily scale as data volumes and workloads increase. Highly scalable, reliable, and exceptionally fast, Milvus supports DML operations (adding, deleting, updating) and near real-time search of vectors on a trillion-byte scale.

With this 2.1 update, Milvus sees significant improvement in its performance, reducing search latency on million-scale datasets to five milliseconds, while further simplifying deployment as well as ops workflow.

Bridging Gaps and Improving Performance

Machine learning is producing vast pools of scalar and vector data on a daily basis. With the introduction of more scalar data types, Milvus 2.1 is bridging this critical gap between data pools.

AI and ML News: AI: Continuing the Chase for Brain-Level Efficiency

“Data silos can now be better integrated and linked, enabling businesses to unlock the full potential of their data,” said Milvus project maintainer Xiaofan James Luan, who also serves as the director of engineering at Zilliz. “When it comes to unstructured data, solutions offered by industry incumbents tend to be add-on capabilities or tools in a legacy database management system, whereas Milvus is designed around unstructured data from day one and is now offering more built-in capabilities to unlock more powerful and integrated data processing.”

Zilliz’s contributions to the 2.1 release specifically include:

• An overall performance boost including reduced latency; highly improved throughput for small-NQ application scenarios, such as reverse image search and intelligent chatbot; support of multiple memory replicas for small tables to increase throughput; and 2x increase in search performance.
• Improved scalar data processing that adds Varchar into supported data types and supports creating indexes for scalar data, taking hybrid search to a more intuitive level.
• Production-grade enhancements and higher availability, with clearer monitoring metrics for observability, easier and more diverse deployment options including embedded Milvus for simple deployment and Ansible for offline deployment, integration that supports Kafka as log storage, and enhanced security supporting password protection and TLS connection.
• A developer-friendly ecosystem in the making that includes more tutorials for building real-world applications, connecting Milvus with open-source vector data ETL framework Towhee; and that adds Feder, an open-source tool that helps Milvus users select the index best suited to their application scenario by visualizing the process of vector similarity search.

In addition to the integration and security features enumerated, Milvus will provide more functionalities essential to modern security mechanisms, including ACL (Access Control Lists) and advanced encryption methods.

AI ML in Marketing: AI and Big Data Analysis Used to Find Brands’ Emotional Connection

[To share your insights with us, please write to sghosh@martechseries.com]