Vector database is not a separate database category
Why every database will become a vector database, sooner or later
In the near future, it’s becoming clear that:
every database will offer some form of vector search.
that includes graph, relational, document, and key-value databases, as well as caches.
the boundary between what’s a vector DB and what’s not will blur.
the current category of specialized “vector dbs” like Pinecone, Weaviate, Milvus, etc, will lose relative momentum and differentiation amidst the competition.
incumbent databases will try to capture the new RAG (retrieval augmented generation) workloads by using their existing workloads / customer base.
As a result, it's worth questioning whether “vector database” is a separate database category, or just a feature that any database can provide.
As generative AI takes off, a sizable fraction % of queries will be done in a “dense vector search” fashion. No database company is insane enough to lose this workload. Thus, pretty much any database that can store text will provide vector search.
And in fact, this “vector db-fication of databases” is already happening.
Until Q2 2023, “vector search” was mainly associated with startup database companies such as Pinecone, Milvus, Weaviate, and so on. But incumbents have quickly caught on, and now every cloud native vendor is entering the “vector search” market. Even Cloudflare, which doesn’t sell databases, has entered the market. That’s because any “data adjacent” company wants a piece of the RAG workload.
Cloudflare launches vectorize, announced on September 27th, 2023
MongoDB Atlas Vector Search launched on June 22nd, 2023
Databricks announced on June 28th, 2023
Oracle integrated vector database announced on September 19th, 2023
of course, companies such as Elastic and Microsoft already had vector DB offerings much earlier.
But there’s more going on here than just big companies catching FOMO. It genuinely makes sense for incumbent database players to offer vector search, because that eliminates unnecessary data movement to separate vector databases. Co-locating vectors and original documents also reduces latency. Thus, customers actually benefit when incumbents jump into this market.
Basically, having separate vector DBs can add to cost and complexity. Imagine you were a MongoDB shop, with over 500m documents stored cross-region. If you are using a separate vector DB, say Pinecone, that may require moving potentially billions of embeddings between two databases, cross regions. This costs a lot, not to mention complex, since you are responsible for generating the embeddings.
It’s faster, cheaper, and simpler if one database (Mongo, Elastic) just supported vector search.
Of course, offering vector search is also a defensive move. Retrieval Augmented Generation (RAG) is set to be one of the top 2 workloads in generative AI (the other being inference). Not offering vector search means losing the RAG workload, and customers migrating to other databases. This is existential threat to managed database companies.
And why not stop there? Incumbent databases will increasingly absorb the entire lifecycle of RAG workloads, including generating embeddings.
embeddings will increasingly be natively supported by databases (i.e. database users can just insert documents and databases will take care of generating embeddings locally to the vector storage)
even end-to-end RAG and re-ranking might be supported out of the box by databases.
Here are some consequences of this convergence trend:
Customers will increasingly question whether they need specialized vector databases at all, or just use one provided by their existing databases.
Every database will try to insert themselves into production RAG workloads.
The roadmaps of database and AI companies will increasingly collide.
Startup vector DB companies will experience rapid slow down in growth: up until H1 2023, they enjoyed the relative lack of familiarity and hesitance of enterprise buyers about committing to Gen AI workloads. But now in Q4 2023, enterprises are much savvier about what vector search is, and prefer solutions that integrate seamlessly with their current data infrastructure. And what is more seamless than your current database adding vector search capability?
Excellent article, universality of vector search is only starting to become understood. But that is still only part of the story, as vectors also offer other unsurpassed advantages, such as capability to handle (tens of) billions of vectors. Such scale has been the rarefied province of only a few, namely Google/Bing, but that is changing too. One can check our system https://yottaanswers.com to see for themselves how it works.
Google is not what they used to be and systems that go for such world-class scale will become more prevalent, ChatGPT is really all about that even though they themselves do not quite get it yet, as they do not focus on scale of their innate knowledge as opposed to style prompting.
Google could/should have switched to a RAG-like system years ago but they have been hamstrung by enormous legacy costs and issues. Emergence of vectors and Generative AI is the price to be paid for not doing the switch and taking opportunity to be leaders in vector space.
Just revisited this article after 2 years.