We almost bought a vector database. Then our search service became one.
Azure AI Search's integrated vectorization quietly removed a whole component from our RAG architecture. Notes on why less stack beat more stack.
🗑️ We almost bought a vector database. Then our search service became one.
For most of this year, every RAG architecture diagram — ours included — had the same box in the middle: a dedicated vector database. New vendor, new licence, new thing to secure, back up and keep in sync with the documents it was supposed to represent. We were in evaluation when Azure's search service made the box unnecessary: vector search went GA this month, the service is now Azure AI Search, and — the part that actually changed our design — the indexer now does the chunking and embedding itself, in preview, straight from the source.
What that meant in practice for our grounded-bot architecture:
1. One pipeline died, quietly. Our previous design had a custom process watching document libraries, chunking files, calling the embedding model, and pushing vectors to a store — code we wrote, scheduled and babysat. The indexer pattern replaces that: point it at the source, define the skillset, and chunking, embedding and index updates happen inside the service. The component we spent the most maintenance time on is now configuration.
2. Sync was the real problem, and it disappears with the pipeline. The dirty secret of a separate vector store is drift: documents change, vectors lag, and your bot confidently cites last quarter's version. When the index is fed by the same indexer watching the source, freshness is the platform's job, not a cron job's.
3. Hybrid beats pure-vector for enterprise questions. Real queries are mixed: half meaning ("what's our stance on data residency"), half exact ("clause 14.2", a product code, a client name). Pure semantic search fumbles the exact half; keyword search fumbles the meaning half. Having vectors, keywords and semantic ranking in one query, against one index, matched how people actually ask.
4. One fewer vendor is a governance win nobody puts on the slide. Vectors are derived from your documents — they inherit every sensitivity the documents have. Keeping them inside the same service boundary, same identity model, same compliance envelope as the rest of the stack meant one less data-flow to explain in review. That review, not the licence fee, was the cost we were happiest to delete.
The honest caveats: the integrated vectorization piece is preview, so we treat it accordingly — watched closely, not yet load-bearing for everything. And dedicated vector databases still earn their place at scales and query patterns we simply do not have.
But the broader lesson has held all year: in a stack moving this fast, the best component is often the one you no longer need. Every box you delete is code you do not maintain, a sync you do not debug, and a security review you do not sit through. Before adding anything new to an AI architecture right now, the question worth asking first: will the platform make this box free within two quarters — and what does it cost us to wait?