Choosing a Vector Store for Amazon Bedrock Knowledge Bases: What Works in Real‑World Tests

According to Artificial Intelligence, Amazon Bedrock Knowledge Bases lets you run Retrieval‑Augmented Generation (RAG) with either a fully managed vector store or a customer‑managed one. The post compares the three supported back‑ends—Amazon OpenSearch Service, Amazon Aurora PostgreSQL + pgvector, and Amazon S3 Vectors—using a product‑catalog search scenario.

How a vector store powers a RAG pipeline

A RAG system first turns every document into a high‑dimensional vector that captures its meaning. When a user asks a question, the same embedding model creates a query vector, and the vector store returns the most similar document chunks. Those chunks become context for the large language model (LLM) that finally writes the answer. The store’s job is therefore to keep millions of vectors searchable with sub‑second latency while staying affordable.

The three back‑ends Amazon Bedrock supports

*All latency numbers are taken from the AWS blog’s benchmark description; exact values vary with workload.

Benchmark: product‑catalog search

The blog used the ESCI “Shopping Queries” data set (1.22 M product descriptions, 5 k sampled queries). Each backend was tested with several embedding dimensions (1024, 512, 256) and two data types (float, binary). The baseline was a 1024‑float, in‑memory OpenSearch index using FAISS + HNSW (ef_construction = 128, m = 24).

Key observations from the tables in the source:

• Latency: OpenSearch Serverless kept the 50th‑percentile latency in the low‑millisecond range even at 10 concurrent queries. Binary embeddings cut memory use but added a small latency penalty. On‑disk mode (32× compression) reduced memory dramatically but pushed latency into the high‑millisecond range.
• Quality: NDCG@10 (a relevance metric) fell only a few points when moving from 1024‑float to 256‑float, suggesting that smaller embeddings can be acceptable for many e‑commerce queries.
• Size: Binary vectors and on‑disk compression slashed the in‑memory footprint by up to 90 %, matching the cost claim for S3 Vectors.
• Aurora + pgvector: Not directly benchmarked in the blog’s product‑catalog test, but the write‑up notes that pgvector supports the same dimensions and distance metrics, with latency that scales with the chosen RDS instance class.
• S3 Vectors: Promised sub‑second query time and up to 90 % lower storage cost; the blog does not give a latency table for the catalog use case, so the exact numbers remain unclear.

The hidden trade‑off: speed versus simplicity versus cost

What actually changes is not just the raw numbers but the operational model you adopt.

• Speed vs. cost – OpenSearch gives the fastest response because the vectors stay in memory, but you pay for that memory. Switching to binary embeddings or on‑disk mode cuts the bill but adds latency that may be noticeable to shoppers during a flash sale.
• Managed vs. self‑service – Aurora and S3 Vectors are “customer‑managed,” meaning you control versioning, backups, and scaling. That gives flexibility but also adds operational overhead (patching, monitoring, capacity planning). OpenSearch Serverless abstracts most of that, but you lose fine‑grained control over instance sizing.
• Feature set – Hybrid search (semantic + keyword) and faceted filtering are native to OpenSearch. Aurora can combine vector similarity with SQL predicates, but you must join tables manually. S3 Vectors currently lack built‑in filtering; you would need to layer a separate metadata store.
• Future‑proofing – The blog mentions that OpenSearch Serverless NextGen collections (expected May 2026) will add GPU‑accelerated indexing and scale‑to‑zero, but they are not yet compatible with Bedrock’s Retrieve API. If your timeline extends beyond that, you might be stuck on Classic collections for now.

Who should care

• E‑commerce teams that need sub‑10 ms response times for a large, constantly changing catalog should start with OpenSearch Serverless, experiment with smaller embedding sizes, and only consider on‑disk mode if memory cost spikes.
• Data‑engineering groups that already run Aurora for transactional workloads may find pgvector attractive for a “one‑database‑to‑rule‑them‑all” approach, accepting a modest latency increase.
• Cost‑sensitive startups with static catalogs can off‑load vectors to S3 Vectors, enjoy the storage discount, and accept the higher latency for non‑time‑critical queries (e.g., internal knowledge bases).

Quick experiment you can run today

1. Pick a small document set (e.g., 10 k product titles) and export it as a CSV.
2. Generate embeddings with Amazon Titan Text Embedding v2 at three sizes: 1024, 512, 256 (the model lets you choose the dimension).
3. Create an OpenSearch Serverless Classic collection and load the vectors using the bulk API.
4. Run a handful of queries through the Bedrock Retrieve API, measuring the 50th‑percentile latency with time or a simple script.
5. Repeat steps 2‑4 with binary embeddings (set data_type to binary). Compare latency and index size (OpenSearch reports the in‑memory footprint).

If latency stays under 20 ms with 512‑float vectors, you’ve found a sweet spot that saves memory without hurting user experience. You can then decide whether to move the same index to a Managed Cluster for finer‑tuned performance or to S3 Vectors for a cheaper long‑term store.

---

Sources

• Selecting a vector store for Amazon Bedrock Knowledge Bases