AIP-C01 Vector Store Solutions: Architecture, Metadata, Scale and Sync

A vector store solution is the storage, indexing, metadata and refresh design that lets a generative AI application find the passages most similar in meaning to a question and hand them to a foundation model (FM) as context. The embeddings make semantic search possible; everything around them decides whether retrieval is precise, fast at scale, connected to the right sources and current.

Task 1.4 has five parts, and a production design touches all of them:

Amazon Bedrock now offers two kinds of knowledge base. A Bedrock Managed Knowledge Base runs the datastore for you, with built-in embedding, reranking, native connectors, ACL-aware retrieval and built-in sync schedules. A customer-managed knowledge base writes into a vector store you choose and operate (OpenSearch Serverless or a managed OpenSearch cluster, Aurora PostgreSQL, S3 Vectors and others). Many trade-offs in this lesson come down to that choice.

Chunking strategies, embedding-model choice, hybrid search, reranking and query rewriting belong to Task 1.5 and are taught in the lesson on retrieval mechanisms; this lesson covers the store underneath them.

Choose the store by what the vectors must sit next to and how they are queried: relational joins point to pgvector, a DynamoDB system of record points to a vector store fed from DynamoDB, large rarely queried sets point to S3 Vectors, and high-throughput semantic search with rich query features points to OpenSearch.

Keep volatile facts out of the vector index

DynamoDB has no vector index of its own, so it pairs with a vector store rather than replacing one. The split also solves a freshness problem: put stable text (descriptions, policies) and its ID in the vector index, and keep fast-changing values such as price or stock in DynamoDB. After retrieval, read the current values by ID (for example with BatchGetItem). Any design that copies volatile values into the index keeps writing to it and can still serve a value that changed since the last write; reading them at answer time from the system of record cannot go stale.

Tier vectors by query rate

S3 Vectors stores embeddings in vector buckets and vector indexes with pay-per-use pricing and no clusters. It can export a whole vector index to OpenSearch Serverless as a point-in-time copy, so a large archive stays in S3 Vectors while one hot, fixed index is served at high request rates from OpenSearch. Note that OpenSearch Serverless bills for capacity units that stay allocated, so it costs more than S3 Vectors for a handful of queries a day.

You let OpenSearch create embeddings itself: register a Bedrock model through an ML Commons connector, embed documents at index time with an ingest pipeline that has a text_embedding processor, and embed questions at search time with a neural query. Indexing and query clients then send plain text, and the existing domain gains semantic retrieval without a new store.

1. Create a connector to the Bedrock model (for example an Amazon Titan embeddings model) and register a remote model from it.
2. Create an ingest pipeline whose text_embedding processor maps a text field to a knn_vector field, and set it as the index's default pipeline.
3. Query with neural, passing the query text and the model ID; OpenSearch embeds it and runs the k-NN search.

The value of the pattern is that embedding happens inside OpenSearch on both paths, indexing and query, with the same model. A design that embeds on only one path leaves the other to client code, and the two can drift onto different models.

Connector permissions have two halves

• The connector role (named in the connector's credential): its trust policy must allow the OpenSearch service principal to assume it, and its permissions must allow bedrock:InvokeModel on the model. Connectors do not use the domain's service-linked role.
• The identity creating the connector: it needs iam:PassRole on the connector role and, when fine-grained access control is on, a mapping to the ml_full_access role in OpenSearch.

An authorization error from a connector is fixed in one of these two places: the connector role's trust and permissions, or the creator's right to pass that role.

A metadata framework attaches typed attributes (date, author, department, product, domain tags) to every document so retrieval can be narrowed with filters or steered towards relevant chunks. For an Amazon Bedrock knowledge base with an S3 data source, attributes come from a metadata sidecar file.

Metadata sidecar files

For reports/q3.pdf, the file must be named reports/q3.pdf.metadata.json: the document's full file name, extension included, plus .metadata.json, stored in the same folder. A file with the wrong name or in another prefix is silently ignored, so filters return nothing even though unfiltered queries work. Each document has one metadata file (up to 10 KB). S3 user-defined object metadata (x-amz-meta-*) and object tags are not ingested as filterable attributes; values such as an object's timestamp or author are useful sources, but they must be written into the sidecar to become attributes. A transformation Lambda can add chunk metadata too, but that is ingestion code to maintain.

{
  "metadataAttributes": {
    "department": { "value": { "type": "STRING", "stringValue": "oncology" },
                    "includeForEmbedding": false },
    "published": { "value": { "type": "NUMBER", "numberValue": 1757808000 },
                   "includeForEmbedding": false },
    "domains": { "value": { "type": "STRING_LIST",
                            "stringListValue": ["payments", "fraud"] },
                 "includeForEmbedding": false }
  }
}

Choose types so the filters you need work

Attributes are STRING, NUMBER, BOOLEAN or STRING_LIST. Range operators (greaterThan, greaterThanOrEquals, lessThan, lessThanOrEquals) accept numbers only, so store dates as numeric epoch values when you need date ranges. Documents with several domain tags should use a STRING_LIST and listContains.

Filter versus includeForEmbedding

A filter excludes every chunk that fails it. When an attribute should instead make matching chunks rank higher while others stay retrievable, use the full metadata format and set includeForEmbedding to true: the key and value are joined to the chunk text before embedding, but the chunk text returned to the model stays the original wording. The simplified format ("region": "emea") stores the attribute for filtering only.

Implicit metadata filtering

Implicit filtering builds a filter from the user's question automatically. Add implicitFilterConfiguration to the retrieval's vectorSearchConfiguration with a model ARN and a schema that describes each attribute (key, type, description); the model reads the question and the schema and applies a matching filter, with no query parser to write. It differs from query decomposition (which splits questions into sub-queries) and reranking (which reorders results); neither of those filters anything. Because the filter depends on the question's wording, implicit filtering is not a way to enforce a fixed scope.

Give each store the filterable field it needs: a jsonb custom metadata column with a GIN index on Aurora, keyword fields on a self-managed OpenSearch index, the faiss engine on OpenSearch Serverless, and non-filterable Bedrock keys on S3 Vectors. Most filter failures (filters that error, do nothing, or return too few results) come from one of these store-level settings rather than from the filter itself.

Custom attributes belong in the fields above. Each store also has a Bedrock-managed metadata field for the service's own bookkeeping (source attribution and similar), which is not a filterable home for your attributes.

Aurora: selective filters and HNSW

With a pgvector HNSW index, the scan returns a limited candidate list and the filter is applied afterwards. If only a tiny share of chunks match, most candidates are discarded and the request returns fewer results than asked for. Filtering after the scan cannot recover candidates the scan never returned, so the fix is to let the scan continue. From pgvector 0.8, iterative index scans (for example hnsw.iterative_scan = relaxed_order) keep reading the index until enough filtered rows are found; a database-level setting applies to new sessions.

S3 Vectors: small custom metadata

A knowledge base stores the chunk text and its own metadata as vector metadata, which is why those two keys must be non-filterable. Custom metadata per vector is small (about 1 KB and 35 keys with a knowledge base), so long text such as an abstract does not belong in metadata files; store it elsewhere and keep attributes short.

Fast vector search at scale comes from right-sized primary shards (work per query), enough replicas on enough nodes (queries in parallel), and k-NN graphs that fit in native memory, compressed if necessary.

Shards and replicas

• AWS recommends roughly 10–30 GiB per shard for latency-sensitive search. Oversized shards make each query slow even when CPU and memory have headroom.
• number_of_shards is set when an index is created (the _split and _shrink APIs exist but require blocking writes to the index). To change it on a live index, create a new index with the right number of primaries, reindex into it, and switch an index alias so the application's query code is unchanged.
• Replicas serve reads, so adding data nodes and replicas raises how many queries the domain handles at once. Replicas do not shrink a shard, so they do not fix single-query latency, and more replicas on the same nodes add little capacity.
• Longer refresh_interval and force merges mainly help indexing; time-based indexes suit logs, not semantic corpora.

k-NN graph memory

HNSW graphs (Faiss or nmslib engines) live in native memory outside the JVM heap. OpenSearch Service gives the heap half of a node's RAM (up to 32 GiB), and by default the k-NN circuit breaker allows graphs to use 50% of the remaining memory. When KNNGraphMemoryUsage hits the limit, graphs are evicted and latency spikes.

To size a domain, estimate graph memory per vector with the HNSW formula 1.1 × (4 × dimensions + 8 × m) bytes, multiply by the vector count, then multiply by the number of copies (primary plus every replica). For example, 20 million 768-dimension vectors with m = 16 need about 3.5 KB each, roughly 66 GiB per copy and 131 GiB with one replica. A 32 GiB node offers (32 − 16) × 50% = 8 GiB of graph memory, so 17 nodes; a 64 GiB node offers 16 GiB, so 9.

Replicas add read throughput, never per-query capacity or indexing speed: adding a replica after the load copies finished segments instead of re-indexing every document.

Use one index per specialised domain when domains differ in embedding model, vector dimension, mapping, scale or update schedule, and route each question to the right index; use a two-tier hierarchical index when a huge corpus makes a flat chunk search slow and imprecise.

Multi-index versus one shared index

Routing is what keeps each search small: a question should touch only the vectors that can answer it, and every extra cluster, pipeline or knowledge base is one more thing to run and sync.

Hierarchical (two-tier) indexing

When questions concern a few documents in a corpus of millions, first search a document-level index of summary embeddings to pick the top documents, then run the chunk search filtered to those document IDs (for example an in filter on a report_id attribute). The first tier is small, so it is fast; the second searches only relevant chunks.

The document scope must be applied inside the second search: post-filtering cannot recover candidates the nearest-neighbour search never returned, so it can return too few results and does no less work. Bedrock's hierarchical chunking (Task 1.5) is a different idea: it returns a larger parent chunk for a matched child chunk, not a whole document, and a data source's chunking strategy cannot be changed after the data source is created.

Connect sources through a knowledge base's data source connectors where one exists, and through a Custom data source with the direct ingestion APIs where it does not. Writing vectors straight into the index behind a knowledge base is never the integration path: it skips Bedrock's parsing, chunking and the fields it manages for each chunk, so the knowledge base cannot track, update or delete those entries.

Managed knowledge bases and native connectors

A Bedrock Managed Knowledge Base offers native connectors for Amazon S3, SharePoint, Confluence, Web Crawler, Google Drive, OneDrive and Custom sources, plus ACL-aware connectors such as ServiceNow, Box, Zendesk and Salesforce. Bedrock crawls, parses, embeds, stores and retrieves, so there is no vector store to operate. Since 30 September 2026, new Confluence, SharePoint, Salesforce and Web Crawler connectors cannot be created on customer-managed knowledge bases (existing ones keep working); for a new project on those sources, a managed knowledge base is the low-overhead path.

Custom data sources for everything else

A homegrown wiki, a content management system or a DynamoDB table usually has no connector but can emit change events. Attach a Custom data source and push changes:

• IngestKnowledgeBaseDocuments adds or replaces a document (keyed by your own document ID) within moments.
• DeleteKnowledgeBaseDocuments removes it; without this path, deleted pages keep appearing in answers.
• A webhook can reach a Lambda function through API Gateway. For DynamoDB, enable Streams with the NEW_IMAGE view (the record carries the new item, and every record carries the key) and handle INSERT/MODIFY with ingest and REMOVE with delete. KEYS_ONLY would force a read back for every change.
• A Custom data source is not synced with ingestion jobs; StartIngestionJob does not apply its changes.

Starting a sync job for every event is the wrong tool for near-real-time changes: jobs take minutes, and a second job cannot start while one is running (one per knowledge base), so further calls are rejected rather than queued.

Enforce document access inside retrieval itself, either with ACL-aware retrieval on a managed knowledge base or with a server-set metadata filter whose value comes from an identity the user cannot change. IAM, guardrails and bucket policies cannot do document-level access control on retrieval.

ACL-aware retrieval

On a Bedrock Managed Knowledge Base, ACL awareness on a supported connector ingests the source's per-user and per-group permissions. Your application passes the authenticated user's email as user context on each retrieval, and results are filtered to what that user may open. For SharePoint, access is also checked in real time, so a permission removed since the last sync is honoured. Copying allowed groups into your own metadata files works only until permissions change between syncs.

Multi-tenant filters

For many tenants on one customer-managed knowledge base, tag each document with a tenant_id attribute and have the backend add an equals filter from a trusted identity value, for example a signed token claim. The value must also be one the user cannot edit: in Amazon Cognito, make the tenant attribute read-only for the app client, because users can update any custom attribute their app client may write. One knowledge base per tenant isolates too, but hundreds of knowledge bases, data sources and syncs add heavy operations and run into quotas.

Scoping to one data source

Every chunk carries reserved metadata you can filter on without custom attributes. Managed knowledge bases use underscore-prefixed names such as _data_source_id and _source_uri; customer-managed knowledge bases use the x-amz-bedrock-kb-* prefix (for example x-amz-bedrock-kb-data-source-id). Remember that managed knowledge bases do not support startsWith or stringContains, so scope with equals.

Keep vectors current with incremental syncs triggered by change events or schedules, push urgent single-document changes through direct ingestion, and control deletions explicitly with the data deletion policy and deletion protection.

Syncs are incremental

StartIngestionJob processes only documents added, modified or deleted since the last sync, so the cost of a daily sync scales with the changes. When only metadata files change (non-CSV content and no transformation Lambda), the sync merges the new metadata into existing vectors without calling the embedding model.

Event-driven and scheduled refresh

• Real-time change detection: turn on Amazon EventBridge notifications for the S3 bucket (object created and deleted events, no polling). An EventBridge rule starts a Step Functions workflow that waits briefly to collect a burst, calls StartIngestionJob once and tracks it with GetIngestionJob. Starting one sync per object does not work: only one ingestion job can run per knowledge base (and per data source) at a time, so calls made while a job is running are rejected and the burst is not reliably picked up.
• Scheduled refresh: managed knowledge base data sources have a built-in sync schedule (daily, weekly on a chosen day, or monthly on a day number). On customer-managed knowledge bases, use EventBridge Scheduler to call StartIngestionJob. Pick the cadence that matches when content is approved, not just how often it changes.

Direct ingestion on an S3 data source

Calling IngestKnowledgeBaseDocuments on an S3 data source updates the knowledge base quickly but does not change the bucket. The next sync treats S3 as the source of truth and re-indexes the old file, so every direct fix must also be written to S3. AWS advises against running direct ingestion and a sync at the same time.

Deleting safely

The order matters: if vectors must be erased, switch the policy to DELETE before deleting the data source. After a RETAIN deletion, the vectors belong to no data source, so no later sync or source-file deletion will remove them.