AI-200 Azure Cosmos DB for NoSQL: SDK Queries, RUs, Vector Search and Change Feed

Connect to Azure Cosmos DB for NoSQL by creating one long-lived CosmosClient with the account endpoint and a Microsoft Entra token credential such as DefaultAzureCredential, then giving the app's identity a Cosmos DB data-plane role. No account key or connection string is needed anywhere.

from azure.identity import DefaultAzureCredential
from azure.cosmos import CosmosClient

client = CosmosClient(url=ENDPOINT, credential=DefaultAzureCredential())
container = client.get_database_client("rag").get_container_client("chunks")

DefaultAzureCredential uses your developer sign-in locally and the managed identity (system-assigned or user-assigned) when the code runs in Azure Container Apps, App Service, Functions or AKS. Authentication then only proves who the caller is. Whether it may read or write items is decided by Cosmos DB's own role-based access control for the data plane, which is separate from Azure RBAC.

Data-plane roles are assigned with az cosmosdb sql role assignment create (or the equivalent Bicep/ARM resource), scoped to the account, a database or a container. A 403 Forbidden on an item operation made with an Entra token means authentication worked but the identity holds no data-plane role for that scope, which is common when an identity was given only an Azure RBAC role. Because the same credential behaves differently for each identity it resolves to, check which identity the code runs as and what Cosmos DB SQL role assignments that identity has. Keyless means no account key is used at all, wherever it is stored; to enforce it, disable local (key) authentication on the account so key-based requests are rejected.

When you know an item's id and its partition key value, use a point read (read_item): it is the cheapest and fastest operation in Cosmos DB, about 1 RU for a 1 KB item, because it goes straight to the item without running the query engine.

conv = container.read_item(item=conversation_id, partition_key=user_id)

Use query_items when you need to filter, sort or return several items. Two habits keep queries safe and cheap:

• Parameterize. Put placeholders such as @sid in the SQL text and pass values in the parameters list of {"name", "value"} objects. Never splice request input into the query string; that is an injection risk.
• Scope to one logical partition. Pass partition_key with the value of the container's partition key path (for a container partitioned on /userId, the user's ID, not some other field). Without it you must set enable_cross_partition_query=True and the query fans out to every physical partition, costing more RU and latency.

items = container.query_items(
    query="SELECT * FROM c WHERE c.sessionId = @sid",
    parameters=[{"name": "@sid", "value": sid}],
    partition_key=user_id,
)

A query such as WHERE c.id = @id returns the same single item a point read would, but it always costs more because the query engine runs.

Measuring RU cost and index use from the SDK

Every response reports its cost in the x-ms-request-charge header. In the Python SDK, read it from container.client_connection.last_response_headers after the call (for a query, after you have iterated the results). It tells you how expensive an operation was, but not why.

To find out which indexes a query used and which would help, pass populate_index_metrics=True to query_items and read the x-ms-cosmos-index-utilization header. It lists the utilized single and composite indexes and potential ones that could lower the query's RU charge, each with an impact score. That is the input for editing the indexing policy. Microsoft recommends enabling index metrics only while troubleshooting, because they add overhead.

The indexing policy decides which properties are indexed, and every indexed path costs RU on every write. By default a container indexes everything (includedPaths /*, consistent mode), which makes any query efficient but makes writes of large documents expensive.

When query patterns are known, invert the default: include only the queried paths and exclude the root.

{
  "indexingMode": "consistent",
  "includedPaths": [
    { "path": "/userId/?" },
    { "path": "/sessionId/?" },
    { "path": "/createdAt/?" }
  ],
  "excludedPaths": [{ "path": "/*" }]
}

/path/? indexes one scalar value; /path/* covers everything beneath a path. Large text bodies, token-usage objects and other unqueried properties then stop costing index maintenance.

Exclude vector paths from range indexing. With /* included, an embedding array is range-indexed element by element, so a 1,536-dimension vector adds 1,536 index entries to every insert. Add the vector path (for example /embedding/*) to excludedPaths. The vector index is a separate structure, so vector queries keep using it.

Composite indexes are required for ORDER BY on two or more properties (and help filters combined with a sort). The rules:

• The paths must appear in the same sequence as the ORDER BY clause.
• The directions must match the query exactly, or be the exact reverse on every path (the engine can read the index backwards). A partial reversal does not work.

"compositeIndexes": [[
  { "path": "/sessionId", "order": "ascending" },
  { "path": "/createdAt", "order": "descending" }
]]

This index serves ORDER BY c.sessionId ASC, c.createdAt DESC (exact match) and ORDER BY c.sessionId DESC, c.createdAt ASC (every direction reversed). It does not serve ORDER BY c.sessionId ASC, c.createdAt ASC, where only one direction is reversed, or ORDER BY c.createdAt DESC, c.sessionId ASC, where the path order differs; each of those needs its own composite index.

Adding a composite index adds index work on writes, so use it to fix a sort or filter, not to reduce write RU.

Choose the weakest consistency level that still meets the requirement, because the two strongest levels cost about twice as much per read. Cosmos DB offers five levels, from strongest to weakest: Strong, Bounded Staleness, Session, Consistent Prefix and Eventual.

Write cost is the same at every level; consistency only changes what reads see and what they cost. A requirement that a user sees their own changes immediately, while others may lag briefly, is the definition of Session: it is the weakest level that guarantees read-your-writes, and the levels above it guarantee it too but at a higher read cost.

The account sets a default, and a client can relax it (never strengthen it). In Python, pass consistency_level="Session" (or a weaker level) when creating a CosmosClient. That lets a read-heavy, staleness-tolerant API cut its read RU on a Strong account without touching the account configuration or the workloads that need Strong.

Session tokens: read-your-writes across processes

Session consistency belongs to whoever holds the session token, not to the account. Each write returns a session token, and the SDK client caches it so that client's later reads wait for a replica that has its writes.

A different process with its own CosmosClient has no record of that token, so for that data it reads as if consistency were Eventual: it may briefly see an older version of an item, or not find a just-created item at all. This matters whenever one component writes and another must read the result straight away, such as separate services, background workers or scaled-out web instances.

To extend read-your-writes to the second process, hand it the session token returned by the write (alongside whatever request or message it receives) and supply that token on its read. Session guarantees then cover that process too, and the account default can stay at Session for every other workload.

To store and search embeddings in Cosmos DB for NoSQL, enable the vector search capability on the account, then give the container a vector embedding policy and, normally, a vector index. The two hold different settings:

"vectorEmbeddings": [{
  "path": "/embedding",
  "dataType": "float32",
  "dimensions": 1536,
  "distanceFunction": "cosine"
}]

Read scores in the right direction. VectorDistance returns a similarity or distance depending on the function: cosine runs from -1 (least similar) to +1 (most similar), dot product from -∞ to +∞ with higher meaning more similar, and euclidean from 0 (identical) upward, so a smaller euclidean score is a closer match. A fixed threshold written for cosine ("keep scores above 0.8") keeps the worst matches under euclidean.

Policies are not edited in place. You can add or remove vector paths on an existing container, but you cannot change an existing vector embedding entry or vector index entry directly; changing one means dropping and re-adding it, which interrupts vector search on that path. To move to a model with more dimensions without downtime, add a new path (for example /embedding2) with its own embedding entry and index, re-embed every item into it, then switch queries over. Embeddings from different models live in different vector spaces, so old vectors cannot be compared with new query vectors, and items never re-embedded would silently drop out of results.

Pick the vector index type from three facts: the embedding's dimensions, how many vectors a single search actually covers, and whether results must be exact or stable.

Dimensions eliminate first: modern embedding models (1,536 or 3,072 dimensions) rule out flat. Then look at vectors in scope, not container size. A multitenant container may hold millions of vectors, but if every query supplies the tenant's partition key and no tenant has more than a few thousand, each search is small and quantizedFlat gives near-exact, stable results. diskANN pays off when single searches really span very large sets. quantizedFlat and diskANN also need at least 1,000 vectors indexed before the index is used; with fewer, a search falls back to a full scan. Running with no vector index works, but every search compares full vectors, so it costs more RU and latency than an indexed search.

A semantic retrieval query uses VectorDistance in ORDER BY with TOP N: the results come back most similar first, and TOP N stops the search returning far more candidates than you need.

SELECT TOP 3 c.text, VectorDistance(c.embedding, @qv) AS score
FROM c
WHERE c.docType = @type
ORDER BY VectorDistance(c.embedding, @qv)

• Always use TOP N. Without it the vector search tries to return many more results, raising RU and latency.
• Filter in the query, not afterwards. A WHERE clause restricts the candidates before TOP N is applied, so you get N matching items if they exist. Dropping rows in application code after TOP 3 can leave fewer than three, or none.
• Scope by partition key. Pass partition_key (for example the tenant ID) so the search runs in one logical partition: lower RU, and other tenants' data never leaves the database.
• A threshold is not a ranking. WHERE VectorDistance(...) > 0.8 filters but does not order, so the rows returned need not be the most similar.

Use the same distance function and embedding model for the query vector that produced the stored vectors.

Full-text search adds keyword relevance (BM25) to Cosmos DB for NoSQL, and hybrid search fuses it with vector similarity into one ranked list. Embeddings are weak at exact identifiers such as part numbers or error codes, which keyword scoring handles well.

Setup: enable full-text search on the account, add a full-text policy to the container (default language and the text paths) and a full-text index on those paths in the indexing policy. Then choose the function by what you need:

SELECT TOP 10 * FROM c
ORDER BY RANK RRF(
  VectorDistance(c.vector, @qv),
  FullTextScore(c.text, @term1, @term2)
)

Reciprocal Rank Fusion combines the two rankings (weights can be supplied). Only ranking functions inside ORDER BY RANK blend results. Filter functions keep or drop items without ordering them, and the plain CONTAINS function is a substring match that neither uses the full-text index nor scores relevance.

The change feed is a persistent, ordered record of inserts and updates in a container, and the change feed processor is the library (in the .NET and Java SDKs) that reads it reliably across several workers. It has four parts:

The change feed estimator is a separate tool that reports how far behind the processor is; it stores no progress.

Scaling rule. Instances form one deployment unit and share leases when they have the same processor name (passed to GetChangeFeedProcessorBuilder), the same lease container configuration (WithLeaseContainer) and different instance names (WithInstanceName). The processor then spreads leases across instances; you cannot have more active instances than leases. Per-instance tuning such as WithPollInterval and WithMaxItems can differ without affecting lease sharing.

Change feed delivery semantics: at-least-once, ordering and deletes

Design every change feed consumer for at-least-once delivery: the processor checkpoints a lease only after the delegate finishes a batch, so if an instance crashes or the delegate throws, the batch is delivered again from the last checkpoint.

• Make side effects idempotent. Record each processed item's id and _etag (or use them as a de-duplication key) so a redelivered batch does not send duplicate notifications or writes. Let failures surface: the checkpoint advances whenever the delegate returns normally, so a handler that hides its own errors marks unprocessed changes as done. Whichever instance takes over a lease resumes from its last checkpoint.
• Ordering is per partition key value. Changes for one logical partition key arrive in the order they were made; across different partition key values there is no global order.
• Latest version only. In the default latest-version mode, an item updated several times between reads may appear only as its current state, so consumers should rebuild from the document they receive rather than expect every intermediate version.
• No deletes. Latest-version mode never emits deletes. The documented pattern is a soft delete: update the item with a marker such as isDeleted: true (that update flows through the feed so consumers can remove it downstream), and set a ttl on it with time to live enabled on the container so Cosmos DB removes it later.

All versions and deletes mode does emit every intermediate version and delete events, but it requires the account to use continuous backup and only returns changes within the backup retention window. It is not available on accounts that use periodic backup.

The change feed processor library is available for .NET and Java only, so a Python team chooses between two documented options.

The Functions trigger runs the change feed processor for you, so leases, checkpoints and balancing are handled.

With the pull model, the job must save its position. After processing, read the continuation token from container.client_connection.last_response_headers["etag"], persist it, and pass it as continuation= on the next run; it takes precedence over start_time.

token = load_token()  # None on the first run
feed = (container.query_items_change_feed(continuation=token)
        if token else
        container.query_items_change_feed(start_time="Beginning"))
for doc in feed:
    refresh_embedding(doc)
save_token(container.client_connection.last_response_headers["etag"])

start_time only sets where a reader with no saved position begins: "Beginning" reads the container's full history and "Now" starts from the current moment. partition_key or feed_range narrow which changes are returned, and mode selects latest version or all versions and deletes; the continuation token also carries the mode, so it overrides a mode passed alongside it. Only the persisted continuation records where the job stopped.