Databricks GenAI Engineer: Mosaic AI Vector Search Indexes, Endpoints, Querying and Sizing

Mosaic AI Vector Search is Databricks' serverless vector database: it stores embeddings in an approximate-nearest-neighbor index and returns the rows most similar to a query, which is the retrieval step of a RAG application or agent. Recent Databricks documentation also calls it Databricks AI Search (formerly Vector Search); the exam guide uses the Mosaic AI Vector Search name, and the core method names (create_delta_sync_index, similarity_search, sync, upsert) are the same.

It has two resources, and the exam expects you to keep them apart:

The relationship runs one way: an endpoint hosts many indexes, and each index lives on exactly one endpoint. They are created, governed and deleted separately. The 50-index limit applies whatever the endpoint type, index type or throughput setting, so when an endpoint is full the answer is another endpoint, not a bigger one. (Limits such as this one are Databricks product limits that can change; check the current documentation for exact figures.)

Vector Search has two index types, distinguished by where the data comes from: a Delta Sync index follows a Delta table, and a Direct Vector Access index is written directly by your code. Delta Sync then splits again by who computes the embeddings.

Choose by the constraints in the scenario:

• No embedding pipeline and no wish to write one: Delta Sync with Databricks-computed embeddings.
• Vectors already in the table (for example from a fine-tuned model) and no budget to embed twice: Delta Sync with self-managed embeddings.
• Records arrive from an application or stream and there is no Delta table, and the team accepts writing upsert code: Direct Vector Access.

Preparing the source table itself (chunking, a unique primary key, enabling change data feed) is covered in the data preparation lessons; here the point is that Delta Sync needs that table and Direct Vector Access does not have one.

You create a Delta Sync index with create_delta_sync_index (or the UI or REST API), naming the endpoint, the source table, the index name, the sync mode, the primary key, and one of two embedding argument pairs:

The pairs do not mix: a text column is never an embedding_vector_column, and a dimension without a vector column describes nothing.

client.create_delta_sync_index(
    endpoint_name="manuals-endpoint",
    source_table_name="main.docs.manual_sections",
    index_name="main.docs.manual_sections_idx",
    pipeline_type="TRIGGERED",
    primary_key="section_id",
    embedding_source_column="section_text",
    embedding_model_endpoint_name="manuals-embedder",
)

Other arguments and options to know:

• pipeline_type: "TRIGGERED" or "CONTINUOUS" (see the next section).
• columns_to_sync: limits which source columns are copied into the index (the primary key and embedding column are always included). It is fixed at creation and is not supported on storage-optimized endpoints, which sync every column.
• Save computed embeddings: for an index with Databricks-computed embeddings, this option writes the vectors Databricks computed to a Unity Catalog table named after the index with _writeback_table appended. It is the way to reuse those vectors (for clustering or analysis) without paying to embed again; the source table never receives them. The table is managed with the index, is deleted with it, and should not be modified.
• Storage-optimized endpoints also require an embedding dimension divisible by 16.

A Direct Vector Access index is created instead with create_direct_access_index, giving a primary key, the vector column, its dimension and a schema for the stored columns.

How an index picks up new data depends on its type and, for Delta Sync, on its sync mode.

Three behaviours that scenarios turn on:

• Triggered indexes do not watch the table. New rows written to the source table are invisible until the next sync, because maintenance or metadata operations on the table itself do not touch the index; only a sync does.
• Direct Vector Access indexes have no link to any table. Deleting a record in the application's database leaves it in the index; you must call the index's delete API with the primary keys. sync() and change data feed only apply to Delta Sync.
• Storage-optimized endpoints support only triggered sync. A continuous index must live on a standard endpoint.

Changing the schema or the embedded column

An existing Delta Sync index does not accept source schema changes or a switch to a different embedding column. The documented approach is to build a new index, switch the application to it once it has synced, then delete the old one. Deleting first, even to reuse the name, leaves users with failed searches until the rebuild finishes. Changing the embedding model itself is covered in the embedding-model lesson; it also requires a re-embedded index.

You query an index with similarity_search on the index object, passing the query, the columns to return, how many results you want, and optionally filters and a query type.

results = index.similarity_search(
    query_text="How do I replace the filter cartridge?",
    columns=["section_id", "section_text", "manual_url"],
    num_results=8,
    filters={"product_line": "pumps", "revision >=": 3},
)

• query_text versus query_vector. Only an index with Databricks-computed embeddings knows which model to apply to text, so it accepts query_text. For self-managed and Direct Vector Access indexes, the application embeds the question itself, with the same model that produced the stored vectors, and passes query_vector; a vector from another model lives in a different embedding space and returns poor matches.
• columns decides what each result row carries (plus a score). Listing a synced column, such as the chunk text or a source URL, returns it without changing the index. num_results, filters and query_type change which rows come back and how they are ranked, not which columns they carry.
• num_results: request what you will use. Asking for far more rows than the chain keeps makes each query slower without changing the top results.
• query_type: "ANN" (default, vector similarity) or "HYBRID", which combines keyword (BM25-style) matching of the text with vector similarity, useful for exact identifiers such as part numbers. On a self-managed index, hybrid search needs both query_text (for the keyword half) and query_vector (for the vector half). Hybrid queries use roughly twice the resources of ANN queries and return at most 200 results. The Python SDK and REST API also offer "FULL_TEXT" (Beta), a keyword-only search that uses no embeddings at all. (When to use hybrid search and reranking for quality is covered in the retrieval lessons.)
• Result limits: the SDK paginates ANN results up to 10,000 rows (a current product limit, like the 200-result hybrid cap), so large result sets (for example several thousand candidates for an evaluation set) come from similarity_search in a notebook rather than from SQL.

Filter syntax depends on the endpoint type

Moving an index to a storage-optimized endpoint therefore means rewriting dictionary filters as SQL-like strings: each endpoint type accepts only its own filter format.

Why a filtered query can come back empty on storage-optimized endpoints

Storage-optimized endpoints overfetch nearest neighbors and then apply the filter. If the matching rows are a tiny share of a huge index and none of them rank among the fetched candidates, the query returns no rows even though matching data exists. Large segments behave normally; very selective filters are the ones that suffer. Databricks documents this overfetch behaviour for storage-optimized endpoints.

vector_search() is a table-valued SQL function (Public Preview) that queries a Vector Search index from a serverless or pro SQL warehouse, so analysts and dashboards can retrieve similar rows without Python.

SELECT *
FROM vector_search(
  index => 'main.docs.manual_sections_idx',
  query_text => 'warranty claim steps',
  num_results => 10
);

• Arguments are passed by name: index, then query_text (for an index with Databricks-computed embeddings) or query_vector (an array of numbers), plus num_results and optionally query_type.
• Supported query types are ANN and HYBRID.
• num_results above 100 is not supported, and there is no paging, so repeating the call does not reach further down the ranking. Use the SDK for larger result sets.
• It does not support Direct Vector Access indexes.

Do not confuse it with ai_query(), which calls a model serving endpoint (an LLM or embedding model) rather than searching an index.

Vector Search finds neighbors with HNSW (an approximate nearest-neighbor graph) and measures closeness with L2 (Euclidean) distance. The score it returns is a similarity derived from that distance:

score = 1 / (1 + d²), where d is the L2 distance between the query vector and the stored vector.

Cosine similarity through normalization. Many embedding models are meant to be compared by cosine similarity. If both stored and query vectors are normalized to unit length, ordering by L2 distance gives the same order as cosine similarity, so normalize embeddings before writing them (and normalize query vectors the same way). Endpoint type, query type and numeric precision do not change the metric.

The score is not the cosine. For unit vectors, d² = 2 − 2 × cos. To turn a cosine cut-off into a score cut-off, substitute: a cosine of 0.7 gives d² = 0.6 and a score of 1 / 1.6 = 0.625. Filtering returned scores against the raw cosine value would discard relevant results.

Choose the endpoint type from the number of vectors, the latency and throughput target, and cost. The figures below are Databricks' published guidance at the time of writing; they change, so treat them as orders of magnitude.

Reading a scenario:

• Customer-facing, tight latency, high QPS, index within capacity: standard. If load tests return HTTP 429 errors at peak, the fix is more provisioned serving capacity, which on a standard endpoint means setting target_qps (at extra cost); client-side changes only reshape the same load. target_qps raises throughput, not vector capacity.
• Hundreds of millions to billions of vectors, latency of around half a second acceptable, low concurrency, tight budget: storage-optimized with triggered sync.
• Fits on either: let cost and latency decide. If the latency budget allows a few hundred milliseconds and traffic is light, storage-optimized is cheaper.
• Raising the embedding dimension shrinks how many vectors a standard endpoint holds. A corpus that fits at 768 dimensions can overflow at 1536. Keep the latency requirement fixed when you solve that: an option that makes room by giving up the latency or throughput the scenario demands does not meet it.

Most Vector Search cost comes from endpoints that exist and sync pipelines that run, and most avoidable latency comes from authentication, oversized requests and the embedding endpoint.

Cost levers

• Idle endpoints still bill. An endpoint hosting indexes incurs serving cost with no queries; Vector Search endpoints do not scale to zero. Consolidate small, lightly used indexes onto one endpoint (up to 50 per endpoint) and delete the empty endpoints.
• Prefer triggered sync when the table changes on a schedule; continuous sync runs a streaming cluster all the time.
• Use storage-optimized endpoints for large, latency-tolerant indexes, since they cost less per vector.
• target_qps and num_results do not reduce base serving cost; target_qps adds capacity and cost.

Latency levers

• Authenticate with OAuth tokens for a service principal rather than personal access tokens. PATs add hundreds of milliseconds per query; OAuth uses network-optimized routing.
• Request only the results you use with num_results (Databricks suggests 10 to 100 for most RAG use); more results means slower queries.
• Use ANN unless keyword matching is needed; hybrid search costs extra resources.
• Do not let the embedding endpoint scale to zero. With Databricks-computed embeddings, every query_text is first embedded by the model serving endpoint; if that endpoint has scaled to zero, the first queries after a quiet period can be slow or time out while the Vector Search endpoint itself looks healthy.
• Separate the query embedding endpoint. A managed-embedding index can use a different serving endpoint for query embeddings (model_endpoint_name_for_query in the SDK) while the configured endpoint embeds ingestion. Heavy backfills then no longer starve user queries. The query endpoint must serve the same model, or query vectors land in a different space and retrieval quality changes.

Because an index is a Unity Catalog object, querying it as a non-owner requires USE CATALOG on the catalog, USE SCHEMA on the schema and SELECT on the index. This applies to a service principal that an agent or serving endpoint runs as.

• SELECT on the source table does not grant access to the index; the query reads the index, not the table.
• SELECT granted at the schema level works, because it is inherited by every object in the schema, but it grants far more than one index, so it fails a least-privilege requirement.
• Endpoint permissions (CAN CREATE, CAN USE, CAN MANAGE) are workspace ACLs on the compute. CAN USE lets a user create indexes on the endpoint; it is not what authorizes reading an index's data.

Passing the end user's identity through to retrieval (on-behalf-of-user authorization) and serving-endpoint access control are covered in their own lessons.