Databricks GenAI flashcards

Design Applications

• Pick the task from the output the business needs: a known list means classification, values in the text mean extraction, shorter text means summarization, another language means translation, closeness in meaning means embeddings.
• Decide single- or multi-label per output field from how many values can truly apply; open-ended names (including new ones) must be extracted, not classified.
• Embeddings scale with items, pairwise chat prompts with pairs; for thousands of labels, retrieve candidates with embeddings and let an LLM choose among them.
• One task per required output, ordered by dependency; classify first when the schema depends on the document type; auditable rules belong in SQL or Python, not a prompt.
• Prefer a task-specific AI Function when it returns exactly what you need; use ai_query to choose your own endpoint or enforce a responseFormat. ai_mask hides entities, ai_extract pulls them out.
• Format instructions must be explicit and measurable, say what not to output, and live in the system prompt for multi-turn chats; temperature, max_tokens and stop sequences are not format controls.
• Few-shot examples fix edge cases without training, but unbalanced examples bias labels — cover every label and input style.
• json_schema enforces your keys, types and enums; json_object only guarantees valid JSON; allow null for values that may be absent or the model will invent them.
• Closed label sets need an enum, a fallback label and clear definitions where labels overlap.
• Inputs are provided plus retrieved data, scoped by a trusted identity; outputs carry the consumer's shape, passage-level citations and a model-judged not-covered flag; batch table in/column out when nobody waits, serving (streamed for long answers) when someone does.

• Trace the data to pick chain components: retriever → prompt template → chat model → output parser, with the parser chosen from the required output (string, list, typed object).
• Skip the retriever when the whole input fits in the context window, and use retrieval, not fine-tuning, to keep answers current with changing documents.
• Use a chat prompt template with a system message for fixed instructions; use a JSON-schema response format when code reads the output, with nullable fields as a type list such as string and null.
• A schema guarantees shape, not truth: build sources from retrieved-chunk metadata in code.
• Rewrite follow-ups into standalone queries from history plus the latest message, and skip the rewrite on the first turn.
• Request fields that decide which documents are allowed belong in retriever metadata filters; the filter column must be synced into the index.
• Fixed steps call for a chain, varied requests for a tool-calling agent, and skewed traffic for a cheap router in front of the agent.
• Tool names, function and parameter comments and retriever descriptions are what the model reads to choose tools; supply facts like today's date in the prompt.
• Order tools by dependencies, gather knowledge before actions, parallelize independent calls; enforce preconditions, least privilege and idempotency inside action tools.
• Knowledge Assistant answers from documents with citations, Information Extraction fills tables from documents, the Multiagent Supervisor routes by subagent descriptions under end-user access; custom logic calls for an Agent Framework agent.

Data Preparation

• Index only authoritative, current sources meant for everyone who can query the index; keep sensitive records out at ingestion.
• If the answer is not in the chunks table, add the source that holds it; retrieval tuning cannot create missing knowledge.
• No text layer means OCR (pytesseract) or ai_parse_document; tables in text PDFs come out with pdfplumber's extract_tables().
• Clean before you chunk: strip running headers, footers, navigation, quoted replies and duplicates, but never punctuation, stop words or tables.
• Split on headings with MarkdownHeaderTextSplitter or HTMLHeaderTextSplitter; split large tables into row groups that repeat the header row.
• Chunks over the embedding limit are silently truncated; measure chunks in the embedding model's own tokens.
• Context budget: prompt + answer reserve + (chunks retrieved × chunk size) must fit the generator's window.
• Use modest overlap (about 10-20 percent) to bridge chunk boundaries without creating near-duplicates.
• Write chunks with saveAsTable to catalog.schema.table, with a unique id and metadata columns, enable change data feed, then create the Delta Sync index.
• Ingest incrementally with Auto Loader and a fixed checkpoint; replace a changed document by deleting its rows and appending new chunks.

• Precision@k divides relevant-in-top-k by k; recall@k divides it by all labeled relevant chunks. Measure both at the k the chain actually passes to the LLM.
• As k grows, recall can only rise and precision usually falls; that is expected, not a regression.
• Recall and precision are set metrics; use MRR when only the first hit matters and NDCG when several graded results and their order matter.
• RetrievalRelevance and RetrievalGroundedness run without ground truth; RetrievalSufficiency (and Correctness) need expectations.
• Retrieval judges read RETRIEVER spans whose output is a list of documents with page_content and optional metadata.
• Sufficiency passing with groundedness failing points to generation; groundedness passing with sufficiency or relevance failing points to retrieval.
• Label evaluation sets with expected facts and source document IDs, not chunk IDs, and compare every configuration on one fixed set.
• Parent-child matches on small chunks and passes parents; semantic chunking splits on topic shifts in unstructured text; enrichment puts source context into the embedded text.
• Hybrid search (keyword + vector, merged with RRF) fixes exact identifiers; a reranker reorders candidates but cannot add ones the first stage missed.
• The built-in reranker reads columns_to_rerank in order within about 2,000 characters, so list the chunk text first; it suits RAG agents, not high-throughput, millisecond-budget search.

Application Development

• LangChain chains fixed, linear steps; LangGraph handles loops, branching and pause-and-resume state; LlamaIndex centres on ingestion and query engines; DSPy compiles prompts against a metric; a single fixed call needs no framework.
• ChatDatabricks is for chat endpoints, DatabricksEmbeddings for embedding endpoints, DatabricksVectorSearch for indexes and UCFunctionToolkit for Unity Catalog functions as tools.
• DatabricksVectorSearch needs only index_name for a managed-embedding Delta Sync index; self-managed and Direct Access indexes also need text_column and the same embedding model that built the vectors.
• Prompt Registry variables use {{double braces}}; LangChain uses {single braces} and escapes literals by doubling; to_single_brace_format() bridges the two without changing the registered version.
• Registry versions are immutable and aliases are mutable: serve from a named alias, register fixes as new versions, evaluate, then move the alias with set_prompt_alias().
• optimize_prompts registers its result as a new version and needs a predict_fn that loads and formats the registered prompt; DSPy optimizes only programs written in DSPy.
• Augment from what the request contains: look up exact records by key fields, expand internal terms before retrieval, add only the detected intent's instructions, and scope access with session fields.
• Delimit and label context, pass fewer chunks best-first, and put the latest question last.
• Move a baseline to a target with positive, specific instructions and a target-style example; temperature, max_tokens and bigger models do not define the target.
• Ask for reasoning before the answer, put the answer in its own parseable field, and define an exact fallback for questions the context cannot answer.

• Assess responses by reading them beside their retrieved context; latency, length and similarity scores cannot reveal hallucinations or unsafe content.
• If the right fact is in the retriever span but the answer is wrong, it is a generation failure; if the fact never reached the prompt, fix retrieval.
• A triggered Delta Sync index only includes new source rows after a sync runs.
• Grounded is not the same as safe: hazardous source material in the index must be removed from that application's corpus.
• Fix over-refusal by making the rule or classifier more precise, never by removing the protection you are required to keep.
• Choose chunking with the same labelled set, embedding model, retriever and prompt across strategies, and apply the stated constraint before comparing quality.
• Match chunking changes to symptoms: boundary cuts call for overlap, split units for larger structure-aligned chunks, diluted answers for smaller sections, and position-dependent misses for chunks within the embedding model's input limit.
• The context window must hold the prompt, history, every retrieved chunk and the answer.
• AI Gateway guardrails (Safety, PII detection) are per endpoint, need no code changes, and their output checks do not run on streaming requests or embeddings models; topic moderation and keyword filtering are deprecated.
• Prompt rules are not enforcement: add Llama Guard with custom categories for model-based rules and deterministic code checks for exact rules.

• Filter on hard requirements first, then choose the smallest, cheapest model that meets the measured quality bar.
• The LLM's context window must hold the full prompt plus the maximum reply; buy no more window than that.
• Structured outputs via response_format constrain decoding to a JSON schema; prompt instructions and examples only encourage a format.
• Use a reasoning model only when it measurably raises accuracy on multi-step work and latency and cost allow it.
• Deploy Instruct versions; start from base versions for fine-tuning and continued pretraining.
• In-workspace processing or future fine-tuning means open weights on Model Serving; a permitted proprietary winner goes behind an external model endpoint.
• An embedding model's input window must cover the longest chunk and the longest query; gte-large-en takes 8,192 tokens, bge-large-en 512.
• Switching embedding models means re-embedding the corpus and the queries with the new model; equal dimensions do not make vectors comparable.
• Screen on the model card (task, languages, context, licence); decide on experiments run on the same evaluation set.
• Match the metric to the task: ROUGE for summaries, BLEU for translation, per-class recall for rare classes, perplexity for domain fit.

• ResponsesAgent wraps any framework's agent with an inferred signature and automatic tracing; streaming needs predict_stream yielding delta events and a final output_item.done event.
• Per-request extras go in custom_inputs; model_config is fixed when the logged version loads.
• Log agents with Models from Code (agent.py ending in set_model); local imports need code_paths; validate the logged URI with mlflow.models.predict.
• Automatic passthrough covers only resources declared at log time: a Genie space also needs its tables, SQL warehouse and functions, and adding a resource means logging a new version.
• On-behalf-of-user auth uses a UserAuthPolicy and a client created inside predict; it is the choice when row filters or audit must reflect each user.
• agents.deploy() creates the endpoint, credentials, Review App, tracing and inference tables; re-running it with the same model name and a new version updates the endpoint in place.
• Evaluation (mlflow.genai.evaluate on a curated dataset) gates releases; monitoring (registered scorers with per-scorer sample_rate and filter_string) watches live traces with the same scorers.
• Ground-truth judges (Correctness, RetrievalSufficiency, ExpectationsGuidelines) stay offline; production traces become regression data via search_traces, merge_records and expert labeling.
• Use Genie for exact answers over governed tables and a retriever for documents; GenieAgent descriptions drive supervisor routing, while the space's example SQL, descriptions and synonyms drive SQL quality.
• The Genie Conversation API is asynchronous and runs SQL as the caller, who needs CAN RUN, CAN USE on the warehouse and SELECT plus USE CATALOG and USE SCHEMA.

Assembling and Deploying Applications

• Pure LangChain chain that others will compose: LangChain flavor. Custom pre/post-processing: pyfunc PythonModel. New agent for Agent Framework: ResponsesAgent.
• load_context runs once per load (clients, tokenizers, artifact files); predict runs per request; __init__ state is pickled when you log an instance.
• Enforce anything every caller must get (masking, JSON shape, removing IDs) in predict, not in the prompt, the clients or a gateway guardrail.
• LCEL: prompt | chat model | output parser; a RAG chain needs a branch per template variable (retriever | format_docs for context, RunnablePassthrough for the question).
• Log chains from code with mlflow.models.set_model; the file must build everything itself, and ModelConfig supplies values via model_config at logging time.
• Packages: extra_pip_requirements appends, pip_requirements replaces (use only one of these and conda_env); local modules: code_paths; data files: artifacts read via context.artifacts.
• Unity Catalog needs a three-level name and a signature; an input_example gives you both a signature and a sample request.
• Embed queries with the model that embedded the index, and pick an embedding model whose window fits your chunks.
• Promote with aliases (no stages); a new version never moves an alias. Create: CREATE MODEL; new version: owner or CREATE MODEL VERSION; load: EXECUTE.
• Test dependencies with mlflow.models.predict in an isolated environment (virtualenv or uv), not env_manager="local".

• An endpoint is serving compute that hosts up to 50 indexes; each index is a Unity Catalog object governed with grants.
• Delta Sync follows a Delta table (Databricks-computed or self-managed embeddings); Direct Vector Access has no table and changes only through upsert and delete calls.
• Databricks-computed embeddings: embedding_source_column + embedding_model_endpoint_name, query with query_text. Self-managed: embedding_vector_column + embedding_dimension, query with query_vector from the same model.
• Triggered sync updates only when sync() or Sync now runs and is the cheapest mode; continuous sync is near real time but runs a cluster all the time and is standard-endpoint only.
• Schema or embedding-column changes need a new index: create it, switch the app, then delete the old one.
• Filters are a dictionary on standard endpoints and a SQL-like string on storage-optimized endpoints, where very selective filters can return nothing because of overfetch-then-filter.
• Scores are 1 / (1 + squared L2 distance); normalize vectors for cosine-equivalent ranking, and convert cosine cut-offs before thresholding scores.
• Standard endpoints: tens of milliseconds, high QPS via target_qps, capacity that shrinks as dimension grows. Storage-optimized: a billion-plus vectors, cheaper, a few hundred milliseconds, triggered sync only, no columns_to_sync.
• Cut cost by consolidating idle endpoints and using triggered sync; cut latency with service principal OAuth, smaller num_results, an always-on embedding endpoint and a separate query embedding endpoint serving the same model.
• A non-owner needs USE CATALOG, USE SCHEMA and SELECT on the index to query it.

• Endpoint permissions are CAN VIEW < CAN QUERY < CAN MANAGE; callers need CAN QUERY, version rollbacks and traffic changes need CAN MANAGE, and UC privileges on the model do not open the endpoint.
• Secrets reach a served model as environment variables written exactly {{secrets/scope/key}}; the endpoint creator needs READ on the scope, callers do not.
• Prompt Registry access and other unsupported resources need manual authentication, preferably service principal OAuth in secret references; overriding those variables disables passthrough for every resource.
• AI Gateway rate limits cap use without removing access: endpoint, default per-user, custom per user or service principal, and group limits; TPM limits are not available on custom model or agent endpoints.
• Pay-per-token suits prototypes and irregular traffic (a priority tier serves latency-sensitive production); provisioned throughput suits high-throughput production, fine-tuned UC models and HIPAA; external model endpoints govern third-party providers with your own key.
• The OpenAI client works with base_url = workspace + /serving-endpoints and model = the serving endpoint name.
• Serve an application by registering the chain in Unity Catalog and deploying it to Model Serving; agents.deploy() also provisions resource credentials, the Review App and tracing.
• ai_query() is for batch over tables: responseFormat structures chat output, failOnError => false isolates bad rows, submit the whole dataset in one query, and run on serverless/pro SQL or DBR 15.4 LTS+, not Classic SQL.
• Served models are stateless across replicas: resend messages for short chats, use Lakebase for resumable sessions and synced tables for millisecond lookups of Delta data.
• Scope long-term memory by the authenticated identity and grant the deployed agent's service principal the access it needs on the store.

• Deploy code, not models, when production data may only be read in production: a bundle with dev, staging and prod targets builds each environment's index and endpoints there.
• CI should run as a service principal using a federation policy that trusts the CI system's OIDC token, so no long-lived secret is stored.
• Component tests call one tool, parser or retriever directly with fixed inputs and a stubbed LLM; LLM-judge evaluation belongs in the promotion gate, not the unit tests.
• Endpoint Ready only means the container started; a smoke test of representative requests catches missing resources before production.
• Triggered Delta Sync indexes need sync() after each load; storage-optimized endpoints support only Triggered sync, and schema or embedding-model changes need a new index built, synced, switched to, then the old one deleted.
• Prompt versions are immutable and aliases are pointers: deployed apps load a named alias, regression tests pin a version number, and @latest follows every registration.
• Promote by pointing the next alias at the evaluated version; roll back by pointing production back, and move any alias that automation copies from as well.
• Alias loads are cached for 60 seconds by default; cache_ttl_seconds shortens that, and 0 queries the registry on every call.
• Managed MCP servers are Databricks-provided with nothing to host; external ones need an existing MCP server registered through a Unity Catalog HTTP connection, with callers granted EXECUTE on the MCP and USE CONNECTION kept for authors and admins; custom ones run on Databricks Apps over Streamable HTTP.
• Per-user data in an interface needs the user's token forwarded (Apps user authorization or a Teams federation policy) plus on-behalf-of-user authorization in the agent.

Governance

• PII detection Mask replaces personal data and lets the request continue; Block rejects it. Use Mask when questions must still be answered.
• Only input-side masking keeps personal data from reaching a model or external provider; the Safety guardrail has no Mask behaviour.
• Output guardrails do not support streaming or embeddings models, and endpoint guardrails never see citations shown straight from Vector Search, so mask the knowledge base at ingestion with ai_mask.
• ai_mask masks only the labels you list: match them to the privacy rule and never mask what retrieval needs. ai_extract and ai_classify do not mask.
• Mask entities rather than dropping records: exclusion, synthetic data and access restrictions lose content or leave the data retrievable.
• Screen user input with a model-based classifier before retrieval and tool calls; screen retrieved or fetched content for indirect injection where it enters.
• Controls that hold when the model obeys an injection live in code and permissions: least-privilege grants, on-behalf-of-user auth, tool-derived parameters, approval, secrets in Databricks secrets.
• Public is not licensed; CC BY-NC forbids commercial use; check every use (retrieval, quoting, training) against the license and terms.
• Store license and provenance columns per chunk, filter by license inside the search, and remember the index schema is fixed at creation.
• Fix problematic text in the data: filter at ingestion, or switch to an authoritative source when the problem pervades it.

Evaluation and Monitoring

• Compare candidate LLMs on one shared, representative evaluation set with identical scorers, plus latency and cost; eliminate on every hard threshold first, then optimise the stated objective.
• MoE generation speed tracks active parameters, but GPU memory must hold the total parameters.
• mlflow.genai.evaluate calls predict_fn(**inputs); records with outputs or a trace column are scored as they are and must not come with a predict_fn.
• To test a new version on captured requests, turn the requests into inputs, drop traces and outputs, and keep predict_fn; to_predict_fn("endpoints:/<name>") evaluates a deployed endpoint.
• Correctness, RetrievalSufficiency and (per Databricks) ToolCallCorrectness need ground truth in expectations; ExpectationsGuidelines needs per-record guidelines instead; RelevanceToQuery, Safety, Guidelines, RetrievalGroundedness and RetrievalRelevance need neither.
• @scorer functions get inputs, outputs, expectations and trace by name; return primitives, one Feedback or a list of distinctly named Feedback; uncaught errors become Feedback with value None.
• Tool order and retrieval results live only in the trace: use trace.search_spans in a code scorer for deterministic checks, or make_judge with {{ trace }} and a model for semantic ones.
• Expectations are ground truth, feedback rates the output; labeling sessions sync expectations to datasets by inputs, and judge.align() needs feedback named exactly like the judge, ideally 50 to 100 traces with about 30% negative.

• Match the metric to the risk: time to first token for streaming chat, throughput for deadline-bound batch work, tokens per request for cost, a judge score for quality.
• Inference tables hold request and response payloads; rows arrive in the background within about an hour, oversized payloads are null with logging_error_codes, and error responses such as 429 may not be logged.
• Usage tracking (system.serving.endpoint_usage) holds exact token counts and requester but no text; join served_entities on served_entity_id for names, and the inference table on the request ID for text.
• usage_context in the request body attributes usage per end user or tenant; rate limits key on identities, never on usage_context.
• agents.deploy() sets up real-time traces and inference tables; endpoints you create yourself need ENABLE_MLFLOW_TRACING=true. Scorers, rate limits and scale to zero are separate choices.
• Agent Monitoring runs registered scorers on sampled live traces; backfill_scorers scores past traces, and custom scorers must be notebook-defined with imports inside the function.
• Custom rate limits override User (Default); a user-specific limit beats a group limit; a user in several groups gets the most generous group limit; the endpoint limit caps everyone; agent endpoints take QPM only.
• Cut cost with scale to zero, pay-per-token for sporadic use, ai_query for batch work and max_tokens for unread output; allocate spend with endpoint tags in system.billing.usage and alert with budgets.

Yes — the whole Databricks GenAI deck is free and needs no account, as is every Databricks GenAI revision lesson. Create a free account to save progress and drill practice questions; full-length mock exams are a one-time unlock per certification.