Databricks GenAI Engineer: Response Quality, Safety and LLM Guardrails

To assess a response qualitatively, read it beside the exact context the model was given and tag each issue you find: is it relevant, complete, grounded, correctly formatted and safe? Each property is judged separately, because one answer can pass some and fail others, and each failure points to a different fix.

MLflow Tracing makes this practical. A trace of a RAG request records the user input, the retriever span (which chunks came back, in what order) and the LLM span (the full prompt and the generated output). Sample traces, read the response against its retrieved chunks, and tag what is wrong. Counting the tags turns an impression ("it hallucinates sometimes") into a priority list (most failures trace back to one stage).

Signals that do not involve reading the response (latency, token counts, answer length, retriever similarity scores) are operational measures. They cannot reveal a hallucination, an unsafe answer or a format problem. Structured, automated scoring with MLflow judges and scorers belongs to evaluation and monitoring; this topic is about recognising the issue types and choosing the fix.

A bad answer is a retrieval failure when the needed fact never reached the prompt, and a generation failure when the fact was in the prompt but the model ignored, contradicted or embellished it. The retriever span settles which one you have, and the fix must target the stage that failed.

• The fact is in the retrieved chunks, but the answer is wrong. Suppose the top chunk names the Berlin office as the contact for EU customers and the answer names a different office. Retrieval, chunking and embeddings did their job on this request; the failure is in generation, so look at the prompt's grounding instructions and at the model.
• The fact is not in the retrieved chunks. Under an "answer only from the context" instruction, saying the answer cannot be found is the model behaving correctly. The cause is upstream, so the fix belongs in retrieval: queries, chunking, filters or the index itself.
• Nothing about the subject is retrieved at all. First confirm the content is in the index. A Mosaic AI Vector Search Delta Sync index in triggered mode only picks up new or changed source rows when a sync runs (manually, from a job or via the API), whereas continuous mode keeps syncing. Retrieval tuning changes how indexed chunks are found and ranked; it has nothing to work with until the content is indexed.

Reranking deserves one more note: it only reorders chunks that were already retrieved. It cannot bring back a chunk that was never returned, nor text that was never embedded.

An ungrounded response contains claims that the retrieved context does not support. In RAG this is usually the model filling a gap from its general (parametric) knowledge: fluent, plausible, and not your organisation's facts. A common pattern is a correct, supported sentence followed by an invented extra detail.

When retrieval is accurate and the answer is in the context, the cheapest fix, needing no retraining, is a grounding instruction in the system prompt: answer only from the supplied context, and say plainly when the context does not contain the answer. The same instruction is the right fix when the knowledge base simply does not cover the question: no amount of retrieval tuning can supply a document that does not exist, so the model must be told to admit the gap.

The guiding rule is to fix the stage that produced the invention. The invented material comes from the model's general knowledge, so changes that only alter how much it sees, how varied its wording is or how much it knows do not tell it where its facts must come from. Sampling settings such as temperature control randomness, not the source of the content. Fine-tuning can help in some cases, but it is slow and costly, so it is rarely the first move.

When a grounding instruction lowers the rate but a requirement says a specific fabrication must never reach users, add a deterministic output check as described below.

A safety issue is an output that could harm, endanger or unfairly treat someone; a quality issue makes the answer less useful. The distinction sets severity: safety issues block a release and need enforced controls, while quality issues are prioritised alongside other improvements.

• Quality: a date in the wrong regional format, a summary that drops a minor caveat, a reply in an overly casual tone, an off-topic but harmless answer.
• Safety: advice that could cause physical injury or a medical harm; help with bypassing a protective device; toxic or insulting replies; decisions or descriptions that rely on protected characteristics such as ethnicity or religion (bias and discrimination risk); disclosing a live credential or someone's personal data.

Grounded does not mean safe. If an HR assistant's index contains salary spreadsheets, it can quote a colleague's pay with every word supported by the context. A groundedness review passes it; a safety review must not. The root cause is the corpus: material this audience must never receive should not be retrievable by this application, so remove it from the index (or filter it out for this application). As long as the text is retrievable, the assistant can repeat it.

Likewise, when the model voices something harmful that grounding, formatting or retrieval changes cannot stop, the response needs an enforced output check, with a prompt rule as a supporting layer rather than the only one.

Over-refusal is a guardrail or prompt rule blocking a safe, in-scope request. It is a real quality failure: a developer assistant that refuses "How do I kill a hung process?" or a photography assistant that refuses "How do I shoot a portrait in low light?" fails its users. Track false positives (safe requests refused) alongside false negatives (harmful content let through); tuning a guardrail means watching both rates on a labelled set of legitimate and harmful examples.

The fix is to make the check more precise, not to remove it. Whatever you change must keep the protection the requirement names while narrowing what it catches.

• Over-broad prompt rule: a blanket instruction that forbids a whole subject area also catches the legitimate questions in it. Rewrite it to name the specific harms to refuse and add examples of allowed requests. This is quick and keeps protection.
• General-purpose safety filter misfiring on a specialist domain: the endpoint Safety guardrail is an on/off choice per direction with a fixed taxonomy, so you cannot tune what it considers harmful. If compliance still requires model-based screening, run a classifier you control, whose categories you define for the domain (see Llama Guard below).

Note that a refusal caused by the model correctly saying "the context does not contain this" is not over-refusal; that points back to retrieval.

You select a chunking strategy by building an index with each candidate strategy, running the same labelled evaluation set against each, and comparing retrieval and answer metrics against the application's constraints. The choice is empirical; it is made before launch, not inferred from proxies or post-launch ratings.

Make the comparison fair by changing only the chunking:

• One labelled question set (questions with expected answers and source documents) for every strategy.
• The same embedding model, retriever settings (such as k), prompt and LLM in every run.

Treat hard constraints (a prompt-token budget, a latency target) as a filter first, then rank the strategies that pass on quality. Quality means metrics measured against the labelled expectations: retrieval metrics such as recall@k or hit rate (did the right source come back?) and answer metrics such as correctness. Numbers the system reports about itself, such as similarity scores, are not quality measures: they are not comparable across chunk sizes and a wrong chunk can score high. Index size is a cost factor, and a configuration that was never run against the set has no evidence behind it.

Detailed chunking designs (parent-document, semantic splitting) and reranking mechanics are covered with retrieval design; here the skill is reading the evaluation.

Fitting chunks into the context window

Every request must fit system prompt + conversation history + all retrieved chunks + the generated answer into the LLM's context window. Generated tokens count against the window too, so you must reserve room for the answer, and every retrieved chunk counts, not just the largest.

Work the budget by subtraction:

1. Start from the context window.
2. Subtract the maximum system prompt plus history.
3. Subtract the answer reserve.
4. Divide what is left by the maximum chunk size and round down.

Example: a 32,768-token window, 4,000 tokens of prompt and history, 2,000 reserved for the answer leaves 26,768 tokens; with chunks capped at 2,000 tokens, 13 chunks fit (26,000) and 14 do not (28,000). Forgetting the answer reserve or the prompt and history produces larger numbers that overflow.

When several configurations fit (say, three 2,048-token chunks or six 1,024-token chunks), the arithmetic only tells you which are possible. Which retrieves better is a question for the evaluation set, because fewer large chunks and more small chunks behave differently. Raising k (the number of chunks retrieved) always eats into the room left for history and the answer.

Evaluation symptoms point to specific chunking changes. Chunks that are too small or split at bad boundaries fragment answers; chunks that are too large dilute them and can be truncated by the embedding model.

The truncation case needs the model's limit. Embedding models silently truncate input beyond their maximum length. On Databricks, databricks-bge-large-en accepts up to 512 tokens, while databricks-gte-large-en accepts up to 8,192. A chunk three times BGE's limit is represented only by its first third, so the rest of it is unsearchable, and every later retrieval step works from that incomplete embedding. The fix therefore sits in the chunk size or the model choice.

Overlap is sized for neighbouring text: it repairs cuts at a boundary, and adds tokens to every chunk.

AI Gateway guardrails are content checks configured on a Mosaic AI Model Serving endpoint that screen requests (input) and responses (output) without any change to application code. Because they live on the endpoint, every app that calls it is covered at once, which makes them the fastest way to apply a screening requirement to applications you cannot change. Databricks now labels these endpoint-level AI Gateway settings as the legacy experience and offers Unity Gateway as the newer governance layer; the exam guide describes the endpoint guardrails, which is what this section covers.

The endpoint guardrails are:

• Safety: filters harmful or unsafe content in requests and responses using a built-in classifier (Llama Guard-based) with its own fixed taxonomy. You cannot add your own categories to it.
• PII detection: finds personal data such as names, addresses and card numbers, with the behaviour Block, Mask or None. It does not accept custom term lists and does not detect harmful content.

Behaviours to know:

• Per endpoint. Each endpoint has its own configuration. When an application starts routing some traffic to another endpoint, that endpoint is unprotected until its own guardrails are enabled. Enable them on every endpoint that produces user-visible text.
• Per direction. An input check sees only what the user sent. Anything the application adds afterwards, such as retrieved documents or tool results, reaches the user through the reply, so only an output check screens it.
• Other AI Gateway features do not block content. Rate limits cap request volume, usage tracking records tokens and cost, and inference tables log requests and responses for later analysis.

Deprecated: AI Gateway's topic moderation and keyword filtering guardrails are deprecated and no longer enforced (support ended in 2025). Older configurations may still display them, but they protect nothing. Implement topic restrictions and term lists as checks you own: pre- and post-processing in the chain (for example a pyfunc model's predict) or a classifier step.

Output guardrail limits: streaming and embeddings

Output guardrails are not supported for streaming requests or for embeddings models. The documented limits apply to output guardrails only.

• Streaming. An output check needs the complete response, so a streamed completion is delivered without output screening. If a requirement says nothing unscreened may ever be displayed, the screened response must exist before anything is shown, which means requesting it without streaming.
• Embeddings. An embedding endpoint returns vectors, not text anyone reads, and output guardrails are not supported for it. User-visible output screening belongs on the endpoint that generates text.

Llama Guard is an LLM fine-tuned to classify a prompt or a response as safe or unsafe against a list of categories supplied in its own prompt. Serve it on an endpoint and call it as a step before and/or after the main LLM in the chain when you need a model-based check the built-in Safety filter does not provide.

• Custom categories need no training. Write the category definitions (for example personalised legal advice or promotion of gambling) into the Llama Guard prompt. Fine-tuning is unnecessary for this and too slow for a near-term launch.
• Its output is a verdict. It generates safe, or unsafe followed by the violated category codes on the next line. The chain parses the first word to continue or return a refusal.
• Check the turn the rule is about. Llama Guard assesses whichever turn you ask it to: the user's prompt, or the agent's response given the conversation. A rule about what the assistant says is checked on the response turn, which means sending the drafted reply.

Robust guardrails are layered: system-prompt rules shape behaviour, and independent checks outside the main model enforce it. A prompt rule is not enforcement, because the same model that can be talked past the rule decides whether to follow it. When red-team testing shows that some share of crafted requests still gets the forbidden content, the prompt layer has reached its limit and the rule needs an enforced check: the endpoint Safety guardrail for harmful content, a classifier or scope check for business rules, and an output check on replies.

Any measure that still asks the main model to police itself is prompting. It can reduce the failure rate but cannot guarantee catching the remainder.

When the rule is mechanical, a deterministic post-processing check is the cheapest guarantee, with no extra model call:

• Citations: compare cited source IDs with the IDs of the chunks actually retrieved for that request, and handle any that do not match.
• Structured payloads: validate output against its schema before it leaves the application, and design an explicit failure path instead of passing invalid output on. Structured-output support in the serving layer reduces malformed output where the model supports it.
• Required text: an exact-text requirement is a string comparison, and a missing string can be added in code.

The Safety guardrail looks for harmful content only; it knows nothing about fabricated IDs, malformed payloads or missing required text. In a pyfunc chain these checks sit in pre- and post-processing, alongside any topic or term checks that replaced the deprecated gateway filters.