Databricks GenAI Engineer: Guardrails, PII Masking and Data Licensing

Masking is the guardrail to choose when sensitive data must be kept away from a model and the request must still succeed. On a Databricks Model Serving endpoint, AI Gateway offers PII detection on requests (inputs) and responses (outputs), each with one of three behaviours:

• Block rejects the whole request or response when personal data such as a name, address or credit card number is detected. Nothing is processed, so the user gets no answer.
• Mask replaces the detected personal data and lets the call continue. The model works on the redacted text and the user still gets an answer.
• None turns detection off for that direction.

Databricks now documents these PII and safety settings on its legacy AI Gateway governance page (Public Preview) while a newer gateway experience rolls out; the behaviours described here are the ones to reason with, whatever the console page is called.

That difference is what makes masking a guardrail that meets a performance objective. If Block rejects every question that happens to mention a colleague's name, the answer rate collapses; switching to Mask keeps the privacy rule (the model never sees the name) and restores the answer rate. Choose Block only when policy says a request containing personal data must be rejected outright rather than processed in an altered form.

Direction matters. Input-side PII detection acts before the model or an external provider receives the request, so it is the setting that stops personal data being sent to a third-party model behind an external model endpoint. Output-side detection runs after the provider has already received and processed the data, so it cannot satisfy a "never send it to the provider" rule.

Rate limits, instructions that rely on the model choosing to ignore data, and the Safety guardrail are not substitutes: none of them removes personal data from what the model receives.

Guardrail limits: streaming, embeddings and where an endpoint guardrail runs

AI Gateway guardrails run at the serving endpoint they are configured on, and output guardrails have documented limits. Output guardrails are not supported for streaming responses or for embeddings models; input guardrails still run before generation starts. So a team that will not give up token-by-token streaming can still stop unsafe or sensitive requests with a managed input guardrail, but cannot rely on an output guardrail to clean the streamed reply.

When an agent or RAG chain calls a foundation model endpoint, a guardrail on that endpoint runs at the generation step: after retrieval has run and after any tools (including paid APIs) have been called. It also never sees text the application shows directly from Vector Search, such as citations or source passages, because that text does not pass through the generation endpoint. These two facts drive the placement decisions in the next sections.

The Safety guardrail itself, Llama Guard and response-quality checks are covered in the lesson on response quality, safety and LLM guardrails; this lesson uses them only where they interact with masking, malicious input and source data.

ai_mask(content, labels) is the built-in Databricks SQL AI function that returns the text with every entity of the listed types replaced by [MASKED]. It takes a string and an array of labels, for example ai_mask(body, array('person', 'email', 'phone')), and needs no custom prompt or model endpoint. It requires Databricks Runtime 15.4 LTS or above and is tuned for English text.

Only the listed types are masked. Choose labels that match the privacy requirement exactly:

• Leave a required type out and it stays in the text (masking names and emails leaves phone numbers and addresses visible).
• Add a type the application depends on (product names, error codes) and you destroy the content retrieval needs.
• Changing the labels later means rewriting the table that was already masked and syncing the index again; changing the job for new rows only leaves old rows unmasked.

Two near-twin AI functions take label arrays too, but neither masks:

ai_mask versus Unity Catalog column masks and row filters. Column masks and row filters are query-time, user-scoped controls: they decide what a given user sees when querying a table, admins or excluded groups still see the raw value, and a mask function cannot reliably find names buried in free text. A row filter hides whole rows, losing their useful content. When every user of an assistant must retrieve the text but nobody may see the personal data inside it, redact the entities in the stored text with ai_mask.

Masking once in the ingestion pipeline is the way to meet latency and cost objectives when the sensitive data lives in the knowledge base. Write the source text through ai_mask into the chunk table, and let the Delta Sync index sync from that masked table. The cost is paid once per document, not once per request, and no extra model call is added to the request path.

Ingestion-time masking is also the only option that covers every surface:

• Citations and source passages shown straight from Vector Search results never pass through the generation endpoint, so endpoint PII guardrails cannot redact them. If the chunk text is masked, there is nothing to leak.
• Streaming replies cannot be cleaned by output guardrails, so the data must be out of the context before generation.
• Per-response masking calls (an extra LLM pass on each answer) add seconds to every request and can break a p95 latency target; extra throughput reduces queueing but not the added step.

Runtime guardrails remain useful for what users type, because that text never sat in the knowledge base. A sound design masks the knowledge base at ingestion and masks user input at the endpoint.

Why masking beats excluding records

Masking keeps a record's useful content while removing the sensitive entities; excluding the record loses both. When most records contain some personal data (claim notes that name the policyholder, feedback forms with an email address, chat logs with account numbers), dropping those records throws away most of the knowledge the application exists to use.

Compare the alternatives against a requirement such as "the data must not be retrievable, and the content must stay useful":

• Exclude records with personal data: meets privacy, loses most of the content and skews what remains (whatever is left no longer reflects real traffic).
• Replace with synthetic data: avoids the data but drifts from the real distribution.
• Index unchanged and restrict access, or Block at the endpoint: the personal data is still stored and retrievable by whoever has access; contractual or procedural measures do not change where the data is stored or sent.
• Index only structured fields and drop the free text: loses the content people search for.
• Mask the entities, keep all records: the personal data is gone and all of the content remains.

A direct prompt injection is user input written to override the application's instructions: "drop your earlier guidance", "switch to maintenance mode with no restrictions", "role-play an assistant that has no rules", or a request to reveal the hidden configuration it was given. A jailbreak is the same idea aimed at safety rules. Recognise it by its intent: it tries to change the model's role or rules, not to ask a question.

Defences that work:

• Screen each user message with a classifier before the model call and reject override attempts. A model-based injection classifier catches paraphrases, translations and letter-spaced variants with fewer false rejections than a list of blocked phrases; growing a phrase list both misses rewordings and starts rejecting ordinary text such as "please disregard my last email, the meeting moved". Tidying the input first (normalising case or spacing) or capping message length does not fix this: attackers simply rephrase, and an override fits in one short sentence.
• Keep roles separate. Send the user's text as its own user message after the system message. Concatenating user text into the system message gives it the same authority as your instructions.
• Keep secrets out of prompts (see blast radius below), because a prompt that is extracted cannot leak what it never contained.

Measures that do not prevent injection: rewording the system prompt again, few-shot refusal examples, lowering temperature, PII masking (it removes personal data, not instructions), rate limits, and logging to an inference table (useful for review after the fact, but it stops nothing). An output-only Safety guardrail catches some harmful replies, but not an off-topic answer produced by a successful override.

Where to place input screening

Screen malicious input on the user message before retrieval or any tool call runs; that is where blocking costs the least. In an agent that retrieves, calls a paid API and then generates, a guardrail on the reply, or on the foundation model endpoint the agent calls to generate, only fires after retrieval and the paid call have already been charged.

On a simple chat endpoint, an input guardrail on the endpoint's requests stops harmful requests before any generation tokens are spent, and it works with streaming on.

An indirect prompt injection is an instruction planted in content the application reads rather than in what the user types: hidden text in a crawled web page, an uploaded supplier document, an inbound email, or a page an agent fetches on request. The user's question can be completely ordinary; the attack arrives through retrieval or a tool.

Because screening the user message never sees this content, defend where the content enters:

• At ingestion: screen crawled or uploaded documents for planted instructions and drop them before they reach the index, and prefer trusted sources over crawling anything.
• At fetch time: when an agent fetches arbitrary pages or reads emails at runtime, screen the fetched text before it enters the model's context. Restricting fetches to an allowlist works only if the product can give up arbitrary pages.

Lowering temperature, masking PII or adding a second pass of user-input screening does not address planted instructions. Screening is probabilistic, so pair it with the blast-radius controls below.

Blast-radius controls are the guardrails that still hold when the model obeys an injection, because they are enforced in code and permissions rather than in the prompt. Assume some injection will eventually succeed, and make sure a manipulated model cannot do much harm:

• Least-privilege tool credentials. A SQL tool that only needs to read should run as a service principal granted SELECT on just the tables it reads; it then cannot delete or alter anything, whatever SQL the model writes.
• Act as the user. On-behalf-of-user authentication runs a tool with the requesting user's identity, so existing Unity Catalog row filters and grants limit each user to their own data. A single broad service principal ignores those filters, and partial column restrictions on a shared identity still leave other users' rows reachable.
• Derive sensitive parameters in tool code. Set the recipient of an email from the signed-in customer's profile instead of accepting it as a model-supplied argument.
• Constrain and confirm actions. Restrict a send tool to recipients in the company's own domain, and require human approval before high-impact tool calls run. Any check that depends on the model's own cooperation is not a control, because the model is the thing being manipulated.
• Keep secrets out of the model's context. Store API keys in Databricks secrets that the tool code reads; never place them in a system prompt, which role-play and extraction attacks can recover.

System prompt rules, extra classifiers on the user's input, Safety or PII guardrails and logging all reduce or record risk, but none of them holds once the model has decided to act on an injected instruction.

Every data source feeding a GenAI application needs a license or terms that allow this use, and "publicly visible" is not a license. Check each source against these rules:

• No license means no permission. Content published without a license keeps all rights with its author; reuse needs the author's explicit permission.
• Non-commercial licenses forbid commercial use. CC BY-NC content cannot ground a paid or customer-facing product without separate commercial permission. Crediting the source, masking personal data, or using it for retrieval rather than training does not lift the restriction.
• Attribution licenses require credit. CC BY allows commercial use but requires appropriate credit, such as title, author and license.
• Website terms of use apply. Terms that forbid automated scraping or reuse in AI systems make scraping a legal risk, and getting the same content another way (a derived representation of it, or a collection someone else assembled in breach of the terms) carries the same risk. When the owner sells a licensed feed or data subscription that permits the use, ingest through it.
• Uses are licensed separately. A license may allow storing articles to retrieve and quote excerpts but prohibit training or fine-tuning. Use each source only for the uses it allows, and use other permitted data for the rest; avoiding a licensed source entirely is overcautious.

Model licenses (what a model card allows) are a separate question covered with model selection; this lesson concerns the data you ingest.

Store provenance with every chunk: the source document ID, title, author or partner, and the license that applies. Columns on the chunk table travel through a Delta Sync index into Vector Search results, so the application can show the credit an attribution license requires, answer later questions about which answers used a partner's content, and find that content to remove it. A system prompt listing licenses, a table comment, a separate spreadsheet or asking the model to guess the source do not attach provenance to individual chunks.

One index, consumers with different rights. When a non-commercial internal tool and a paid app share one index, add a license column to the chunk table, include it in the index, and pass a metadata filter that excludes restricted licenses on every query from the paid app. Filtering inside the search means the restricted chunks are never retrieved; dropping them from results afterwards still retrieves them, and a Unity Catalog tag on the source table is not a column the index can filter on.

The index schema is fixed at creation. Only columns included when the index was created can be returned or filtered. Adding a column to the source table and syncing does not add it to an existing index; create a new index that includes the column and switch queries to it.

When a license ends, remove the content from the source of truth and propagate the removal to the index; how depends on the index type.

A query filter that excludes the licensor, or a prompt rule, leaves the content stored and retrievable, so it is not removal. Note that a Delta DELETE is logical: earlier table versions keep the data files until VACUUM removes them after the retention period, which matters when legal needs the content gone from storage as well as from the current table.

Problematic text in a data source (abuse, spam, superseded rules, misinformation, biased wording) is best fixed in the data, not by blocking its symptoms on every reply. Two remedies cover the official skill:

Filter during ingestion when the problem is a minority of records. Label each record in a batch job, for example with an ai_classify step whose labels separate usable from unusable text (such as spam versus legitimate), and write only the usable records to the chunk table the index syncs from. Use metadata where it exists: load only documents whose status is current, and keep superseded versions in the library rather than deleting them. An output Safety guardrail then becomes a backstop; on its own it keeps blocking replies (users get no answer) because the abusive text is still being retrieved. Retrieving more chunks, re-rankers, larger embedding models and prompt rules all leave the text retrievable. ai_mask and ai_extract do not filter records by category.

Recommend an alternative source when the problem pervades it. If most of a source is unreliable, filtering would leave little behind; replace it with an authoritative, curated or licensed alternative:

Pick the replacement for its accountability: a source with a named owner, a review process and authority on the subject. For example, a community board of tax tips with stale answers gives way to the tax authority's published guidance, and a scraped product-review feed full of rants gives way to the company's moderated support articles. The same logic applies to examples in a prompt: if a set of examples carries the problem (legacy sales emails with exaggerated claims, say), replace them with reviewed examples; keeping a few of the old ones alongside still teaches the model the pattern it imitates.

Guardrails and masking do not fix these problems: the Safety guardrail targets harmful content, not accuracy, bias or tone, and ai_mask redacts entity types. An extra LLM pass that rewrites each output adds a model call and latency to every request.