Identify and Resolve Fabric Errors: Pipelines, Dataflows, Notebooks, and More

The method that works across every Fabric item type has six steps, and DP-700 scenarios reward you for following them in order rather than jumping to a fix.

1. Locate the failing run. Start in the Monitoring hub (for jobs) or the item's own editor (for continuous items like Eventstreams) and find the failed execution.
2. Read the actual error. Drill to the deepest surface available — the failed activity's error details, the failing table in a refresh, the exception in a cell, the failure record from a KQL command. Never diagnose from the top-level status alone. Watch for the opposite trap too: some faults (a Spark job waiting in a queue, an ingestion that writes empty columns) produce no error at all, so a missing result can be the only symptom.
3. Classify the failure. Almost every error falls into one of five classes: authentication (expired credentials, missing permission), connectivity (unreachable source, gateway, firewall), schema or data (type mismatch, missing column, malformed rows), resource (out of memory, capacity throttling, timeout), or logic (bad expression, wrong path, wrong query).
4. Reproduce at the smallest scope. Preview the failing query on a subset, run the failing cell alone, ingest one file, or execute the T-SQL statement directly. Small reproductions turn a vague run failure into a specific message.
5. Fix the class, not the symptom. An expired credential needs a connection update, not a retry policy; a type mismatch needs a schema fix, not more memory.
6. Rerun from the failure point and prevent recurrence. Rerun the pipeline from the failed activity, then add retries for genuinely transient classes and an Activator alert so the next failure is heard immediately.

Keep the classification list in your head — the rest of this lesson is those five classes expressed per item type.

A pipeline fails at the activity level, so diagnosis always starts by opening the run (from the Monitoring hub or the pipeline's run history) and finding the failed activity's error code and message in its output. The run-level status only tells you that some activity on the executed path failed without a recovery path.

The common failure classes map cleanly:

• Authentication — the connection used by a source or sink has expired or lost permission. The fix is updating the connection's credentials, not editing the pipeline.
• Connectivity — the source is unreachable, or an on-premises source is used without a healthy data gateway. Verify the gateway status and network path.
• Schema and data — a Copy activity hits a column mapping mismatch or a value that will not convert to the destination type. Fix the mapping or the destination schema; for recurring dirty data, adjust the copy settings for handling incompatible rows.
• Resource and transient — throttling from the source, a timeout on a long copy, or a temporary service fault. These are what the activity's general settings exist for. Transient classes deserve retries; deterministic ones (auth, schema) do not — retrying a bad credential three times just fails three times.
• Logic — a dynamic expression evaluates to a wrong path or parameter, or an activity's dependencies route the run the wrong way. These are covered in the next section.

Timeout and retry work together

Every activity has a timeout (default 12 hours, maximum 7 days), a retry count (default 0) and a retry interval (default 30 seconds). A retry starts only after an attempt fails. An activity that hangs rather than fails therefore never triggers its retries until the timeout ends the attempt — with the default, that is 12 hours later. For a call that occasionally stalls, set a timeout just above its normal duration and a retry count, so a stalled attempt is cut off and retried quickly.

Rerun and drill-through

After fixing, use rerun from failed activity so completed upstream work is not repeated. Failures in an invoked child — a Notebook, Dataflow or Invoke pipeline activity — surface on the parent's activity with a summary message; the underlying error lives in the child's own run. For a notebook, follow the activity's link (or find the run in the Monitoring hub) to its Spark application detail page: it holds an item snapshot of the notebook exactly as it ran, including the code, cell outputs and the parameter values the pipeline passed, plus a Logs tab with the driver log (stdout/stderr) and the Spark UI link. For a dataflow, open that dataflow's refresh history and the failing table.

Pipeline logic errors come from two places: an expression that reads the wrong part of another activity's output, and dependency wiring that routes the run differently from what you intended. Both are diagnosed from the run's input and output JSON for each activity, which shows expressions after evaluation and the exact shape every activity produced.

Activity output shapes

An expression such as @activity('Name').output… fails, or evaluates to null, when its property path does not match what the activity actually emitted. The shapes to know:

So toggling First row only, or pointing a ForEach at the Get Metadata output object instead of its childItems array, breaks the expression even though nothing else changed. Open the upstream activity's output JSON in the run to confirm the exact path before editing the expression. (A reference to a misspelled activity name is caught by pipeline validation before the run starts.)

Dependency conditions and the Skipped status

Each dependency arrow between activities carries a condition:

• On success — runs the next activity only if the upstream activity succeeded.
• On fail — runs it only if the upstream activity failed.
• On completion — runs it whether the upstream activity succeeded or failed.
• On skip — runs it only if the upstream activity was Skipped.

An activity is Skipped when its dependency condition is not met — for example, everything after a failed step in an On success chain. When an activity has dependencies on several activities, all of them must be met (the conditions are ANDed); there is no built-in OR across different upstream activities.

The pipeline run's outcome is derived from its leaf activities (activities with nothing downstream). If any leaf fails, the run fails. A skipped leaf takes the outcome of its parent. That gives the two standard patterns their different results:

• Try-catch — a step with only an On fail handler. If the step fails and the handler succeeds, the handler is the leaf and the run succeeds.
• Do-if-else — a step with an On success path and an On fail path. If the step fails, the On fail path succeeds but the skipped On success path takes its failed parent's outcome, so the run fails — which is often what you want, because the failure stays visible.

One error handler for a sequential chain

To catch a failure anywhere in a chain A → B → C, connect the handler to the last activity (C) with both On fail and On skip. If C fails, On fail fires; if A or B failed, C is Skipped and On skip fires. Wiring On fail from every activity does not work, because those dependencies are ANDed and can never all be met. On completion is not an error path: it also fires on success.

A Dataflow Gen2 failure is diagnosed from its refresh history: open the failed refresh and find which table (query) and which activity failed, because a dataflow with many queries usually fails in exactly one of them while the rest succeed.

Classify by phase:

• Evaluation errors — the mashup engine could not execute the query: a source credential expired, an on-premises source has no reachable gateway, a source column referenced by a step was renamed or removed, or a step produced an error value. Reproduce by opening the query in the dataflow editor, where the failing step is flagged and you can inspect the error on the exact cell or column.
• Staging errors — dataflows can stage data in a staging Lakehouse before loading it; failures here are usually resource-shaped (very large intermediate results) or environmental. If a query does not need staging, disabling staging for it is a legitimate fix as well as an optimization.
• Load-to-destination errors — the write to the Lakehouse, Warehouse, or other destination failed: a value in the query does not convert to the destination column type, a schema changed so the fixed column mapping no longer matches, or the writer lacks permission on the destination. Fix the query's types, update the destination mapping, or grant the dataflow's destination connection write permission on the target.

Type conversion at the destination

When a column holds values that will not convert to the destination type (for example N/A in a column that loads into a decimal column), fix the data in the query: set the column's type in Power Query (Decimal number) and handle the conversion errors with Replace errors (for example with null) or by removing the error rows. The principle is to make the data fit the contract, not the contract fit the data: the destination column's type is what downstream consumers rely on, and a value only converts once the query itself produces a value of that type. Settings that change how rows move do not change whether a value converts.

On-premises gateway and staged data

With an on-premises data gateway, the gateway does more than read the source. Writing to a Lakehouse destination goes over HTTPS (port 443), but when staging is on, the gateway also reads the staged data back through the staging Lakehouse's SQL analytics endpoint over TCP 1433. If the gateway's network blocks outbound 1433 to *.datawarehouse.fabric.microsoft.com, *.datawarehouse.pbidedicated.windows.net and *.dfs.fabric.microsoft.com, the refresh fails in the WriteToDatabaseTableFrom activity with a socket error (10013) even though the individual tables report Succeeded. Open port 1433 to those endpoints, or, where that is not possible, restructure so the gateway does not need the staged read: combine the queries into one and disable staging and fast copy for it.

Two recurring exam patterns: first, a dataflow that works on demand but fails on schedule is usually an authentication or gateway problem tied to the scheduled context; second, when the error text mentions the destination table rather than the source, stop debugging the source steps — the query evaluated fine, and the types, mapping or destination are the problem.

A notebook fails with an exception in the failing cell's output, and that traceback is your first diagnostic. Read it bottom-up: the last frames name the real cause — an unresolved table or path, a column not found, a schema mismatch on write, a Python or PySpark API misuse.

The failures that need more than the cell output are the resource ones:

• Driver out-of-memory — the classic cause is pulling distributed data onto the driver: collect() on a large DataFrame, toPandas(), or building huge local objects. Apache Arrow speeds up the toPandas() conversion, but the mechanism is unchanged: every row still lands in driver memory. The fix is to stop materializing on the driver: aggregate or filter first, write results to a table, or sample. When the reason for pandas was custom Python logic, use pandas UDFs or mapInPandas, which run that logic in pandas on each executor's batch and keep it distributed.
• Executor out-of-memory or lost executors — typically skewed or oversized partitions, or an explosive join. Repartition, fix the skewed key, avoid unnecessary wide shuffles, or scale the Spark pool's node size if the workload legitimately needs it. The Spark UI's stage and task view shows the skew: a handful of tasks running far longer or spilling far more than their peers.

Distinguish surfaces the way the exam does: cell output for code and data errors, driver and executor logs for session-level and infrastructure failures, the Spark UI for where memory and time actually went. All three are reachable from the run's Spark application detail page in the Monitoring hub.

Notebooks that work interactively but fail in a pipeline or schedule usually fail for one of three reasons that have nothing to do with the code's logic: a library that is not available in the run, a parameter value that was not applied, or a Spark session that could not start.

Libraries: environments, not inline installs

A ModuleNotFoundError means the package is not in the session. Inline commands such as %pip install and %conda install install into the current session only, so the package disappears when the session ends, and inline installation is disabled by default in pipeline runs (it can be turned on with the _inlineInstallationEnabled parameter, but that reinstalls on every run). The recommended fix is to add the library to an environment (public or custom libraries), publish it, and attach the environment to the notebook or set it as the workspace default — every session, interactive or pipeline, then starts with the library. Library version conflicts surface when the environment is published or the session starts; read the publish log or the driver log.

Parameters: the injected cell

A pipeline's Notebook activity passes base parameters to a notebook whose parameters cell is toggled. At run time Fabric inserts a new cell directly beneath the parameters cell that assigns the passed values. Two consequences:

• Only variables whose names match exactly (case included) are overridden; a mismatched name is simply added as a new variable, and the notebook silently runs with its default.
• Any later cell that assigns the same variable overwrites the pipeline's value. Keep defaults only in the parameters cell.

Supported base parameter types are string, int, float and bool. The run's item snapshot shows the injected cell, so you can confirm what the notebook actually received.

Session start: capacity admission (HTTP 430)

Each capacity has a limit on Spark VCores (with bursting). When a new job needs cores that are not available, the response depends on the job type:

So a 430 is a capacity problem, not a code problem: stop idle interactive sessions, spread job schedules, reduce pool sizes, or scale the capacity. A pipeline notebook that starts late is usually queued, not failing — unless the queue itself is full.

Eventhouse (KQL database) errors are dominated by ingestion failures, and the diagnostic is a management command: .show ingestion failures returns the failure records retained by default for the last 14 days, including the database and table, the ErrorCode, the failure details, the IngestionSourcePath (which file or blob failed), whether the failure is Permanent or Transient, and OriginatesFromUpdatePolicy (whether an update policy caused it). Filter it with KQL like any table, for example by table name or time.

The failure kind drives the fix. Transient failures (such as General_ThrottledIngestion) are retried by the service and generally resolve. Permanent failures fail every retry until you change something, and the data does not arrive by itself — after fixing the cause you must re-ingest the failed files listed in IngestionSourcePath. The ErrorCode names the cause:

• Mapping errors — for example BadRequest_MappingReferenceWasNotFound: the ingestion properties name a mapping that was dropped or renamed. Re-create the mapping or reference an existing one.
• Format errors — the payload is not the declared format (declared JSON but actually CSV, compressed unexpectedly, or corrupt files).
• Access errors — for example Download_Forbidden: the service could not read the source blob. Fix the storage access, then re-ingest. Missing database permissions for the ingesting identity fail similarly.

Silent gaps: nulls without failures

Not every data problem is a failure. With direct ingestion through a table's JSON ingestion mapping, a field whose path no longer resolves (the producer renamed it) or a value that no longer converts to the column's type (for example 1,204 ms into a long column) is ingested as an empty (null) value. The row lands, nothing appears in .show ingestion failures, and the only symptom is a column that suddenly goes empty. Fix the mapping path, or clean the value upstream so it converts; there is no error to find.

Update policy failures

An update policy runs a query (often a stored function) over newly ingested source data and writes the result into a target table. Its IsTransactional setting decides what a failure does:

Find policy failures with .show ingestion failures | where OriginatesFromUpdatePolicy == true. A common cause: editing the function the policy calls (changing its output schema) is not validated against the policy, so the edit succeeds and the policy starts failing on the next ingestion.

Query-side errors are the remaining class: KQL semantic errors (unknown column, wrong operator usage) or queries that exceed limits on very large scans — resolved by correcting the query or narrowing its time range and filters rather than raising limits first.

An Eventstream fails at its edges — sources and destinations — and you diagnose it inside the Eventstream editor using data preview, data insights (per-node metrics), and runtime logs, since a continuously running stream has no run history in the Monitoring hub sense.

On the source side, the dominant class is connectivity and authentication: an external source's connection string, key, or credential is wrong or expired, or the network path to the source is blocked. The symptom is a source node showing errors in the runtime logs, or simply no incoming events in data insights. Verify by checking incoming event metrics: if nothing arrives at the first node, stop debugging transformations — the problem is between the source and Fabric.

For an Azure Event Hubs source, the stored connection holds one shared access policy name and one of its two keys. Regenerating that key, or deleting the policy, breaks the Eventstream source while producers (which use their own keys) keep sending happily — events pile up in the event hub but stop arriving in Fabric. Update the connection with the current key. The rule: only a change to the exact policy-and-key pair the connection stores breaks it; changes elsewhere on the namespace or event hub leave it working.

Transformation nodes fail on data shape: an operation references a field that stopped appearing in the events, or a type assumption breaks when a producer changes its payload. Data preview at each node shows exactly where the shape diverges from what the downstream operator expects.

On the destination side, the classic error is a schema mismatch with an Eventhouse destination: the events no longer match the target table's schema or mapping, so events flow through the stream but fail to land. Fix the mapping or table, exactly as with direct Eventhouse ingestion. Lakehouse destinations fail similarly on type conversion into the Delta table.

Also recognize the resource class: a stream whose incoming rate outruns its processing shows growing latency in data insights. That is a throughput problem — addressed by simplifying in-stream operations or scaling — not an error to hunt in logs.

T-SQL errors in a Fabric Warehouse or a Lakehouse SQL analytics endpoint fall into three classes you should name on sight:

• Permission and read-only errors — the caller lacks access to the object: the fix is granting the appropriate permission (GRANT SELECT and friends), workspace role, or item permission. Separately, the SQL analytics endpoint is read-only for table data, so any INSERT, UPDATE, DELETE or CREATE TABLE against it fails by design; writes to Lakehouse tables happen through Spark, pipelines, or dataflows. Views, functions and permissions can still be created there.
• Unsupported-surface errors — a statement, constraint or data type the Warehouse does not support. These need a rewrite, never a retry (see below).
• Transaction and conflict errors — see write-write conflicts below.

Unsupported surface: constraints and data types

Fabric Warehouse implements a subset of SQL Server's T-SQL. PRIMARY KEY and UNIQUE constraints are supported only as NONCLUSTERED and NOT ENFORCED, added with ALTER TABLE … ADD CONSTRAINT after the table exists, not inline in CREATE TABLE; foreign keys are likewise NOT ENFORCED. Several familiar column types are unsupported in tables, each with a standard replacement:

Write-write conflicts

Fabric Warehouse runs every transaction under snapshot isolation; a statement that sets another isolation level is ignored. Conflicts are detected at the table level: two transactions that modify the same table concurrently conflict even if they touch different rows. The first to commit wins; the other fails and rolls back with error 24556 or 24706. Because detection is per table and the isolation level is fixed, the fix works on timing: retry logic around the losing transaction, and scheduling or serializing writers to the same table.

A shortcut holds no data, so its errors are always about the target or the identity used to reach it. Which identity that is depends on the shortcut type:

Consequences worth knowing. Because an external shortcut reads with its connection's credential, granting individual users roles on the storage account does not fix a credential failure — update the connection. For passthrough shortcuts, fix the user's permission on the target item, not on the item that holds the shortcut. And the engine matters: a Lakehouse SQL analytics endpoint in delegated identity mode reads shortcut targets with the item owner's identity, while Spark passes the caller's. A user who lacks access to the target can therefore query a shortcut table through SQL and still get access denied reading it from a notebook; the fix is read access for the user at the target.

Path failures are the other class. If one shortcut reports not found while other shortcuts that use the same credential still read fine, authentication is not the problem — the target folder was moved, renamed or deleted in storage. Shortcuts do not follow such changes. Edit the shortcut's target path (or delete and re-create it with the same name, so code that references it keeps working).

Rehearse this table — it compresses the whole topic into the exam's question shape.

Scenario: a nightly pipeline runs a notebook that reads an ADLS Gen2 shortcut and writes a Lakehouse table; it has failed for two nights. The notebook activity shows a failure; drilling into the notebook run, the cell throws an access error on the shortcut path. Classify: authentication — and because it fails for every identity, it is the shortcut's external connection, not a user permission. The storage key used by the connection was rotated. Fix the connection's credential, rerun the pipeline from the failed activity, and add an Activator rule on failed job events so the next credential rotation is caught the same night.