Detecting Threats on the Microsoft Sentinel Platform for SC-200: Hunting Queries, Bookmarks, Hunts, KQL Jobs, Summary Rules, Notebooks and the MCP Server

You run and monitor hunting queries on the Microsoft Sentinel Hunting page (in the Defender portal: Microsoft Sentinel > Threat management > Hunting), where you select Run all queries or run a selection and then read each query's result counts and how they changed. The Queries tab lists the hunting queries installed with Content hub solutions alongside your own custom queries, written in KQL and run mainly against the workspace's analytics-tier tables.

The columns after a run.

The delta windows are fixed: they always compare the last 24 hours with the 24 to 48 hours before, not this run with your previous manual run and not week on week. Sort by Results delta to find the biggest absolute jumps, and by Results delta percentage to find the queries that rose the most relative to their own normal volume; noisy queries dominate the absolute delta and the raw Results column.

Favorites run themselves. Queries you mark as favorites run automatically each time you open the Hunting page, so their counts are current on arrival without selecting and running them by hand.

The MITRE ATT&CK tactic bar at the top of the Queries tab counts queries per tactic using each query's MITRE mapping. It updates dynamically with the current filters: hide N/A and zero-result queries and the bar shows which tactics your active queries cover, which makes it a quick coverage check rather than a static inventory.

You create a custom hunting query with New query on the Hunting page's Queries tab, giving it KQL, a description, entity mappings and MITRE ATT&CK tactics and techniques; you change a built-in query by cloning it, because only queries whose content source is custom can be edited in place.

What to fill in on a new query.

• Query — the KQL, against the workspace's analytics-tier tables.
• Entity mappings — entity type (Account, Host, IP and so on), identifier and the result column that holds it. Bookmarks created from the query's results inherit these mappings, so no one has to map entities bookmark by bookmark.
• MITRE ATT&CK — tactic, technique and sub-technique. These drive the tactic bar's coverage counts, and bookmarks inherit them too.

Once saved, the query sits beside the Content hub queries: it is included in Run all queries and gets Results, Results delta and Results delta percentage values. Only a query saved on the Hunting page's Queries tab takes part in these runs and metrics; KQL kept anywhere else is just stored text.

Editing depends on where the query came from.

The content source alone decides whether a query is editable in place; no other setting on the query changes that.

A hunting bookmark saves specific result rows together with the query that produced them, so a finding survives after the query window moves on and can be investigated, grouped and escalated to an incident. You select rows in a query's results on the Sentinel Hunting page and choose Add bookmark.

Where bookmarks work. Bookmarks belong to the Microsoft Sentinel hunting experience. The unified advanced hunting page in the Defender portal does not support them, so to preserve rows as bookmarks a hunter runs the query on the Sentinel Hunting page.

What a bookmark carries.

• The row results, the query and its time range.
• Entity and MITRE mappings, defaulted from the query's own mappings (you can adjust them).
• Tags and notes. Tag every bookmark from one campaign with the same tag and filter the Bookmarks tab on it; new bookmarks join the view as soon as they are tagged.

Bookmarks as data. Every bookmark is written to the HuntingBookmark table in the workspace, so you can filter, summarize and join everything the team has preserved with other sources such as SigninLogs to look for corroborating evidence.

Acting on a bookmark.

• Investigate (in the bookmark's details pane) opens the investigation graph and timeline, connecting that single finding to the users, hosts and IP addresses involved.
• Incident actions on the Bookmarks tab offer Create new incident or Add to existing incident for the selected bookmarks.

Bookmarks are evidence, not triggers: they do not rerun a query or alert on future matches.

A hunt (preview) is a single Microsoft Sentinel object, created on the Hunting page's Hunts tab, that records a hypothesis and its outcome together with the hunt's own query copies, bookmarks, entities, comments and the incidents and analytics rules it produced. It is the tool for running and tracking a proactive investigation end to end.

What a hunt holds.

• Hypothesis and status. You name the hunt, describe the hypothesis, and set its hypothesis state (for example Validated or Invalidated) and status as the work progresses. These record progress and feed the metrics; they do not create rules or export anything by themselves.
• Queries — the queries you pick are cloned into the hunt. The clones are independent of the workspace's Queries tab and of other hunts, so narrowing a query inside a hunt changes nothing for colleagues running the original.
• Bookmarks, entities and comments — evidence collects inside the hunt; the Entities tab is built from the entities in the hunt's bookmarks.

Turning findings into outcomes.

The Hunts tab's metrics bar counts validated hypotheses, new incidents and new analytics rules, which gives a SOC lead a measure of what hunting produced. Other tools can each hold part of this record, but only a hunt keeps hypothesis, cloned queries, evidence and outcomes in one object.

Livestream is no longer available in Microsoft Sentinel; to keep a hunting query working after the hunt, Microsoft points you to KQL jobs, analytics rules and playbooks, each of which covers a different need.

Promoting a high-value query. After a compromise, Microsoft's guidance is to turn hunting queries that give high-value insight into detection rules, so the insight surfaces as alerts. From the query's results choose New alert rule > Create Microsoft Sentinel alert; the analytics rule wizard opens with the KQL prepopulated. In a hunt, use Create analytics rule on the hunt's Queries tab instead.

Requirements often combine these needs, and then each need gets its own tool from the table; nothing on the Hunting page itself schedules a query. Analytics rules read the analytics tier; data that exists only in the data lake tier is promoted first (next section). Writing analytics rules in depth belongs to the detections lesson.

A KQL job is a one-time or scheduled KQL query that runs against Microsoft Sentinel data lake tables (and federated tables) and writes its results to a table, either promoting them to the analytics tier or keeping them in the lake tier. You create jobs in the Defender portal under Data lake exploration > Jobs (Create job, or Create from template).

Where the output goes, and what it's called.

For comparison, notebook output uses _SPRK (lake) and _SPRK_CL (analytics), and summary rules write plain _CL tables. Choose the System tables destination when only lake tools will read the data and storage cost matters.

New or existing table. A job writes to one output table. It can create a new table or append to an existing one; when it appends, the query's output schema must match the destination table's schema, or the job fails at run time. Adding a column with extend is enough to break it: project back to the existing columns, add the column to the table, or write to a new table.

What a job can read. Lake data going back up to 12 years; several source workspaces at once (same-named tables are unioned by default, or use workspace()); and federated tables such as Microsoft Entra ID, Microsoft 365 and Microsoft Resource Graph, which you query without ingesting them into a workspace.

Keep promotion cheap. Analytics-tier ingestion and storage cost more, so filter to the rows you need (for example indicator matches), summarize before writing when only counts are needed, and project only the columns you need. Promoting a filtered result is far cheaper than moving a whole table to the Analytics tier with long retention.

TimeGenerated is rewritten for old rows. When a job writes rows whose TimeGenerated is older than two days, ingestion overwrites it with the ingestion time. Copy the original timestamp into its own column (for example | extend EventTime = TimeGenerated) and chart on that, or a six-month attack timeline collapses into the day the job ran.

Permissions. Users need a supported Microsoft Entra ID role to create and schedule jobs (covered with Sentinel roles in the platform lesson). Creating a new analytics-tier table is done by the data lake's system-assigned managed identity (named msg-resources-<guid>), which needs Log Analytics Contributor on the destination Log Analytics workspace. If appends succeed but new-table jobs fail, that assignment is the missing piece.

Templates. Create from template opens the wizard with the template's query, destination table, schedule and description filled in; you only select the destination workspace.

A KQL job runs either once or on a schedule (by the minute, hourly, daily, weekly or monthly), and its first run must start at least 30 minutes after you create or edit it.

Scheduling rules.

• The wizard rejects a start time less than 30 minutes away; the same rule applies after an edit.
• Editing a one-time job runs it again immediately, even if you changed only the description. If the job appends to an existing table, the rerun writes every row a second time.
• Ingestion latency. New rows typically take up to about 15 minutes to become queryable in the lake. A job that reads up to now() keeps missing the late tail. Set the window's end to now() minus a delay and its start to the end minus the lookback; to write each event exactly once, make the lookback equal to the run interval. Overlapping windows catch late data but create duplicates.

// Hourly job: delay 15 minutes, lookback 1 hour
CommonSecurityLog
| where TimeGenerated between (ago(75m) .. ago(15m))
| where DeviceAction == "Deny"

Service limits (per tenant).

Unsupported in jobs: adx(), arg(), externaldata(), ingestion_time() and user-defined functions. Every other KQL operator and function works, including joins across tables and workspaces. To use a list from a CSV, load it into a table first; to use Resource Graph, query its federated table instead of arg().

Jobs are disabled automatically when one of their source workspaces is deleted.

A summary rule runs a KQL query (usually with summarize) against a source table on a fixed bin schedule and writes the aggregated results to a custom _CL table on the Analytics plan, so dashboards and rules read a small, fast table while the verbose raw data stays in a low-cost tier. It is the documented pattern for high-volume, low-security-value logs such as web proxy or network flow data.

Design.

• Bin size (binSize): 20, 30, 60, 120, 180, 360, 720 or 1,440 minutes. A rule processes only recent incoming data, at most 24 hours back; it has no historical time range and cannot produce weekly rollups.
• Source plan decides the operators. Over an Analytics source you can use almost the full query language, except cross-resource and cross-service queries and the plugins that reshape the schema (bag_unpack, narrow, pivot). Over a Basic source, all operators work on the single source table, and you can enrich with the lookup operator against up to five Analytics tables (for example a threat intelligence table); join is not the way. Auxiliary and data lake sources differ mainly in cost (below). User-defined functions aren't supported.
• One workspace. A rule runs inside one Log Analytics workspace, writes to a table in that workspace and doesn't support cross-resource queries (including ARG() cross-service queries).
• Templates. In Content hub, filter Content type by Summary rules and install a template; then on the Summary rules page's Templates tab select it and Create, and adjust the prepopulated wizard before saving.
• Up to 100 active rules per workspace. Output rows carry _RuleName, _BinStartTime and _BinSize.

Cost. Summary rules have no charge of their own. You pay for (1) the query: free over an Analytics source, a data scan charge over a Basic, Auxiliary or lake source; and (2) ingestion of the results, which always uses the Analytics plan.

Operations.

Use a summary rule for continuous aggregation of new data inside one workspace on bins of up to a day; use a KQL job for anything historical, one-off, cross-workspace, federated or on a weekly or monthly cadence. Both can write to the analytics tier, but they differ on time range and scope.

Many requirements need both: a one-time KQL job for the historical search and a summary rule for the ongoing daily summary. Moving a whole table to the Analytics tier satisfies most needs too, at the highest cost. Notebook jobs can also aggregate, but they need Python code and a Spark pool, so they suit analysis that KQL can't express rather than plain rollups. arg() works in neither tool; Resource Graph data reaches a KQL job through its federated table.

Data lake notebooks are Jupyter notebooks that run Python and Apache Spark against Microsoft Sentinel data lake tables from Visual Studio Code with the Microsoft Sentinel extension; the Sentinel MCP server lets AI agents such as GitHub Copilot query the same security data in natural language. Both suit hunting that KQL alone handles poorly: data frames, statistics, charts or agent-driven exploration.

Notebooks. After signing in through the extension you choose a Microsoft Sentinel runtime pool (Small, Medium or Large) and work through the MicrosoftSentinelProvider class:

from sentinel_lake.providers import MicrosoftSentinelProvider
data_provider = MicrosoftSentinelProvider(spark)
data_provider.list_databases()
df = data_provider.read_table("DeviceProcessEvents", "SOC-Workspace")

• read_table(table, database_name, database_id): database_name is the workspace and defaults to System tables, so a workspace table needs its workspace named; add database_id only when workspace names aren't unique.
• save_as_table writes to the lake tier when the name ends _SPRK and to the analytics tier when it ends _SPRK_CL. Append is the default; the analytics tier supports only append, and overwrite works only in the lake tier.
• Libraries: only the Azure Synapse runtime libraries (pandas, PySpark and so on) and the Microsoft Sentinel provider. %pip install and custom libraries aren't supported, whatever the pool size.

Notebook jobs run a notebook on demand or on a schedule (by the minute, hourly, daily, weekly or monthly) without anyone opening VS Code. Create them from the Sentinel extension in VS Code.

• Submitting uploads the notebook and configuration to the data lake; the job no longer depends on your local file. To change the code, download the job definition or latest notebook, edit it, choose Edit job and Submit. Renaming a job changes only its label.
• The Defender portal's Jobs page lists notebook jobs and can enable or disable them, but can't edit them.
• Parameters have three levels: notebook values (defaults in the parameter cell), job configuration values (saved with the job and used by scheduled runs) and runtime values (entered for Run now, applying to that run only). To change scheduled runs, edit the job configuration value and resubmit.
• Concurrency: three notebook jobs run at once and further jobs are queued — unlike KQL jobs, which are rejected beyond five.

The Sentinel MCP server is a hosted service authenticated with Microsoft Entra ID: nothing to deploy and no app registration or secret. It offers scenario-focused tool collections:

• Connect in VS Code: Ctrl+Shift+P > MCP: Add Server > HTTP, enter the collection URL, name the server, choose its scope and sign in. In Copilot Chat agent mode the tools appear under Configure Tools.
• Roles: at least Security Reader (Security Operator or Security Administrator also work). The entity analyzer tools additionally need Security Copilot Contributor, because they consume security compute units (SCUs); Security Copilot Owner is only for monitoring usage.
• Call order: the data exploration tools need a workspace ID to target the right workspace, so an agent calls list_sentinel_workspaces first, then search_tables to find tables and schemas, then query_lake to run KQL.
• Entity analyzer limits: analyze_user_entity covers at most a seven-day window and works only for users with a Microsoft Entra object ID.