Data Privacy and Governance: Macie, Redshift Sharing, Config, and Sovereignty

Amazon Macie is the AWS service that discovers sensitive data in Amazon S3 — it does not protect, mask, move or block anything itself; it tells you where the risk is. Macie detects data with managed data identifiers, built-in detectors for common categories such as names, addresses, credit card and passport numbers, national ID numbers and credentials, and with custom data identifiers that you define for organization-specific values such as internal staff IDs or contract numbers. Macie inspects S3 objects only; it does not scan Amazon RDS, DynamoDB or other data stores directly.

Tuning custom data identifiers

A custom data identifier is a regular expression plus optional refinements, and each refinement solves a different false-positive problem:

Custom data identifiers cannot be edited after creation (this keeps the history of findings immutable for audits), and a discovery job's settings cannot be changed once created either. To change detection criteria you create a new identifier — test it first with the console's Evaluate option or the TestCustomDataIdentifier API — and use it in new jobs, or copy an existing job and select the new identifier.

Discovery jobs versus automated discovery

Macie finds sensitive data in two ways, and the choice turns on coverage versus cost:

The two combine: keep automated discovery running for the estate and add a one-time job at 100% sampling depth when a regulator or merger needs evidence that every object in one bucket was checked. Separately from content scanning, Macie keeps an S3 bucket inventory and raises policy findings when a bucket's settings change to public access, to shared or replicated with accounts outside your Macie organization, or to default encryption reverted to SSE-S3 — these assess bucket configuration, never object contents.

Macie is organization-aware: a delegated administrator account manages it for all member accounts. Findings are published to Amazon EventBridge and AWS Security Hub CSPM, and EventBridge is the automation hook — a finding can trigger a Lambda function that tags data, moves an object to a quarantine bucket or opens a ticket. Any requirement to discover, classify or inventory PII in S3 resolves to Macie (not GuardDuty, which detects threats; Inspector, which finds software vulnerabilities; or Comprehend, which detects PII in text you send it and has no integration with Macie findings); requirements to restrict or mask the PII you found resolve to Lake Formation or masking features.

Macie identifies the PII and AWS Lake Formation enforces who can read it — the exam guide names this pairing explicitly. The pattern is a pipeline: Macie finds sensitive data in data lake objects; an EventBridge rule on the finding runs automation that records it as governance metadata, typically a Lake Formation LF-tag such as a classification key with values like public and pii; and Lake Formation tag-based access control (LF-TBAC) grants query access by tag instead of table by table.

How LF-tags make this scale

• Inheritance: a tag assigned to a database is inherited by its tables, and a table's tags by its columns. A column can override the inherited value with a different value of the same key.
• Expressions match values, they cannot negate: in a grant expression, different keys are ANDed and multiple values of one key are ORed, but there is no "not pii". So you cannot grant on "everything except pii".
• The working pattern: tag each database classification=public so every new table and column inherits it, grant analysts SELECT on classification=public, and have the Macie-driven automation override only the sensitive columns to classification=pii. New tables are covered automatically, and an overridden column silently drops out of the analysts' results — no per-table grants or per-table data filters to maintain.

Lake Formation also supports direct column-level grants (SELECT on a table with named columns excluded) and data filters for row-level and cell-level security; these suit a handful of tables, while LF-tags suit a lake that grows weekly.

Conditions for the controls to actually apply

• Athena (and Redshift Spectrum, EMR) enforce Lake Formation column permissions only when the table's S3 location is registered with Lake Formation and the default IAMAllowedPrincipals grant has been removed, so access is no longer decided by IAM alone.
• An Athena view that omits columns is not a security boundary if the same role can still query the base table.
• IAM or S3 object-level denies are all-or-nothing per file: every file holds all of a row's columns, so they cannot hide individual columns.

A concrete loop: a healthcare platform lands files daily; automated discovery flags patient identifiers in a new table; EventBridge runs a Lambda that overrides those columns' tag to pii; from then on the general analytics role's Athena queries no longer return those columns, while a clinical-research role granted on pii keeps access — nothing was copied or manually re-permissioned. Detect, tag, enforce is the answer shape to look for.

Amazon Redshift data sharing gives other Redshift warehouses live access to your data without copying or moving it — consumers always query current, transactionally consistent data, and no pipeline exists to lag. Shared objects are read-only to consumers unless the producer explicitly grants write privileges on them. A producer creates a datashare, adds objects (schemas, tables, views, user-defined functions) and grants it to a consumer; the consumer creates a database from the datashare and queries it with ordinary SQL.

Consumers use, and pay for, their own compute. The consumer is charged for the compute (and any cross-Region transfer) its queries use; the producer pays for storage. That is the isolation argument: a heavy analytics team cannot drain the producer's capacity. Options that run the consumer's queries on the producer — concurrency scaling on the producer cluster, a separate WLM queue on it, or a cross-account role calling the producer through the Redshift Data API — all consume and bill producer capacity. Federated query is a different feature: it reaches Amazon RDS and Aurora PostgreSQL/MySQL databases, not another Redshift warehouse.

Prerequisites

• Node type: both sides must be RA3 provisioned clusters or Redshift Serverless. DC2 and other older node types can be neither producer nor consumer — the fix is moving to RA3 or Serverless, not resizing, enhanced VPC routing or concurrency scaling.
• Encryption: in the same account, producer and consumer must have the same encryption type (both encrypted or both unencrypted); across accounts, across Regions and for Lake Formation-managed datashares, both must be encrypted. They do not need to share a key, and no KMS key-policy change is involved.

The workflow

GRANT USAGE ON DATASHARE is always the producer's statement; consumers grant permissions on the database they created, not on the datashare. Datashares also work across Regions (with transfer charges), and AWS Data Exchange datashares extend the mechanism to licensing data to external subscribers, with AWS Data Exchange managing entitlements and billing.

Choose datashares whenever another team, account or Region needs to query warehouse data without copies, ETL or staleness. Wrong-answer shapes are scheduled UNLOAD/COPY, nightly snapshot restores, and exports queried with Spectrum (all copies), or handing out producer credentials (no isolation). If consumers are not Redshift at all, a datashare is the wrong tool — it is warehouse-to-warehouse.

Preventing data from reaching disallowed AWS Regions is a preventive-control problem, and the strongest tool is a service control policy (SCP) in AWS Organizations. An SCP attached to the organization root (or the relevant OUs) that denies requests whose aws:RequestedRegion is outside an approved list applies to every principal in every member account — including account administrators — and to accounts added later. Exempt the global services that operate through a single Region endpoint (IAM, Organizations, Route 53, CloudFront, Support and similar) so identity management keeps working. AWS Control Tower packages the same guardrail as its Region deny control.

Know the scope limits:

• SCPs never restrict the organization's management account, so the guardrail covers member accounts only — keep workloads out of the management account.
• Alternatives are weaker: a permissions boundary must be attached role by role and administrators can change it; Config rules and EventBridge alerts on CloudTrail events detect after the fact rather than prevent.

Side doors a Region-deny SCP does not close

Several features copy data to another Region while every API call is made in an allowed Region. S3 Cross-Region Replication is the classic case: the replication rule is created by s3:PutReplicationConfiguration on the source bucket, in the allowed Region, so aws:RequestedRegion passes; and if the destination bucket belongs to an account outside your organization, nothing in your SCPs applies to it at all. Close it by denying s3:PutReplicationConfiguration (except for an approved platform role) in the SCP. The same reasoning covers DynamoDB global table replicas (deny the table-update actions that add replicas), Redshift cross-Region snapshot copy, AWS Backup cross-Region copy jobs, and AMI or EBS snapshot copies — each legitimate for disaster recovery and each a sovereignty leak if ungoverned. Blocking s3:CreateBucket in other Regions does not help when the destination bucket already exists in another account, and S3 Block Public Access solves a different problem (public exposure, not location).

Detection backs up prevention: AWS Config (next section) can reveal any resource that nonetheless exists in an unexpected Region — but only if Config records in every Region, because a Region where the recorder is off reports nothing. An organization aggregator then shows those resources centrally. Macie policy findings flag buckets shared or replicated with accounts outside the Macie organization (the accounts it manages together through AWS Organizations or invitation). The layers: SCP = preventive; Config = detective.

AWS Config answers "what did this resource look like, how did it change, and is it compliant": it records the configuration of supported resources as configuration items and keeps a resource timeline, so you can open an S3 bucket, security group or Redshift cluster and see its exact settings at any past point and the before-and-after values of each change. CloudTrail is the other half: it records who made the API call.

Rules and automatic remediation

Config rules evaluate resources on change or periodically. Managed rules cover common cases — for example a rule that checks S3 buckets use SSE-KMS default encryption (with an optional list of allowed KMS keys), public-access checks, encrypted volumes, required tags — and custom rules (Lambda or Guard) express anything else. Note that since January 2023 every S3 bucket has SSE-S3 default encryption, so the realistic drift is a bucket moved from SSE-KMS with your key back to SSE-S3, not an unencrypted bucket.

Remediation options differ in how much code you own:

• Config rule + automatic remediation with an AWS-owned Systems Manager Automation runbook — corrects drift within minutes, no custom code; you supply parameters such as the KMS key.
• EventBridge rule on the CloudTrail event + a Lambda function — works, but it is custom code to write and maintain.
• Config rule + SNS notification — detection only; a person still fixes it.
• An SCP denying the change — preventive, but it also blocks legitimate changes and does not correct existing drift.

Organization scale

Conformance packs bundle rules and remediation into a deployable baseline that changes what each account evaluates. An aggregator does not evaluate anything; it collects configuration and compliance data from many accounts and Regions (or the whole organization) into one view in a delegated administrator account. One consolidated view without changing per-account evaluation is an aggregator; pushing a standard set of rules is a conformance pack.

For a data platform, Config is how you prove the posture you claim: that data buckets have kept SSE-KMS and public access blocks, that no Redshift cluster was ever publicly accessible, that a widened security group was remediated within minutes. Trusted Advisor checks AWS best practices, not your own policies or history.

Data sovereignty means data stays subject to the laws of the jurisdiction where it is stored — and often must physically stay there. The foundational AWS property is that customer data in a Region does not leave it unless you move it: S3, Redshift, DynamoDB (without global tables) and EBS store data regionally. Sovereignty engineering is therefore mostly the discipline of not moving it — the Region-deny SCP and the replication controls above are the enforcement half.

Storage residency is not key residency. Keeping encryption keys in-country — single-Region KMS keys in an approved Region, or an external key store whose keys sit in an in-country HSM — controls who can decrypt, which is valuable, but it does not satisfy a mandate that the data itself be stored and processed in the country. Ciphertext stored in a foreign Region is still data stored abroad. Likewise, creating every KMS key in one Region does not stop anyone storing data elsewhere (it can be encrypted with another key or not use KMS at all). Multi-Region keys exist to make ciphertext portable across Regions, the opposite of what most sovereignty mandates want.

Other near-misses: Amazon CloudFront geo restriction controls which countries can receive content, not where data is stored; and a disaster-recovery replica in a foreign Region breaks a residency mandate however it is encrypted — recovery copies must go to another approved, in-jurisdiction Region. When users abroad need the data, the residency-safe answer is to serve their queries against the single in-country store (accepting the latency) rather than replicate it outward to be near them.

The full posture for "customer data must stay in country X": provision only in the approved Regions; deny every other Region with an organization SCP plus the cross-Region copy and replication actions; encrypt with customer managed keys in those Regions; record AWS Config in all Regions with an organization aggregator to surface anything outside them; and keep the evidence trail in CloudTrail and Config history, including CloudTrail's record of every KMS key use. That stacks a preventive control (SCP), a data-layer control (regional keys) and a detective control (Config) rather than relying on any single mechanism.

Amazon SageMaker Catalog is the business data catalog inside Amazon SageMaker Unified Studio, and it manages data access through a publish-and-subscribe model built on projects. Producers publish data assets — Glue Data Catalog tables, Redshift tables — into the catalog with business metadata and glossary terms so consumers can find and understand them. Consumers work inside projects: governed workspaces that bundle team members, tools and the data the project has been granted. When a project needs a published asset, a project member requests a subscription; the publishing project (the data owner) approves or rejects it; and on approval the underlying grants (Lake Formation permissions for lake tables, Redshift access for warehouse tables) are fulfilled automatically to the subscribing project.

The governance value is the workflow itself: access is requested against a curated catalog, approved by the owner accountable for the data, scoped to a project rather than an individual, and recorded. The anti-patterns are the shortcuts — hand-written bucket policies for another team's roles, copying the data into the consumer's own storage, or adding consumers as members of the owner's project (which gives them the owner's whole workspace rather than one approved asset). Domains and domain units mirror the organization's structure, so approval authority sits with the business units that steward each data area.

Differentiate the catalogs: the AWS Glue Data Catalog is the technical metastore (schemas, partitions, locations) that query engines depend on; SageMaker Catalog is the business layer on top — discovery, semantics, ownership and the subscription workflow. Crawlers, partitions or Hive-compatible metadata mean Glue; publishing assets, approval workflows, projects or self-service requests mean SageMaker Catalog.

A data governance framework is the organizational system that keeps data trustworthy and compliant: policies defining classification levels and handling rules; named roles — data owners accountable for each domain and data stewards maintaining quality and metadata; cataloging so data is findable with its meaning and sensitivity attached; lineage so any dataset can be traced to its sources and transformations; quality controls; and auditability. The AWS services in this lesson are the mechanical enforcement of that framework — Macie implements classification, SageMaker Catalog implements cataloging and stewardship workflows, Lake Formation implements access policy, and Config and CloudTrail implement auditability.

Data sharing patterns describe how data moves between producers and consumers. The first question is whether consumers query the producer's data in place or receive a copy.

Signal words: "one team governs all data" points at the centralized lake; "domains own and serve their data as products, discovered in one catalog under common standards" is the mesh; "zero-copy, always current, warehouse to warehouse" is a datashare; "license or monetize datasets to external organizations" is AWS Data Exchange. When a mandate forbids copies in consumer accounts, eliminate zero-ETL, snapshot sharing and replication even though they are convenient — they all create data in the consumer's account.