Applying Authorization Mechanisms: Lake Formation, IAM, and Redshift

AWS gives data platforms two complementary authorization systems, and the first exam skill is knowing which one answers a given question. IAM authorizes API actions: can this principal call s3:GetObject on this bucket, glue:GetTable on this catalog resource, athena:StartQueryExecution at all. AWS Lake Formation authorizes data access: can this principal SELECT from this table, and if so, which columns and rows. Lake Formation sits on top of the AWS Glue Data Catalog and vends temporary, scoped-down credentials to integrated engines, so a single grant is enforced identically whether the query comes from Athena, Redshift Spectrum, or EMR.

The two layers cooperate rather than compete. A principal querying a governed table still needs IAM permission to call the query engine (for example Athena and Glue APIs), but it does not need direct S3 permissions on registered lake locations — Lake Formation supplies short-lived credentials for exactly the data the grant allows. That credential-vending model is why the exam favors Lake Formation for fine-grained, multi-team lake access: without it you would be maintaining brittle S3 bucket policies that cannot express column- or row-level rules at all. Two caveats decide many scenarios, and both are covered below: with default settings Lake Formation grants are not enforced at all (IAM still decides), and Lake Formation never intercepts a direct S3 API call that an IAM policy allows.

AWS managed policies get you started, but they are deliberately generic — AmazonS3ReadOnlyAccess grants read access to every bucket in the account, and the exam expects you to recognize that as a least-privilege failure. You write a custom (customer managed) policy whenever a managed policy is too broad, too narrow, or missing a condition your business requires. Customer managed policies are reusable across principals, versioned (you can roll back), and yours to scope precisely; inline policies embed in a single principal and are best reserved for one-off exceptions you never want reused.

A policy statement has four load-bearing parts: Effect (Allow or Deny), Action (the API calls), Resource (the ARNs), and optional Condition blocks. Evaluation logic is a favorite exam trap: everything is implicitly denied by default, any applicable Allow grants access, and an explicit Deny anywhere always wins — across identity policies, resource policies, permissions boundaries, and service control policies. Explicit Deny is therefore your guardrail tool: for example, deny s3:PutObject unless the request includes the correct encryption header, or deny all actions when aws:SourceVpce does not match your VPC endpoint.

How the policies combine: same account versus cross-account

Whether a request succeeds depends on which account owns the resource. Within one account, an Allow in either the caller's identity policy or the resource's policy (such as a bucket policy) is enough — for S3 and most services with resource policies. The exceptions are AWS KMS keys and IAM role trust policies: there the key policy or trust policy itself must allow the access, so an identity-policy Allow alone is not enough. Across accounts, the request needs an Allow on both sides: the caller's identity policy in its own account and the resource policy in the owner's account. Service control policies (SCPs) and permissions boundaries are filters, not grants — they can only remove permissions, so neither one ever makes a request succeed on its own.

The last row is the one that surprises people. Suppose a pipeline role's identity policy allows s3:* on every bucket, and a bucket in the same account carries a statement that denies all S3 actions to every principal when aws:SecureTransport is false. A call made over plain HTTP matches that condition, so the Deny applies and the call fails — the broad identity Allow does not matter, because policies are not evaluated in an order where an earlier Allow wins. The same call over HTTPS does not match the Deny and succeeds on the identity Allow. Any condition key behaves this way (aws:SourceVpce, aws:PrincipalOrgID, aws:SourceIp): work out whether the request matches the Deny's condition first, and only then look for an Allow. Cross-account S3 reads with bucket owner enforced Object Ownership (ACLs disabled) follow the same rule: the bucket policy must allow the other account's role, and that role's identity policy must allow the S3 actions; ACLs play no part.

A concrete data-engineering example: a Glue job must read raw data from one prefix and write curated Parquet to another. The custom policy allows s3:GetObject on the raw prefix ARN, s3:PutObject on the curated prefix ARN, and s3:ListBucket on the bucket with an s3:prefix condition — nothing account-wide, no s3 wildcard actions. Remember that object actions take object-level ARNs with a path, while ListBucket takes the bucket ARN; mixing those up is a classic reason a scoped policy mysteriously fails.

Least privilege means every principal holds exactly the permissions its job requires — no wildcard actions, no account-wide resources, no permanent grants for temporary needs. In practice you construct it iteratively: start from what the workload actually calls, scope Resource to specific ARNs and prefixes, then tighten with conditions. Useful condition keys for data work include aws:SourceVpce and aws:SourceVpc (require the private path), aws:PrincipalTag and aws:ResourceTag (attribute-based control), s3:prefix (limit listing), and aws:SecureTransport (refuse unencrypted connections).

Scoping S3 permissions to a prefix

S3 actions split by resource type, and a least-privilege policy has to respect the split. Object actions such as s3:GetObject and s3:PutObject act on object ARNs, so they are scoped to a path like arn:aws:s3:::lake-bucket/raw/sales/*. s3:ListBucket acts on the bucket ARN (arn:aws:s3:::lake-bucket, no path), so you narrow it with a Condition on the s3:prefix key, for example StringLike s3:prefix raw/sales/*. Putting ListBucket on the object path, or GetObject on the bare bucket ARN, matches nothing and the call is denied.

AWS gives you tooling so least privilege is not guesswork. IAM Access Analyzer can generate a policy from the CloudTrail activity of a role — run the pipeline for a while, then generate a policy that permits only what it actually did. Its unused access findings report roles, access keys and permissions that have gone unused over a tracking period, which tells you what to remove; policy generation tells you what to keep. Do not confuse these with external access findings, which answer a different question — which resources are shared with principals outside your zone of trust — and say nothing about whether a role uses its permissions. Access Analyzer also validates policies against best practices, and the last accessed information on roles shows which granted services have never been called. The IAM policy simulator, by contrast, only tests what a policy text allows; it has no usage evidence.

Permissions boundaries for delegated role creation

A permissions boundary is a managed policy set as the ceiling on a user's or role's permissions: the effective permissions are the overlap of the identity policy and the boundary. Its main use is safe delegation. When a platform team lets data teams create their own pipeline roles, it allows iam:CreateRole (and iam:PutRolePolicy, iam:AttachRolePolicy) only with a Condition that iam:PermissionsBoundary equals the ARN of the approved boundary policy, and denies removing or changing that boundary. Every role the data team creates is then capped, whatever policy they attach to it. The alternatives do not fit this goal: an SCP applies to every principal in the account, not just the roles a team creates, and a session policy limits only the session of whoever passes it — it places no ceiling on the roles that person creates.

Two habits close the loop. First, prefer temporary elevation over standing privilege: an engineer who occasionally needs to delete lake objects should assume a scoped role for that task, not carry the permission daily. Second, remember iam:PassRole: any policy that lets a user hand a role to Glue, Lambda, or CloudFormation must restrict which roles can be passed, or the user can escalate by passing an administrator role to a service they control. Exam options that casually include iam:PassRole on Resource star are showing you the wrong answer.

Authorization extends to the credentials your applications use, and skill 4.2.2 names the two stores you must be able to choose between. AWS Secrets Manager is purpose-built for secrets: encryption with AWS KMS is mandatory, automatic rotation is built in (managed rotation for RDS, Aurora, Redshift, and DocumentDB, or a custom rotation Lambda), secrets can be replicated to other Regions, and resource policies on individual secrets enable clean cross-account sharing. AWS Systems Manager Parameter Store is a general configuration store: String and StringList parameters for plain settings, SecureString parameters (KMS-encrypted) for sensitive ones, organized in hierarchical paths, with a standard tier that costs nothing.

The exam decision rule is short: rotation or cross-account secrets means Secrets Manager; cheap configuration means Parameter Store. Authorization applies identically to both — grant secretsmanager:GetSecretValue or ssm:GetParameter on specific ARNs only, and use path-based ARNs (for example a /prod/etl/ parameter hierarchy) so one policy statement cleanly covers a team's parameters without exposing anyone else's. Both services log retrievals to CloudTrail, which is what turns credential access into something you can audit.

Sharing a secret with another account

Cross-account secret access needs three Allows to line up. The secret's resource policy in the owning account must allow the other account's role secretsmanager:GetSecretValue; the role's identity policy in its own account must allow GetSecretValue on the secret ARN and kms:Decrypt on the key; and the secret must be encrypted with a customer managed KMS key whose key policy allows that role to decrypt. A secret encrypted with the AWS managed key aws/secretsmanager cannot be read from another account, because an AWS managed key's policy cannot be edited to admit outside principals — re-encrypt the secret with a customer managed key first. Secret replication does not help here: it copies a secret to other Regions within the same account, not to other accounts.

Inside Amazon Redshift, authorization is enforced by the database engine itself, layered under whatever IAM allows at the API level. Redshift supports database users (individual identities), groups (collections of users that receive GRANTs together), and roles — the role-based access control model where privileges attach to roles, roles can nest inside other roles, and users receive whatever their granted roles carry. Redshift also ships system-defined roles for common duty separations, such as operator, DBA, and security-administrator functions, so you can delegate administration without handing out superuser.

Authority is granted with SQL: GRANT SELECT on specific schemas or tables to a group or role, GRANT USAGE on a schema before its tables are reachable, and column-level GRANTs when a team may see a table but not its sensitive columns. The pattern to internalize: schema-per-team plus role-per-function keeps grants manageable; per-user GRANTs sprawl and rot.

Existing tables versus future tables

GRANT SELECT ON ALL TABLES IN SCHEMA orders TO ROLE reporting covers only the tables that exist when it runs; a table created next week is not included. To cover future tables, use ALTER DEFAULT PRIVILEGES. Default privileges apply to objects created by a particular user: written as ALTER DEFAULT PRIVILEGES FOR USER loader_user IN SCHEMA orders GRANT SELECT ON TABLES TO ROLE reporting, they cover every table loader_user creates in that schema from now on. Without the FOR USER clause they apply only to objects created by the user running the statement — so an administrator who runs it without naming the loading user has granted nothing on the tables the pipeline creates. A typical setup runs both statements: the GRANT for today's tables, the default privileges for tomorrow's.

Row-level security and how it differs from column controls

Redshift row-level security (RLS) filters which rows a query returns, inside the database, for every SQL client. You CREATE RLS POLICY with a predicate (for example, department matches the current user's department through a lookup table), ATTACH it to a table for specific users or roles, and turn RLS on for the table with ALTER TABLE ... ROW LEVEL SECURITY ON. Once RLS is on, it is deny by default: a user or role with no attached policy that permits rows sees no rows at all, not every row. A principal that must see everything — say a regional sales director over an orders table — needs either a policy that returns all rows or the IGNORE RLS system permission granted to its role. Creating, attaching and exempting are security-administrator tasks (superuser or the sys:secadmin role). One RLS policy scales as departments are added, where a view per department multiplies objects.

Masking is taught in depth in the encryption and masking lesson; here the point is that only RLS answers a row requirement.

The bridge between IAM and the database is temporary credentials. Rather than maintaining passwords for every analyst, principals with the right IAM permissions can request short-lived database credentials (the GetClusterCredentials pattern for provisioned clusters, and IAM-based sign-in for Redshift Serverless), optionally auto-creating the user and joining the appropriate groups at login. Redshift also integrates with IAM Identity Center so warehouse access follows your workforce identities. And when Redshift itself needs to touch other services — COPY from S3, Spectrum queries against the Glue Data Catalog — it authorizes through an attached IAM role, never through embedded keys. A scenario about analysts needing governed warehouse access without password management is pointing at IAM-issued temporary database credentials plus groups or roles carrying the GRANTs.

Lake Formation centralizes data-lake authorization in three moves. First, you register S3 locations with Lake Formation, handing it responsibility for those paths; registration uses a role Lake Formation assumes to access the data. Second, you define databases and tables over that data in the Glue Data Catalog. Third, you grant permissions — SELECT, INSERT, DESCRIBE, ALTER and so on — to principals (IAM users and roles, and cross-account principals) on catalog resources, using the console, the GRANT-style API, or LF-Tags. A separate data location permission controls who may create databases and tables that point at a registered S3 location; it governs catalog creation, not who can read the data. Data lake administrators manage grants; you can also delegate grant authority itself with grantable permissions.

Enforcement is what makes this powerful. When an integrated engine — Amazon Athena, Redshift Spectrum, Amazon EMR (with runtime roles on EMR on EC2, and in EMR Serverless jobs), AWS Glue, or Amazon QuickSight — reads a governed table, Lake Formation checks the principal's grants and vends temporary S3 credentials scoped to exactly the permitted data. The querying principal needs no direct S3 policy on the lake buckets at all, which eliminates the bucket-policy sprawl that plagues IAM-only lakes.

Default settings and the IAM bypass

A freshly set-up Lake Formation does not enforce its own grants. For backward compatibility, the default Data Catalog settings use only IAM access control for new databases and tables, which they implement by granting Super to the IAMAllowedPrincipals group on each new resource. While that grant exists, anyone whose IAM policy allows the Glue and S3 calls can read the whole table, and a column-limited Lake Formation grant appears to do nothing. To enforce Lake Formation, revoke IAMAllowedPrincipals on the existing databases and tables and clear the defaults that would re-create it: the account-level settings for new databases and tables, and the per-database option Use only IAM access control for new tables in this database. The per-database option matters because it is stamped onto each table at creation: if it stays selected, tables created in that database later are granted to IAMAllowedPrincipals again, however carefully the existing ones were locked down.

Even with IAMAllowedPrincipals gone, Lake Formation enforces grants only on reads that go through integrated engines with vended credentials. It does not intercept S3 API calls: an analyst whose IAM policy still allows s3:GetObject on the lake bucket can download whole files with the AWS CLI. Removing direct S3 permissions from analyst roles is part of the cutover. Hybrid access mode exists for gradual migrations: on a location registered in hybrid mode, principals you opt in are governed by Lake Formation grants, while others keep their existing IAM access — useful for moving teams over one at a time, not a way to block direct S3 reads.

Cross-account sharing lets a producer account grant catalog permissions to a consumer account without copying data or writing cross-account bucket policies; it takes a grant in the producer, acceptance and a resource link in the consumer, and local grants by the consumer's administrator.

1. Producer grants to the account. The producer's data lake administrator grants, for example, SELECT (with grant option) on a table to the consumer's account ID. With the named-resource method, Lake Formation shares the resource through AWS Resource Access Manager (AWS RAM).
2. Consumer accepts. The consumer's data lake administrator accepts the AWS RAM invitation (accounts in the same organization with RAM sharing enabled do not need to).
3. Consumer creates a resource link. Shared tables live in the producer's catalog (they are listed in the consumer's Lake Formation console under the producer's catalog ID), so engines such as Athena and Redshift Spectrum cannot query them until the consumer creates a link in its own catalog. The consumer administrator creates a resource link — a local catalog object that points at the shared table or database.
4. Consumer grants to local principals. Analysts need DESCRIBE on the resource link, so they can see it, and SELECT on the link's target (the shared table), so they can query it. A resource link accepts only DESCRIBE and DROP; granting SELECT on the link itself is not how data access is given.

The producer manages only account-level grants; the consumer administrator decides which of its own principals get access. Revoking the producer's grant cuts the whole consumer account off at once.

A concrete scenario: your central data platform account owns the curated lake. The marketing account's analysts need three tables, one of which contains customer PII columns they must not see. With Lake Formation you share the tables cross-account with column-level exclusions, the marketing admin grants their analysts SELECT, and every Athena query in the marketing account automatically respects the column filter. No bucket policies changed hands, no data was copied, and revoking the share instantly cuts access. That single-point-of-control story — grant once, enforced by every engine — is the recurring reason Lake Formation is the correct exam answer over hand-built IAM plumbing.

Named-resource grants do not scale: a lake with hundreds of tables and dozens of teams would need thousands of individual grants, each one touched every time a table is added. LF-Tags (Lake Formation tag-based access control) fix this by making authorization declarative. You define tag keys and values — say classification with values public, internal, restricted, or department with sales and finance — attach them to databases, tables, and columns, and grant principals permissions on tag expressions instead of resources: analysts get SELECT wherever classification equals internal. New tables inherit their database's tags, so freshly landed data is governed the moment it is cataloged, with zero new grants. Tag a column restricted and it silently disappears from the analysts' query results.

Compare that with a wildcard grant on All tables in a database. A wildcard also reaches tables added later, but it grants the same permission on every one of them, so it cannot respect tables of different sensitivity inside the same database. When sensitivity varies table by table, LF-Tags are the mechanism; a wildcard fits only a database whose tables are all equally shareable.

Below table level, Lake Formation offers true fine-grained control. Column-level security includes or excludes specific columns from a grant. Row-level security uses data filters with filter expressions — the analyst granted a filter of region equals EMEA sees only EMEA rows. Combining a column list with a row filter yields cell-level security: different principals see different columns of different rows of the same physical table. All of it is enforced by the engines, not by views you must maintain per audience.

Step back and you have three authorization models to choose among, and the exam names all three. Role-based access control (RBAC) maps permissions to job functions — Redshift roles, or an IAM role per team — simple and auditable, but roles multiply as the organization grows. Tag-based access control is Lake Formation's LF-TBAC: permissions follow data classification tags, scaling effortlessly as resources are added. Attribute-based access control (ABAC) drives decisions from attributes on the principal and resource — in IAM, conditions comparing aws:PrincipalTag to aws:ResourceTag mean one policy serves every team, because a user tagged department sales automatically reaches resources tagged department sales.

Writing an ABAC condition

The ABAC condition puts the resource's tag as the condition key and the caller's tag as the policy variable: StringEquals aws:ResourceTag/project : ${aws:PrincipalTag/project}. A new project then needs only a tagged resource and tagged roles, with no policy edit. Keep the near-twin keys apart: aws:RequestTag/key reads tags passed in the request (used to control which tags may be set when creating or tagging a resource), and s3:ExistingObjectTag/key reads an object's tags, not the bucket's. Amazon S3 general purpose buckets support ABAC on bucket tags since November 2025: once ABAC is enabled on a bucket (the PutBucketAbac API), S3 evaluates aws:ResourceTag conditions against that bucket's tags, so one identity policy can give each project role access to only its own project's buckets.

Rule of thumb: RBAC for a stable, small set of functions; ABAC or LF-Tags when teams, users, and datasets grow faster than you can write grants.

Tie the mechanisms together. Your company runs a central lake account, a Redshift warehouse, and three consumer teams — finance, marketing, and data science — with a compliance rule that PII is visible only to finance, and that all access must be auditable and revocable in one place.

You register the curated S3 locations with Lake Formation, revoke IAMAllowedPrincipals and clear the IAM-only defaults, remove direct S3 access from analyst roles, and define LF-Tags: classification (general, pii) and domain (finance, marketing). Columns holding personal data are tagged pii; everything else general. Finance's role is granted SELECT where classification is general or pii within domain finance; marketing and data science get SELECT on classification general only. Data-science notebooks on EMR use runtime roles so each job carries its own identity into Lake Formation, and a row-level data filter limits marketing to rows where consent equals true. When a new table lands in the curated database, it inherits the database tags and is instantly governed — nobody writes a new grant.

In Redshift, warehouse access uses IAM-issued temporary database credentials; a finance role and an analyst role carry the schema GRANTs, a row-level security policy scopes a shared revenue table by business unit (with IGNORE RLS granted only to the finance leadership role that needs every row), and default privileges for the ETL user make each new table queryable by the analyst role without a fresh GRANT. Pipeline roles get customer managed IAM policies scoped to their exact S3 prefixes and their one Secrets Manager secret, with iam:PassRole limited to the specific pipeline roles. Connection strings and API keys for third-party sources live in Secrets Manager with rotation; non-secret job settings live in Parameter Store under a /prod/etl/ hierarchy. Every layer answers an exam theme: IAM authorizes the machinery, Lake Formation authorizes the data, tags scale the grants, and nothing holds more privilege than its job requires.