Applying Authorization Mechanisms: Lake Formation, IAM, and Redshift

AWS gives data platforms two complementary authorization systems, and the first exam skill is knowing which one answers a given question. IAM authorizes API actions: can this principal call s3:GetObject on this bucket, glue:GetTable on this catalog resource, athena:StartQueryExecution at all. AWS Lake Formation authorizes data access: can this principal SELECT from this table, and if so, which columns and rows. Lake Formation sits on top of the AWS Glue Data Catalog and vends temporary, scoped-down credentials to integrated engines, so a single grant is enforced identically whether the query comes from Athena, Redshift Spectrum, or EMR.

The two layers cooperate rather than compete. A principal querying a governed table still needs IAM permission to call the query engine (for example Athena and Glue APIs), but it does not need direct S3 permissions on registered lake locations — Lake Formation supplies short-lived credentials for exactly the data the grant allows. That credential-vending model is why the exam favors Lake Formation for fine-grained, multi-team lake access: without it you would be maintaining brittle S3 bucket policies that cannot express column- or row-level rules at all. One migration gotcha: existing catalogs default to an IAMAllowedPrincipals compatibility setting that keeps IAM-only enforcement in place until you deliberately revoke it, so new Lake Formation grants may appear not to take effect until that legacy setting is removed.

AWS managed policies get you started, but they are deliberately generic — AmazonS3ReadOnlyAccess grants read access to every bucket in the account, and the exam expects you to recognize that as a least-privilege failure. You write a custom (customer managed) policy whenever a managed policy is too broad, too narrow, or missing a condition your business requires. Customer managed policies are reusable across principals, versioned (you can roll back), and yours to scope precisely; inline policies embed in a single principal and are best reserved for one-off exceptions you never want reused.

A policy statement has four load-bearing parts: Effect (Allow or Deny), Action (the API calls), Resource (the ARNs), and optional Condition blocks. Evaluation logic is a favorite exam trap: everything is implicitly denied by default, any applicable Allow grants access, and an explicit Deny anywhere always wins — across identity policies, resource policies, permissions boundaries, and service control policies. Explicit Deny is therefore your guardrail tool: for example, deny s3:PutObject unless the request includes the correct encryption header, or deny all actions when aws:SourceVpce does not match your VPC endpoint.

A concrete data-engineering example: a Glue job must read raw data from one prefix and write curated Parquet to another. The custom policy allows s3:GetObject on the raw prefix ARN, s3:PutObject on the curated prefix ARN, and s3:ListBucket on the bucket with a prefix condition — nothing account-wide, no s3 wildcard actions. Remember that object actions take object-level ARNs with a path, while ListBucket takes the bucket ARN; mixing those up is a classic reason a scoped policy mysteriously fails.

Least privilege means every principal holds exactly the permissions its job requires — no wildcard actions, no account-wide resources, no permanent grants for temporary needs. In practice you construct it iteratively: start from what the workload actually calls, scope Resource to specific ARNs and prefixes, then tighten with conditions. Useful condition keys for data work include aws:SourceVpce and aws:SourceVpc (require the private path), aws:PrincipalTag and aws:ResourceTag (attribute-based control), s3:prefix (limit listing), and aws:SecureTransport (refuse unencrypted connections).

AWS gives you tooling so least privilege is not guesswork. IAM Access Analyzer can generate a policy from the CloudTrail activity of a role — run the pipeline for a while, then generate a policy that permits only what it actually did. Access Analyzer also validates policies against best practices and flags resources shared outside your zone of trust. The last accessed information on roles shows which granted services have never been used, telling you what to prune. Permissions boundaries cap the maximum permissions an identity policy can grant — handy when platform teams let data teams create their own pipeline roles but need a ceiling on what those roles can ever do.

Two habits close the loop. First, prefer temporary elevation over standing privilege: an engineer who occasionally needs to delete lake objects should assume a scoped role for that task, not carry the permission daily. Second, remember iam:PassRole: any policy that lets a user hand a role to Glue, Lambda, or CloudFormation must restrict which roles can be passed, or the user can escalate by passing an administrator role to a service they control. Exam options that casually include iam:PassRole on Resource star are showing you the wrong answer.

Authorization extends to the credentials your applications use, and skill 4.2.2 names the two stores you must be able to choose between. AWS Secrets Manager is purpose-built for secrets: encryption with AWS KMS is mandatory, automatic rotation is built in (managed rotation for RDS, Aurora, Redshift, and DocumentDB, or a custom rotation Lambda), secrets can be replicated to other Regions, and resource policies on individual secrets enable clean cross-account sharing. AWS Systems Manager Parameter Store is a general configuration store: String and StringList parameters for plain settings, SecureString parameters (KMS-encrypted) for sensitive ones, organized in hierarchical paths, with a standard tier that costs nothing.

The exam decision rule is short: rotation or cross-account secrets means Secrets Manager; cheap configuration means Parameter Store. Authorization applies identically to both — grant secretsmanager:GetSecretValue or ssm:GetParameter on specific ARNs only, and use path-based ARNs (for example a /prod/etl/ parameter hierarchy) so one policy statement cleanly covers a team's parameters without exposing anyone else's. Both services log retrievals to CloudTrail, which is what turns credential access into something you can audit.

Inside Amazon Redshift, authorization is enforced by the database engine itself, layered under whatever IAM allows at the API level. Redshift supports database users (individual identities), groups (collections of users that receive GRANTs together), and roles — the role-based access control model where privileges attach to roles, roles can nest inside other roles, and users receive whatever their granted roles carry. Redshift also ships system-defined roles for common duty separations, such as operator, DBA, and security-administrator functions, so you can delegate administration without handing out superuser.

Authority is granted with SQL: GRANT SELECT on specific schemas or tables to a group or role, GRANT USAGE on a schema before its tables are reachable, and column-level GRANTs when a team may see a table but not its sensitive columns. Redshift additionally supports row-level security policies that filter which rows a user's queries return — for example, regional sales analysts see only their region's rows from the shared fact table. The pattern to internalize: schema-per-team plus role-per-function keeps grants manageable; per-user GRANTs sprawl and rot.

The bridge between IAM and the database is temporary credentials. Rather than maintaining passwords for every analyst, principals with the right IAM permissions can request short-lived database credentials (the GetClusterCredentials pattern for provisioned clusters, and IAM-based sign-in for Redshift Serverless), optionally auto-creating the user and joining the appropriate groups at login. Redshift also integrates with IAM Identity Center so warehouse access follows your workforce identities. And when Redshift itself needs to touch other services — COPY from S3, Spectrum queries against the Glue Data Catalog — it authorizes through an attached IAM role, never through embedded keys. A scenario about analysts needing governed warehouse access without password management is pointing at IAM-issued temporary database credentials plus groups or roles carrying the GRANTs.

Lake Formation centralizes data-lake authorization in three moves. First, you register S3 locations with Lake Formation, handing it responsibility for those paths; registration uses a role Lake Formation assumes to access the data. Second, you define databases and tables over that data in the Glue Data Catalog. Third, you grant permissions — SELECT, INSERT, DESCRIBE, ALTER and so on — to principals (IAM users and roles, and cross-account principals) on catalog resources, using the console, the GRANT-style API, or LF-Tags. Data lake administrators manage grants; you can also delegate grant authority itself with grantable permissions.

Enforcement is what makes this powerful. When an integrated engine — Amazon Athena, Redshift Spectrum, Amazon EMR (with runtime roles on EMR on EC2, and in EMR Serverless jobs), AWS Glue, or Amazon QuickSight — reads a governed table, Lake Formation checks the principal's grants and vends temporary S3 credentials scoped to exactly the permitted data. The querying principal needs no direct S3 policy on the lake buckets at all, which eliminates the bucket-policy sprawl that plagues IAM-only lakes. Cross-account sharing follows the same grant model: share a database or table to another account or organization, where the receiving admin grants it onward to local principals — far cleaner than replicating data or juggling cross-account bucket policies.

A concrete scenario: your central data platform account owns the curated lake. The marketing account's analysts need three tables, one of which contains customer PII columns they must not see. With Lake Formation you share the tables cross-account with column-level exclusions, the marketing admin grants their analysts SELECT, and every Athena query in the marketing account automatically respects the column filter. No bucket policies changed hands, no data was copied, and revoking the share instantly cuts access. That single-point-of-control story — grant once, enforced by every engine — is the recurring reason Lake Formation is the correct exam answer over hand-built IAM plumbing.

Named-resource grants do not scale: a lake with hundreds of tables and dozens of teams would need thousands of individual grants, each one touched every time a table is added. LF-Tags (Lake Formation tag-based access control) fix this by making authorization declarative. You define tag keys and values — say classification with values public, internal, restricted, or department with sales and finance — attach them to databases, tables, and columns, and grant principals permissions on tag expressions instead of resources: analysts get SELECT wherever classification equals internal. New tables inherit their database's tags, so freshly landed data is governed the moment it is cataloged, with zero new grants. Tag a column restricted and it silently disappears from the analysts' query results.

Below table level, Lake Formation offers true fine-grained control. Column-level security includes or excludes specific columns from a grant. Row-level security uses data filters with filter expressions — the analyst granted a filter of region equals EMEA sees only EMEA rows. Combining a column list with a row filter yields cell-level security: different principals see different columns of different rows of the same physical table. All of it is enforced by the engines, not by views you must maintain per audience.

Step back and you have three authorization models to choose among, and the exam names all three. Role-based access control (RBAC) maps permissions to job functions — Redshift roles, or an IAM role per team — simple and auditable, but roles multiply as the organization grows. Tag-based access control is Lake Formation's LF-TBAC: permissions follow data classification tags, scaling effortlessly as resources are added. Attribute-based access control (ABAC) drives decisions from attributes on the principal and resource — in IAM, conditions comparing aws:PrincipalTag to aws:ResourceTag mean one policy serves every team, because a user tagged department sales automatically reaches resources tagged department sales. Rule of thumb: RBAC for a stable, small set of functions; ABAC or LF-Tags when teams, users, and datasets grow faster than you can write grants.

Tie the mechanisms together. Your company runs a central lake account, a Redshift warehouse, and three consumer teams — finance, marketing, and data science — with a compliance rule that PII is visible only to finance, and that all access must be auditable and revocable in one place.

You register the curated S3 locations with Lake Formation and define LF-Tags: classification (general, pii) and domain (finance, marketing). Columns holding personal data are tagged pii; everything else general. Finance's role is granted SELECT where classification is general or pii within domain finance; marketing and data science get SELECT on classification general only. Data-science notebooks on EMR use runtime roles so each job carries its own identity into Lake Formation, and a row-level data filter limits marketing to rows where consent equals true. When a new table lands in the curated database, it inherits the database tags and is instantly governed — nobody writes a new grant.

In Redshift, warehouse access uses IAM-issued temporary database credentials; a finance role and an analyst role carry the schema GRANTs, and a row-level security policy scopes a shared revenue table by business unit. Pipeline roles get customer managed IAM policies scoped to their exact S3 prefixes and their one Secrets Manager secret, with iam:PassRole limited to the specific pipeline roles. Connection strings and API keys for third-party sources live in Secrets Manager with rotation; non-secret job settings live in Parameter Store under a /prod/etl/ hierarchy. Every layer answers an exam theme: IAM authorizes the machinery, Lake Formation authorizes the data, tags scale the grants, and nothing holds more privilege than its job requires.