Applying Authentication Mechanisms: IAM, Secrets Manager, and Networks

AWS authenticates every API call by verifying a cryptographic signature attached to the request. When you call an AWS service — from the console, the AWS CLI, an SDK, or another AWS service — the request is signed with a set of credentials, and IAM resolves those credentials to a principal: an IAM user, an IAM role session, or the account root user. Only after the principal is identified does authorization (policy evaluation) happen. Authentication asks who are you; authorization asks what are you allowed to do. The DEA-C01 exam expects you to keep those two ideas separate.

Credentials come in two flavors. Long-term credentials are an access key ID and secret access key attached to an IAM user — they never expire on their own, which makes them a standing liability if leaked. Temporary credentials are issued by AWS Security Token Service (STS) when a principal assumes a role: an access key, secret key, and session token that expire automatically after the session duration. Almost every exam scenario that mentions hard-coded keys, leaked keys, or credentials in code has the same shape of answer: replace the long-term credentials with an IAM role and temporary STS credentials.

For data engineering this matters everywhere. A Glue job, a Lambda function, a Redshift cluster loading from S3, an EMR cluster's EC2 instances — none of them should hold access keys. Each assumes a role, AWS delivers short-lived credentials to the compute automatically, and rotation is handled for you. Human access follows the same principle: prefer federated identities that receive temporary role credentials (through IAM Identity Center — see the workforce section below) over IAM users with passwords and access keys.

IAM gives you three identity building blocks, and the exam tests when to use each. An IAM user is a permanent identity with its own credentials, meant for the rare cases where a specific person or legacy system truly needs long-term keys. An IAM group is a collection of users used purely to attach policies once and have them apply to every member — groups cannot be referenced as a principal in resource policies and cannot be assumed; they are a permissions-management convenience, not an identity that authenticates. An IAM role is an assumable identity with no long-term credentials at all: a trust policy defines who may assume it, and permission policies define what the resulting session can do.

Roles are the default answer for anything automated. A role's trust policy is the authentication gate: for a Lambda execution role it trusts the Lambda service principal; for cross-account access it trusts the other account; for federation it trusts an identity provider. A common gotcha: assuming a role replaces your permissions entirely — the session has the role's permissions, not the union of the role's and the caller's.

Groups and resource policies. Because a group is not a principal, a bucket policy, access point policy, KMS key policy or role trust policy that names a group ARN is rejected as an invalid principal. To give every member of a group access to a bucket in the same account, attach an identity-based policy that allows the S3 actions to the group itself; same-account access needs an allow from either the identity policy or the resource policy, so the bucket policy does not have to mention the group at all, and access follows group membership automatically.

Cross-account access: a role in the resource account or a resource-based policy

There are two ways to let principals in account A reach data in account B, and they differ in where the grant lives. With a role in account B, account B writes the trust policy (who may assume it) and the permissions policy (what the session can do); the callers in account A need only sts:AssumeRole on that role ARN. The whole grant, and its revocation, is controlled in account B — change or delete the role and access ends. With a resource-based policy (a bucket or access point policy) that names account A or its principals, access needs an allow on both sides: account B's resource policy and an identity policy in account A that allows the same actions. That suits services where the caller keeps its own identity (for example, writing logs while staying in its own session), but the owner cannot define or revoke the whole grant alone.

Third parties and the confused deputy

When a third-party service (an analytics SaaS, a monitoring vendor) assumes a role in your account, it usually serves many customers from one AWS account. If your trust policy trusts only that account, another customer of the same provider could configure your role ARN in their own tenant and have the provider — the confused deputy — assume your role on their behalf. The fix is the sts:ExternalId condition: the provider generates an external ID that is unique to your customer relationship, shows it to you, and passes it on every AssumeRole call; your trust policy requires that value. Conditions on source account or IP address cannot help here, because every customer's request comes from the same provider account and the same provider infrastructure — only the external ID tells the tenants apart. The external ID is not a secret password; it is a tenant identifier the customer cannot choose.

Also know the vocabulary of skill 4.1.2: you create and update IAM groups and roles for people and workloads, and you attach policies to endpoints and services — VPC endpoint policies and service-linked or service roles — so that both identities and the paths they travel are governed.

Each compute or orchestration service authenticates to other AWS services through a role you hand it, and you should be able to name the pattern for each. AWS Lambda has two separate policies that are easy to confuse. The execution role decides what the function can do: the function assumes it at invocation, and its policies must grant exactly what the function touches — for a pipeline function, perhaps reading one S3 prefix, writing to one Kinesis stream, and fetching one secret from Secrets Manager — plus CloudWatch Logs writes, which the AWSLambdaBasicExecutionRole managed policy covers. The function's resource-based policy decides who may invoke it: an S3 event notification, an EventBridge rule, API Gateway or another account needs a lambda:InvokeFunction statement there. A function that cannot read its bucket needs a fix to its execution role; a bucket notification that never triggers the function needs a fix to the function's resource-based policy.

Amazon API Gateway: authenticating callers

API Gateway plays both sides of authentication. Inbound, a REST API offers three ways to authenticate callers. IAM authorization requires requests signed with SigV4, which suits AWS workloads that already run with IAM roles. A Cognito user pool authorizer validates tokens issued by the Amazon Cognito user pools you configure — and only those; it cannot validate tokens from another identity provider. A Lambda authorizer runs your own code to validate any bearer token or request attributes, so it is the REST API choice for tokens from an external OpenID Connect or OAuth provider (HTTP APIs also offer a built-in JWT authorizer for external issuers). API keys with usage plans identify clients for throttling and quotas; they are not an authentication mechanism and should not be the only control on an API.

Amazon API Gateway: the integration role

Outbound, an AWS service integration (for example, an API that writes records straight to a Kinesis data stream with no Lambda in between) uses an integration role set as the integration's credentials. Two policies must be right: the role's permissions policy allows the backend action (kinesis:PutRecord on that stream), and its trust policy allows apigateway.amazonaws.com to call sts:AssumeRole. If the trust policy is missing, API Gateway cannot assume the role and the call fails as an internal configuration error even though the permissions look correct. The alternative, invoke with caller credentials, makes API Gateway call the backend with each caller's own identity — which works only by giving every caller permission on the backend resource itself, so it is the wrong choice when callers must not be able to reach that resource directly.

The AWS CLI

The CLI authenticates with whatever credentials its profile resolves. For people, use IAM Identity Center sign-in (covered in the workforce section); on EC2 or in containers, the CLI automatically picks up the instance profile or task role from the metadata endpoint — no keys on disk. A named profile can also assume a role (role_arn plus a source_profile or credential_source), so an engineer authenticates once and pivots into a scoped role for sensitive operations.

iam:PassRole: handing a role to a service

Whenever you configure a service to run as a role — a Glue job role, a Lambda execution role, an EC2 instance profile, a CloudFormation service role — you pass that role to the service, and IAM checks iam:PassRole. Three mechanics are tested. First, PassRole is a permission of the principal doing the passing (the engineer or the deployment pipeline), not of the role being passed; adding it to the role's own policy does nothing. Second, the policy's Resource is the ARN of the role being passed, not the ARN of the Glue job, function or stack that receives it. Third, the iam:PassedToService condition key limits which service the role may be handed to (for example, glue.amazonaws.com), so the same permission cannot be reused to give the role to EC2 or Lambda. Passing a role is not assuming it: the trust policy should name the service principal, not the engineer. An AccessDenied error naming iam:PassRole is fixed by granting PassRole on that role's ARN, scoped with iam:PassedToService — never by granting PassRole on *.

AWS CloudFormation service roles

CloudFormation deserves special attention because it is a privilege-escalation vector. Without a service role, a stack operates with the permissions of the person who launches it, so creating a template that builds IAM roles requires that person to hold IAM write permissions. With a service role, CloudFormation assumes the role you name for every stack operation; the person needs CloudFormation permissions plus iam:PassRole on that service role, but never holds the role's permissions themselves — they can create IAM roles only through a stack. Passing a role differs from assuming one: assuming a role hands the caller the role's permissions directly, usable outside any stack. Scope CloudFormation service roles tightly, and control who may pass them with PassRole on the specific role ARN, conditioned on cloudformation.amazonaws.com.

IAM Identity Center is the AWS service for giving a workforce single sign-on to many AWS accounts from one place. You connect it to your identity source (an external SAML 2.0 identity provider such as Okta or Microsoft Entra ID, Active Directory, or its own directory), define access once as permission sets, and assign a permission set to users or groups for any accounts in AWS Organizations. Identity Center then provisions each permission set as an IAM role in every assigned account, and users sign in once and receive temporary credentials for the account and permission set they choose.

Per-account SAML federation delivers single sign-on and temporary credentials too, so the deciding requirement is usually central definition and assignment across accounts — that is what Identity Center adds.

The AWS CLI with Identity Center. Running aws configure sso creates a profile with an sso_session, an sso_account_id and an sso_role_name; aws sso login opens the browser sign-in and caches a token, and the CLI then fetches temporary credentials for that account and permission set. sso_role_name can name only a permission set assigned to you — it cannot reach an ordinary IAM role. To use an ordinary role (one whose trust policy allows your Identity Center role), add a second profile with role_arn set to that role and source_profile set to your SSO profile: the CLI signs in through Identity Center, then chains an AssumeRole call. No access keys are stored anywhere, and remember that a chained role session is limited to one hour.

AWS Secrets Manager exists because some authentication cannot be replaced by IAM roles: databases, third-party APIs, and SaaS platforms still want usernames, passwords, and API keys. Secrets Manager stores those secrets encrypted with AWS KMS, controls retrieval through IAM policies, logs every access in CloudTrail, and — its defining feature — rotates credentials on a schedule. A consumer such as a Glue connection or a Lambda function references the secret and fetches it at run time, so rotation needs no code change or redeployment.

Choosing the rotation mechanism

Secrets Manager rotates in one of two ways. Managed rotation is performed by another AWS service for a secret it manages — for example, an Amazon RDS or Aurora master user password, an Amazon DocumentDB master user password, or an Amazon Redshift admin password, that you let the service manage in Secrets Manager; no Lambda function is involved. Secrets Manager also offers managed external secrets, which rotate credentials for onboarded SaaS partners without a Lambda function. Rotation by a Lambda function covers everything else: for RDS, Aurora, Redshift and DocumentDB database secrets AWS supplies rotation function templates that the console can deploy for you, and for other databases or third-party systems you supply a function that knows how to change the credential. A self-managed database on Amazon EC2 therefore always needs a rotation Lambda function — managed rotation is available only for databases an AWS service manages. A related trap: IAM database authentication (short-lived tokens instead of passwords) is a feature of Amazon RDS (MariaDB, MySQL and PostgreSQL engines) and Aurora (MySQL- and PostgreSQL-compatible), not of a database you run yourself on EC2.

Staging labels and rotation strategies

Each rotation creates a new version of the secret and moves staging labels between versions. AWSCURRENT marks the version clients get by default; AWSPENDING marks the new version while the rotation function creates it, sets it on the database, and tests it; at the end, AWSCURRENT moves to the new version and AWSPREVIOUS marks the one that was current before. What those labels mean for clients depends on the strategy:

A pipeline that caches credentials and cannot tolerate sign-in failures needs alternating users, because only that strategy keeps the previous credentials valid while clients catch up — changing the schedule of a single-user rotation does not change what happens at each rotation. Whatever the strategy, applications should fetch the secret at connection time (or use a caching client that refreshes) and retry with a fresh value on an authentication failure.

The rotation function's network path

A rotation function calls the Secrets Manager API (to read and write secret versions and labels) and connects to the database (to set and test the new password). When the function is attached to a VPC — necessary to reach a database in private subnets — it needs both paths: to Secrets Manager through an interface VPC endpoint (Secrets Manager has no gateway endpoint) or a NAT gateway, and to the database through a database security-group rule that allows the function's security group on the database port. A rotation that hangs until the function times out usually points to a network path rather than permissions (an IAM problem returns an explicit AccessDenied error), and fixing only one of the two paths moves the failure to the next step. Any fix has to keep both paths open at once.

Choosing where to store a credential — Secrets Manager versus AWS Systems Manager Parameter Store SecureString parameters, which hold values cheaply but have no built-in rotation — is Task 4.2 material, covered in the authorization lesson. A concrete pipeline pattern for this task: a Glue connection or Lambda function references a Secrets Manager secret for the Redshift password; its IAM role is granted secretsmanager:GetSecretValue on that one secret's ARN; rotation runs every 30 days; nothing is ever hard-coded.

The network layer is your first authentication filter: traffic that cannot reach a service cannot even attempt to log in. VPC security groups are stateful virtual firewalls attached to elastic network interfaces — instances, Lambda functions in a VPC, Glue connections, Redshift clusters, EMR nodes, and interface endpoints all carry them. They support allow rules only (anything not allowed is denied), and because they track connections, the reply to a connection an inbound rule allowed goes back out even if the group has no outbound rules at all. The most exam-relevant trick is security-group referencing: instead of allowing an IP range into your Redshift cluster's port, allow inbound from the security group attached to your Lambda or EMR compute. Membership in the group becomes the credential, and the rule survives IP changes and ignores unrelated servers sharing the same subnet. Updating security groups (skill 4.1.1) usually appears as: a new consumer needs database access, so you add an inbound rule referencing the consumer's security group on the database port.

Network ACLs are the other VPC firewall, and the contrast is tested:

Traffic must pass both: a permissive network ACL never overrides a security group that blocks, and a security group never overrides a network ACL deny. Use a network ACL deny to block a known-bad range for a whole subnet; use security groups for application-level allow lists.

VPC endpoints and AWS PrivateLink

VPC endpoints keep pipeline traffic off the public internet. Gateway endpoints exist only for Amazon S3 and Amazon DynamoDB — they are free and work through route-table entries, with no network interface and no security group. Interface endpoints, powered by AWS PrivateLink, put an elastic network interface with a private IP in your subnets for most AWS services (Kinesis, Glue, Secrets Manager, STS, the Redshift Data API, and many more), for third-party and cross-account services, and for S3 and DynamoDB too. Interface endpoints carry security groups and hourly plus per-GB charges; S3 and DynamoDB interface endpoints are what you use when the traffic comes from on premises or a peered network that cannot use a gateway endpoint's route tables. For VPC workloads reaching S3 and DynamoDB with no hourly cost, choose the gateway endpoints.

Private DNS is what makes an interface endpoint transparent. With private DNS enabled on the endpoint (and DNS support and DNS hostnames enabled on the VPC), the service's default Regional hostname — for example, the Secrets Manager endpoint the SDK uses by default — resolves inside the VPC to the endpoint's private IPs, so unchanged SDK and CLI code goes through PrivateLink. Without private DNS, code has to be pointed at the endpoint-specific DNS name. The endpoint's security group must also allow HTTPS (port 443) from the calling workloads.

AWS Glue connections in a VPC. When a Glue job uses a connection to reach a database in private subnets, Glue places network interfaces in the connection's subnet. That subnet needs a route to Amazon S3 — an S3 gateway endpoint or a NAT gateway — because Glue reads its scripts and temporary data from S3; without one the job fails to start with an error that it could not find an S3 endpoint or NAT gateway for the subnet. The connection's security group also needs a self-referencing inbound rule (all TCP from the same security group) so Glue's own workers can talk to one another, and the database's security group must allow the Glue security group on the database port.

Endpoint policies versus aws:SourceVpce

Both endpoint types accept an endpoint policy — a resource policy applied to the network path itself, which is exactly what skill 4.1.5 means by applying IAM policies to endpoints. It and the bucket policy's endpoint condition control opposite directions, and the exam asks which one fits:

An endpoint policy never grants anything on its own — the identity policy and the bucket policy must still allow the request — and it governs only traffic that actually uses the endpoint. Same-Region S3 traffic from a subnet whose route table has the gateway endpoint route goes through the endpoint, but subnets without that route, buckets in other Regions, or other network paths are not covered by it. Used together, the two controls give layered authentication: valid IAM credentials and the approved private network path. One related condition-key trap: aws:SourceIp is not present for requests that arrive through a VPC endpoint, so an IP-range condition cannot identify traffic from inside a VPC (and a deny-unless-in-range condition would block it). Use the VPC-specific keys instead — aws:SourceVpce (the endpoint), aws:SourceVpc (the VPC), or aws:VpcSourceIp (the private IP).

S3 Access Points let one shared bucket serve many consumers, each through its own named entry point with its own policy, without copying data. Instead of one enormous bucket policy (which has a size limit and becomes risky to edit), you create an access point per application or team, each with a policy scoped to that consumer's prefixes and actions. The bucket policy delegates access control to access points — commonly with a statement that allows any action when the request comes through an access point owned by the bucket owner's account (the s3:DataAccessPointAccount condition). Each access point policy can then grant principals in the same or other accounts; a cross-account principal still needs an allow in its own account's identity policy, just as with a bucket policy.

Network origin. Every access point has a network origin chosen at creation and fixed afterwards. An Internet origin accepts requests from anywhere (still subject to policies); a VPC origin names one VPC, and the access point rejects any request that does not come from that VPC — even with valid credentials. Because the origin cannot be changed, a different network restriction means a different access point, and several access points with different origins can front the same bucket side by side. Because the bucket delegates to access points, data is reachable only through the paths their policies grant.

Do not confuse access points with S3 Multi-Region Access Points, which give one global endpoint that routes requests to buckets in several Regions for failover and latency. They solve multi-Region routing, not per-team policy on one bucket. When a scenario describes many consumers with different permissions on one bucket, or data that must be reachable only from within a VPC, (single-Region) S3 Access Points are the intended answer.

The managed/unmanaged distinction (skill 4.1.6) is about how much of the security stack — including authentication — AWS operates for you. With fully managed and serverless services such as Amazon Athena, AWS Glue, AWS Lambda, Amazon Redshift Serverless, and Amazon MSK Serverless, there are no servers you can log into: authentication is IAM end to end, AWS patches the infrastructure, and your responsibility shrinks to writing good policies and protecting your data. With unmanaged or self-managed deployments — databases on EC2, self-hosted Kafka or Airflow, an EMR cluster where you can SSH to nodes — you inherit OS users, SSH keys, database password files, and the patching burden along with them.

Partly managed services sit between the extremes. Amazon EMR on EC2 provisions and monitors the cluster, but the EC2 instances live in your VPC with security groups, instance profiles, and optional SSH access you must govern; EMR Serverless removes those hosts entirely. Amazon RDS manages the OS but leaves database-level users and passwords to you — which is why it pairs so naturally with Secrets Manager rotation.

For the exam, translate scenarios like this: if the requirement stresses minimal operational overhead or no credential management, choose the more managed option (Glue over self-managed Spark, Redshift Serverless over a database on EC2, EMR Serverless over EMR on EC2). If the scenario grants a need for OS-level control or custom software, accept the unmanaged option and expect follow-up questions about the security groups, key pairs, and instance roles you now must maintain.

Amazon SageMaker Unified Studio (part of the next generation of Amazon SageMaker) is a single environment where data engineers, analysts, and ML builders work with data, SQL, notebooks, and models — and its access model is built from three nested constructs the exam names explicitly (skill 4.1.7). A domain is the top-level boundary: it holds the user directory integration (typically IAM Identity Center for single sign-on, so people authenticate with corporate credentials rather than IAM users), the associated AWS accounts, and global policies. Domain units are an organizational hierarchy inside the domain — create one per business unit or team, delegate administration of each unit to that team's owners, and organize assets under them so authority is distributed instead of centralized.

Projects are where actual work and authentication meet. A project is a collaboration workspace with its own membership, its own tooling, and — critically — its own project IAM role. When a member runs a query, a notebook, or a job inside the project, the work executes with the project role's permissions, not with the individual's personal AWS credentials. Data and compute made available to the project are available to its members; people outside the project do not see them. Membership therefore becomes the unit of access control: add an analyst to the sales-analytics project and they can immediately query that project's assets with governed, temporary credentials.

The exam-level takeaway: domains establish who can sign in (federated authentication), domain units establish who administers what, and projects establish which role your work runs as. If a scenario asks how to give a cross-functional team shared, governed access to lakehouse data in SageMaker Unified Studio without minting IAM users, the answer is to create a project, add the members, and let the project role carry the permissions.

Put the layers together with a realistic build. Your team ingests orders through Amazon API Gateway, transforms them with AWS Lambda, stages them in Amazon S3, and loads them into Amazon Redshift — and security review demands no long-lived credentials and no public network paths.

At the API layer, API Gateway authenticates client apps with a Cognito user pool authorizer (the apps sign in to that user pool), and an integration role whose trust policy allows apigateway.amazonaws.com writes valid requests to a Kinesis data stream. The Lambda consumer has an execution role scoped to that stream, its S3 staging prefix, and one Secrets Manager secret ARN. Because the function runs in private subnets, it reaches S3 through the gateway endpoint and Secrets Manager and STS through interface endpoints — the endpoint policies allow only this account's principals, and the bucket policy requires the request to arrive through the VPC endpoint. The staging bucket delegates access control to access points and exposes one VPC-origin access point for the ingest role, so that grant works only from inside the VPC.

For the load, Redshift sits in private subnets with a security group allowing inbound only from the Lambda function's security group on the database port. The loader fetches the current database password with GetSecretValue; Secrets Manager rotates it on a 30-day schedule using the alternating-users strategy so loads never fail mid-rotation. Redshift's own COPY authenticates to S3 by assuming an attached IAM role — no keys in the COPY statement — and the bucket policy allows that role to read the staging prefix. The cluster has Enhanced VPC Routing enabled, so COPY traffic leaves through the cluster's VPC and the S3 gateway endpoint; without it, COPY reaches S3 outside the VPC and the bucket's endpoint condition would reject the load. Every credential in this pipeline is either a temporary STS credential or a rotated secret, which is exactly the posture DEA-C01 scenarios reward.