Encryption at Rest on AWS: KMS Keys, Key Policies, and Service Integration

Every KMS-integrated service encrypts under one of three key ownership models, and choosing between them is a control-versus-effort decision the exam tests repeatedly.

AWS owned keys live in AWS-internal accounts: you cannot see, audit, or choose them — DynamoDB's default encryption is the classic example. AWS managed keys (aliases like aws/s3) exist in your account and log to your CloudTrail, but you cannot edit their key policies, control rotation, or — critically — use them across accounts. Customer managed keys are the only model where you own the key policy, opt into rotation, create grants, schedule deletion, and authorize other accounts. The exam pattern: the moment a requirement mentions cross-account sharing, key-level audit boundaries, or the ability to revoke a service's access to data, the answer is a customer managed key.

Within a customer managed key there is a second choice: where the key material comes from. KMS-generated material is the default and supports every feature, including automatic rotation on a period you choose. Imported material (bring your own key) gives provenance and an expiry you set, but you must keep a durable copy to re-import, and it rotates only on demand — Task 5.3 covers its custody. Unless a requirement names key-material provenance, KMS-generated material in a customer managed key is the answer.

Symmetric AES-256 keys are the default and what every service integration uses. Asymmetric RSA and ECC keys exist for sign/verify and for encrypt-outside-AWS patterns — the public key is downloadable while the private key never leaves KMS — but know them at concept level.

Pick the key service by who must control the HSM and which API the application speaks, and pick the encryption type by who must be prevented from seeing plaintext. AWS KMS is the default because every AWS service integrates with it; AWS CloudHSM exists for requirements KMS cannot meet.

A custom key store's keys live in CloudHSM, but an application can reach them only through KMS calls, so it does not satisfy a requirement to keep unmodified PKCS#11 or JCE code. Conversely, CloudHSM used directly has no native integration with S3 or EBS server-side encryption — your application does the encrypting. With CloudHSM you also own availability: run at least two HSMs in different Availability Zones.

Server-side vs client-side. Server-side encryption (SSE-S3, SSE-KMS, EBS and RDS encryption) means the service encrypts after receiving plaintext — simple, transparent, and enough when the threat is lost media or unauthorized API access. Client-side encryption means the application encrypts before the data leaves it, usually with the AWS Encryption SDK or the AWS Database Encryption SDK over KMS data keys, so the storage service (and anyone with access to it) holds only ciphertext. Choose client-side when the requirement says the storage layer, its administrators, or the service must never see plaintext, or when individual fields must stay encrypted inside a database. The cost is that you lose server-side features that need plaintext, such as querying encrypted fields, and you carry the key-handling code.

Unlike most AWS resources, a KMS key does not implicitly trust its own account. The key policy is the root of trust: no principal — not even the account root user — can use or manage a key unless the key policy allows it, directly or by delegation.

The default key policy that KMS applies when you create a key programmatically without supplying a policy contains one statement: it allows the account principal (arn:aws:iam::111122223333:root) the full kms:* action set. That statement is not really about the root user — it is the delegation switch that enables IAM policies to work. With it present, an IAM identity policy in the account can grant KMS permissions on the key; without it, IAM policies are inert for that key. Delete or over-restrict that statement and you can create an unmanageable key — one no principal can administer — recoverable only by contacting AWS Support. Treat edits to it like production changes.

Two guard-rails and one trap sit around this. The policy lockout safety check makes CreateKey and PutKeyPolicy reject a policy that would stop the calling principal from changing the key policy afterwards; it can be overridden with BypassPolicyLockoutSafetyCheck, which you should essentially never use. The check only looks at the caller at that moment, though — a key can still become unmanageable later. The trap is deleted principals: a key policy stores a named IAM role or user by its unique principal ID. If that principal is deleted, the policy shows the ID instead of the ARN, and recreating a role with the same name does not restore access, because the new role has a new ID. When the deleted principal was the only one allowed to manage the key and the account-principal statement is gone, only AWS Support can recover the key.

Access to a key therefore flows through two policy paths that combine: the key policy can authorize a principal directly, or the key policy delegates to IAM and an identity policy supplies the allow. Cross-account use requires both halves: the key policy must allow the external account (or specific principals in it), and the external account must separately grant its principals IAM permission on the key's ARN. Either half alone fails — a favorite distractor. Only customer managed keys support this; aws/service keys cannot cross accounts. In the caller's IAM policy the Resource must be the key ARN: an alias ARN in the Resource element controls the alias itself (for example kms:UpdateAlias), not the key behind it. To write IAM policies in terms of aliases, use the kms:RequestAlias or kms:ResourceAliases condition keys instead.

At organization scale, choose the organization condition key by who is calling:

Grants are the third access path, alongside key policies and IAM policies: a temporary, programmatic delegation attached to a key. A grant names a grantee principal, a subset of allowed operations (decrypt, encrypt, generate data key, describe, retire, and a few others — grants can only allow, never deny), and optional constraints that pin the grant to a specific encryption context.

Grants exist mainly so AWS services can use your keys on your behalf without long-lived policy entries. The canonical example is EBS: when you attach an encrypted volume, EC2 creates a grant on the volume's KMS key so the service can decrypt the volume's data key during attachment, and retires the grant when it is no longer needed. This is why service-integration key policies commonly allow kms:CreateGrant conditioned on kms:GrantIsForAWSResource — the condition restricts grant creation to AWS services acting for you, rather than letting a human principal mint delegations.

This is why the default key policy that the console writes for key users has two statements. The first allows the cryptographic operations (kms:Encrypt, kms:Decrypt, kms:ReEncrypt*, kms:GenerateDataKey*, kms:DescribeKey). The second allows kms:CreateGrant, kms:ListGrants and kms:RevokeGrant with kms:GrantIsForAWSResource set to true. A principal who launches an instance or attaches a volume encrypted under a customer managed key needs that second statement, because EC2 creates the grant on that principal's behalf; with only the first statement, the launch or attachment fails. The principal does not need retire permissions — the service retires its own grants when the resource no longer needs the key.

That points at the risk the exam wants you to see: kms:CreateGrant is a delegation right. Any principal holding it can create new access to the key for another principal, sidestepping your policy review. Audit it the way you audit iam:PassRole, scope it with kms:GrantIsForAWSResource where only services need it, and review existing grants with the list-grants APIs during investigations.

Operationally, grants are eventually consistent — a freshly created grant can take a short time to become usable everywhere, which is why the API returns a grant token that can be passed to requests to bridge the gap. Concept-level knowledge of that behavior is enough for the exam.

KMS never bulk-encrypts your data. The direct Encrypt API caps plaintext at 4 KB — enough for another key or a small secret, not for objects. Everything larger uses envelope encryption: call GenerateDataKey and KMS returns two things — a plaintext data key and the same data key encrypted under your KMS key. You encrypt the data locally with the plaintext key, discard that plaintext key from memory, and store the encrypted data key alongside the ciphertext. To read, you send the encrypted data key to Decrypt, receive the plaintext key back, and decrypt locally. Every SSE-KMS integration — S3, EBS, RDS — runs this flow under the hood, which is why revoking access to the KMS key revokes access to all data encrypted under it: without the key, the stored data keys are unrecoverable ciphertext.

Three related APIs complete the set. GenerateDataKeyWithoutPlaintext returns only the encrypted data key — for a component that provisions keys now but will encrypt later (it calls Decrypt when it needs the plaintext key), so no plaintext key exists before it is needed. ReEncrypt takes ciphertext and re-encrypts it under the same or a different KMS key inside KMS, so plaintext never leaves the service; it cannot encrypt plaintext you hand it. The AWS Encryption SDK packages the whole flow — data-key generation, local encryption, and storing the encrypted data key in the message — so client-side code does not hand-roll it.

Encryption context is the second mechanism you must know cold. It is a set of non-secret key-value pairs bound to a cryptographic operation as additional authenticated data (AAD): it is not encrypted, but for symmetric keys the exact same context must be supplied to decrypt. That yields two benefits. Cryptographic binding — S3, for example, uses the object's location as context, so an encrypted data key copied next to a different object will not decrypt. And audit — the context appears in every CloudTrail record for the KMS call, telling you precisely which resource a Decrypt served. You can also enforce it: the kms:EncryptionContext: condition keys in key policies restrict a principal to operations carrying specific context values, scoping one key safely across many datasets.

Rotation. Customer managed symmetric keys with KMS-generated material support optional automatic rotation on a period you choose between 90 and 2,560 days (365 by default), and you can also rotate on demand with RotateKeyOnDemand without changing the automatic schedule. AWS managed keys always rotate yearly and you cannot change that. Rotation swaps in new backing key material but retains all previous material so old ciphertexts still decrypt; the key ID and ARN never change, and nothing is ever re-encrypted. Rotation is therefore transparent to applications — and it does not retroactively protect data already encrypted: if key material were compromised, rotation alone does not fix the past. Remediating exposed material means re-encrypting the affected data with new data keys (for S3, copying the objects in place; for volumes and databases, a new encrypted copy), not just rotating. Keys with imported key material cannot auto-rotate; instead you import new key material into the same key and run on-demand rotation (RotateKeyOnDemand). Creating a new key and re-pointing the alias (manual rotation) is reserved for keys that support neither, such as asymmetric, HMAC and custom-key-store keys.

Multi-Region keys. KMS keys are Regional, which normally forces decrypt-and-re-encrypt when data crosses Regions. Multi-Region keys solve this: a primary key replicated into other Regions shares the same key ID and key material, so ciphertext encrypted in one Region decrypts in another without calling back to the original Region. They are replicas, not a global key: each has its own key policy and is managed independently.

The benefit is narrower than it sounds. It applies to client-side encryption — the AWS Encryption SDK, the AWS Database Encryption SDK, or your own envelope code — where a standby application in the second Region must decrypt data keys that were encrypted in the first. Most integrated AWS services, including S3 cross-Region replication, treat a multi-Region key as a single-Region key: they decrypt in the source Region and re-encrypt under the key you name for the destination. And interoperability comes only from the multi-Region relationship: two separate single-Region keys into which you imported identical material still cannot decrypt each other's ciphertext.

Deletion. You cannot delete a key immediately. ScheduleKeyDeletion imposes a pending window of 7 to 30 days (default 30) during which the key is unusable but recoverable via CancelKeyDeletion. Once deleted, every ciphertext under it — every object, snapshot, and data key — is permanently unrecoverable; that is what unrecoverable means here: the data, not just the key. The operational discipline is disable first: disable the key, watch CloudTrail for failing calls that reveal forgotten dependencies, and only then schedule deletion.

S3 offers four server-side modes, and the choice turns on who controls the key and who can read the data. S3 encrypts every new object by default with SSE-S3: AES-256 with keys S3 manages entirely. It is free and invisible — which is exactly its limitation: no key of yours to audit, no key policy to control, no cross-account key governance, and no CloudTrail evidence of key usage per object.

SSE-KMS switches the data-key source to a KMS key — aws/s3 or, for real control, a customer managed key. Now every object's data-key operation is authorized by the key policy and logged to CloudTrail, access can be revoked centrally by disabling the key, and cross-account bucket sharing can be governed at the key. It also adds a second, independent permission: even if a bucket policy accidentally allows public read, anonymous requests for SSE-KMS objects fail, while SSE-S3 objects are served decrypted. The cost is literal: each object operation calls KMS, which bills per request and consumes the KMS request quota — at high request rates, workloads start seeing throttling.

SSE-C keeps the key entirely with you: each upload and each read sends the key in request headers over HTTPS, S3 encrypts or decrypts and discards it, and lose the key and you lose the object. That makes sharing awkward — every reader, including another account or an AWS service, must be given the key itself — so it suits only workloads that already manage keys outside AWS. Since April 2026 S3 blocks SSE-C by default on new general purpose buckets (and on existing buckets in accounts with no SSE-C objects); a workload that needs it must unblock it in the bucket's default encryption configuration (BlockedEncryptionTypes).

S3 Bucket Keys are the classic optimization: S3 requests a short-lived bucket-level key from KMS and derives per-object keys from it locally, cutting KMS calls — and therefore both throttling and the KMS bill — dramatically. A KMS request-quota increase relieves throttling only; every request is still billed. Know three side effects. Bucket Keys apply only to objects written after you enable them; existing objects keep per-object keys until rewritten (for example with CopyObject or S3 Batch Operations). The encryption context and the CloudTrail resource change from the object ARN to the bucket ARN, so key policy or IAM conditions written against object ARNs in kms:EncryptionContext:aws:s3:arn must be updated. And CloudTrail shows far fewer per-object KMS events.

Client-side encryption (encrypt before upload, typically with the AWS Encryption SDK) is the remaining option: S3 stores ciphertext it cannot read, and key custody is entirely yours — see the key-service section above.

Enforcement. Default encryption applies only when a request carries no encryption headers — a client that sends its own x-amz-server-side-encryption-aws-kms-key-id overrides it. So set bucket default encryption to SSE-KMS with your chosen key (and Bucket Keys enabled), then add a bucket policy that denies s3:PutObject when the key does not match:

{
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:PutObject",
  "Resource": "arn:aws:s3:::example-bucket/*",
  "Condition": { "StringNotEquals": {
    "s3:x-amz-server-side-encryption-aws-kms-key-id": "arn:aws:kms:us-east-1:111122223333:key/<key-id>"
  } }
}

A method-only condition on s3:x-amz-server-side-encryption equal to aws:kms proves that some KMS key was used, not which one. Because a negated operator also matches when the header is absent, the key-ID statement above also rejects uploads that send no key ID and rely on the default — so clients must name the key explicitly, which is usually what an audit wants.

EBS and RDS fix encryption when a volume or database is created, so encrypting existing data always means creating an encrypted copy; DynamoDB is always encrypted and only the key choice changes.

EBS. Enable the account setting encryption by default and every new volume and snapshot copy is encrypted without asking — but the setting is per account and per Region, and it does nothing to volumes that already exist. Existing unencrypted volumes cannot be flipped in place: snapshot the volume, then create a new volume from that snapshot with encryption enabled under your key (copying the snapshot with encryption first is optional), and swap the new volume in. Sharing is where key choice bites: snapshots encrypted with the default aws/ebs key cannot be shared with another account at all. You must copy the snapshot re-encrypting under a customer managed key, share the copy, and grant the target account access in the key policy so it can decrypt and copy. Encrypted snapshots can never be made public.

RDS. Storage encryption is decided at instance creation and cannot be toggled later. You cannot create an encrypted read replica of an unencrypted instance, and you cannot encrypt a snapshot of an unencrypted instance except by copying it with encryption enabled — so the path is snapshot, copy the snapshot with encryption, restore a new instance. The same sharing rule applies: snapshots encrypted with the default aws/rds key cannot be shared, so copy under a customer managed key first. Because KMS keys are Regional, copying an encrypted snapshot cross-Region requires specifying a key in the destination Region; and an encrypted snapshot cannot be restored as an unencrypted instance.

DynamoDB is always encrypted at rest — the design decision is only which key: the AWS owned default (free, invisible), the aws/dynamodb managed key (key usage auditable in your CloudTrail), or a customer managed key (full policy control, revocable). You can switch between them on a live table with no downtime. Do not confuse two kinds of logging: CloudTrail data events for DynamoDB record item-level API calls such as GetItem and PutItem, whereas KMS key-usage events appear in your CloudTrail only when the table uses a KMS key in your account — the AWS owned default key never produces them.

Organization-wide enforcement combines a preventive and a detective layer. Preventively, an SCP denies creating unencrypted volumes when the ec2:Encrypted condition key is false — and it must cover both ec2:CreateVolume and ec2:RunInstances on volume resources, because launching an instance creates volumes through its block device mappings. Detectively, AWS Config rules such as encrypted-volumes (which evaluates attached volumes) and rds-storage-encrypted find what already exists — but a rule only evaluates the account it is deployed in, so deploy it to every member account as an organization Config rule or organization conformance pack from the management account or a delegated administrator, and feed findings to remediation.

When the threat is modification or deletion — ransomware, insider tampering, regulated retention — encryption is not the control; immutability is. Versioning keeps every version, Object Lock makes versions write-once-read-many (WORM) for a period or indefinitely, and MFA delete adds a second factor to permanent deletion.

S3 Object Lock requires versioning and protects versions, not keys. Each version can carry a retention period (a retain-until date) and, independently, a legal hold (no expiry; stays until someone with s3:PutObjectLegalHold removes it). Retention comes in two modes:

A bucket's default retention (mode plus days or years) stamps a retain-until date on each new version as it is written. You can enable Object Lock on an existing bucket, but it is irreversible: Object Lock can never be turned off and versioning can no longer be suspended. Default retention is not retroactive — versions already stored stay unprotected until retention is applied to them, typically in bulk with an S3 Batch Operations Object Lock retention job.

Know how deletes behave. A delete that names a version ID is a permanent delete, and S3 rejects it while that version is under retention or a legal hold. A simple delete that names only the key succeeds: S3 adds a delete marker as the current version, so listings hide the object, but the locked version is untouched and becomes noncurrent. Delete markers are not WORM-protected — removing the marker restores the object to view immediately.

MFA delete requires an MFA code to permanently delete a version or to change the bucket's versioning state. Only the bucket owner's root user can enable it, and only through the CLI or API (PutBucketVersioning with the MFA parameter) — the console has no option, and an IAM administrator cannot do it. It also cannot be used on a bucket that has an S3 Lifecycle configuration. For most designs Object Lock is the more practical tamper-resistance control.

S3 Glacier Vault Lock applies to Amazon Glacier (the original vault-based service, distinct from the S3 Glacier storage classes), which stopped accepting new customers on 15 December 2025 — new archives go to the S3 Glacier storage classes and are protected with S3 Object Lock. For existing vaults, locking is a two-step process: InitiateVaultLock attaches the lock policy in an in-progress state and returns a lock ID that expires after 24 hours; you test the policy, then CompleteVaultLock makes it immutable — or AbortVaultLock discards it so you can start again. A vault also keeps an ordinary vault access policy that stays editable after locking, which is where access that changes over time (an auditor's temporary read, for example) belongs; compliance controls such as minimum archive age go in the lock policy.

Integrity controls prove that code or data has not changed since a trusted point: digital signatures for code, checksums and signed digests for files.

Lambda code signing. AWS Signer signs a .zip deployment package with a signing profile; a code signing configuration attached to a function lists the allowed signing profiles and an UntrustedArtifactOnDeployment policy. At deployment Lambda checks integrity (the package matches its signature), expiry, mismatch (signed by an allowed profile) and revocation. The policy decides what happens when the expiry, mismatch or revocation checks fail: Warn, the default, deploys anyway and records the failure (a CloudWatch metric and CloudTrail); Enforce also blocks the deployment. When code signing is enabled, layers added to the function must be signed by an allowed profile too. Checks run only at deployment: attaching a configuration to a function whose current code is unsigned leaves that code running until the next deployment. Lambda code signing covers .zip packages, not container images.

Container images. AWS Signer signs images through the Notation CLI and its Signer plugin, storing signatures alongside the image in Amazon ECR, with no signing keys for you to manage. Verification has to happen where the image is used: on Amazon EKS an admission controller (for example one built on Ratify or Kyverno with a Signer trust policy) refuses pods whose images lack a valid signature from your profile. The principle is that verification belongs where an image is admitted to run, not where it is pushed: a signature check at admission holds no matter who could write to the registry. Vulnerability scanning answers a different question — whether an image has known flaws — and says nothing about who built it.

CloudTrail log file integrity validation delivers hourly digest files that hold SHA-256 hashes of each log file and are signed with SHA-256 with RSA; aws cloudtrail validate-logs proves whether log files were modified, deleted or are missing. Logging design is Task 1.2; here it is the file-validation example.

S3 additional checksums let S3 verify data on upload and store a checksum you can compare later (GetObjectAttributes or HeadObject). Supported algorithms are CRC64NVME (S3's default), CRC32, CRC32C, SHA-1 and SHA-256. A full-object checksum covers the whole file; a composite checksum is a checksum of the part checksums, so its value depends on part boundaries. For multipart uploads, full-object checksums exist only for the CRC algorithms; SHA-1 and SHA-256 produce composite checksums. Never treat a multipart upload's ETag as the file's MD5 — it is not. For objects already at rest, S3 Batch Operations can compute checksums in bulk.

Lifecycle controls move or delete data on a schedule so retention is enforced by configuration rather than by people; each service has its own mechanism.

S3 Lifecycle rules transition objects between storage classes and expire them, filtered by prefix, tags or size. On a versioned bucket the actions split: Expiration on current versions only adds a delete marker — every version remains and storage keeps growing — while NoncurrentVersionExpiration (with NoncurrentDays and optionally a number of newer versions to keep) actually removes noncurrent versions, and ExpiredObjectDeleteMarker cleans up delete markers left with no versions behind them. Lifecycle respects Object Lock: it never deletes a version that is still under retention or a legal hold, and deletes it once the retention has passed. Lifecycle cannot be combined with MFA delete on the same bucket.

Amazon EFS lifecycle policies tier files by access pattern: TransitionToIA moves files not accessed for a set period to Infrequent Access, TransitionToArchive moves them on to Archive, and TransitionToPrimaryStorageClass set to AFTER_1_ACCESS moves a file back to Standard when it is read. Without that last setting, a file that is accessed again stays in the cheaper class. Metadata always stays in Standard.

Amazon FSx for Lustre supports backups only on persistent file systems that are not linked to an S3 data repository. It takes automatic daily backups in a window you set, with a retention of 0 to 90 days (0 turns them off), and you can take user-initiated backups. Automatic daily backups are deleted with the file system; user-initiated backups, including those AWS Backup creates, are kept after the file system is deleted. AWS Backup can also manage FSx for Lustre backups alongside other services, under its plans' retention rules and vault controls.

A ransomware-resilient backup is one that an attacker holding the production account cannot delete or alter: copies kept in another account (or a service-owned one), protected by an immutable lock, and encrypted under keys the recovery side can use.

Amazon Data Lifecycle Manager (DLM) automates EBS snapshots and EBS-backed AMIs: snapshot and AMI policies target resources by tag, set schedules and retention, and can copy to other Regions. For a cross-account copy, the source account's snapshot policy shares the snapshots, and the target account creates a cross-account copy event policy that reacts to those share events and makes its own copy — sharing alone leaves the snapshot owned (and deletable) by the source. The event policy must be created in the same Region as the source snapshot policy; to land copies elsewhere, name other Regions as copy targets in it. Encrypted snapshots must use a customer managed key that the target account's role can use (unencrypted snapshots can also be copied), since aws/ebs snapshots cannot be shared.

AWS Backup centralizes backups across services with backup plans — rules for schedule, lifecycle (transition to cold storage, retention) and copy actions to vaults in other Regions or accounts — and resource assignments, usually by tag. Cross-account copy works only within one organization: enable it from the management account, and allow backup:CopyIntoBackupVault in the destination vault's access policy. Encryption decides what copies: resource types fully managed by AWS Backup (for example Amazon EFS) are encrypted with the vault's key and copy with any key type, whereas types that are not (for example EBS and RDS) keep the source key, so cross-account copies need a customer managed source key shared with the destination account.

AWS Backup Vault Lock makes a vault WORM. Governance mode can be removed by principals with the right IAM permissions. Compliance mode has a grace time (ChangeableForDays, at least 3 days) during which the lock can still be changed or deleted; after it, no one — not the root user, not AWS — can change or remove the lock or delete recovery points before their retention ends. A lock's minimum and maximum retention make any new backup or copy job whose retention falls outside the range fail; existing recovery points are unaffected. AWS Backup Audit Manager is a separate, reporting-only feature: it evaluates backups against compliance controls and reports the results, but enforces nothing.

Logically air-gapped vaults go further: they are always locked in compliance mode, store backups in a service-owned account, are encrypted with an AWS owned key by default (or a customer managed key), and are shared through AWS RAM — even with accounts in another organization — so a shared recovery account can run restores from the vault itself. Multi-party approval adds a way to regain access if the owning account is compromised or closed.

AWS DataSync is the secure online transfer service: an agent near on-premises storage moves data to S3, EFS or FSx with TLS in transit, the destination's at-rest encryption applies on arrival, and VerifyMode makes the service confirm integrity — checking only the transferred data (recommended), the whole dataset at the end, or not at all. AWS Transfer Family (SFTP and similar) also encrypts in transit but leaves any post-transfer comparison to you. AWS Snowball Edge is no longer available to new customers (since 7 November 2025); AWS points them to DataSync.

Work the scenario the exam loves: a Lambda function reads objects from an SSE-KMS bucket. It worked yesterday; today every GetObject fails with AccessDenied, even though the execution role clearly allows s3:GetObject and kms:Decrypt. Run the checklist in order.

1. Key policy before IAM. If the key lives in another account — or the key policy's delegation statement was tightened — the role's IAM allow is worthless: cross-account use needs the key policy to allow the external account and the caller's IAM policy to allow the key ARN. Check the key policy first, always.

2. Grants. Was access flowing through a grant that has been retired or revoked? List the key's grants; service-created grants disappearing explains sudden service failures.

3. Encryption context. If the key policy or IAM policy carries kms:EncryptionContext: conditions, a request whose context does not match is denied even though the action is allowed. Compare the context in the failing CloudTrail event against the condition values — CloudTrail records both the error and the context.

4. The network path. A Lambda in a VPC reaching KMS through an interface endpoint is also subject to the VPC endpoint policy; an endpoint policy that omits the key or principal denies the call regardless of every other policy. Key policies conditioned on aws:sourceVpce or kms:ViaService fail the same way when requests arrive by an unexpected route.

5. Key state. A disabled or pending-deletion key fails every operation — and cascades: EBS volumes fail to attach, SSE-KMS reads fail, RDS instances using the key become inaccessible. This is the everything-broke-at-once scenario; CloudTrail shows the failing Decrypt calls naming the key ARN. Re-enable the key or CancelKeyDeletion within the pending window — the reason the window and the disable-first discipline exist.

Separately, KMS throttling from high-volume SSE-KMS traffic is a quota problem, not a permissions problem. S3 Bucket Keys cut the request volume and so fix both throttling and cost; a quota increase fixes only the throttling.