DVA-C03 Encryption and Sensitive Data: KMS, S3 Encryption, Certificates and Secrets

Encryption at rest protects data where it is stored (S3 objects, EBS volumes, database storage, logs); encryption in transit protects data while it moves between a client and a service, or between two services, using TLS. Most AWS storage services encrypt at rest with keys from AWS Key Management Service (AWS KMS), and the real design decision for a developer is usually which kind of key and who may use it, not whether to encrypt.

Choose a customer managed key when a requirement says the team must write the key policy itself, share encrypted data with another account, disable the key in an emergency to cut off access, or control the rotation schedule. An AWS managed key gives you an audit trail but no say over its policy; an AWS owned key gives you neither.

Disabling a customer managed key immediately stops every encrypt and decrypt with it, which makes the data it protects unreadable until the key is re-enabled. Scheduling a key for deletion starts a mandatory 7–30 day waiting period; once it is deleted, data encrypted under it is lost for good, so disabling is the reversible emergency switch.

Amazon S3 encrypts every new object at rest; you choose how through the bucket's default encryption setting or per-request headers. Since January 2023 the baseline is SSE-S3, so an object is never stored unencrypted, but SSE-S3 gives you no key policy and no per-use CloudTrail record of a KMS key.

S3 Bucket Keys

With SSE-KMS, S3 normally asks KMS for a data key for each object, so a bucket receiving millions of small objects generates millions of KMS requests: that costs money and can hit the account's KMS request quota, producing throttling errors. An S3 Bucket Key lets S3 create a short-lived bucket-level key from your KMS key and derive object keys from it locally, cutting KMS requests (and cost) dramatically while every object stays encrypted under the same KMS key. It is a checkbox in the bucket's default encryption settings — no code change. Because the number of KMS requests is what drives cost and throttling, a Bucket Key changes the bill and the request rate without changing which KMS key protects the data.

SSE-C on new buckets

Since April 2026, new general purpose buckets block SSE-C by default: their default encryption configuration lists SSE-C under blocked encryption types (BlockedEncryptionTypes in PutBucketEncryption), and any write that carries the customer-key headers fails with 403 AccessDenied, whatever the IAM policy allows. If an application genuinely needs SSE-C (for example keys held in an on-premises HSM), update the bucket's default encryption configuration to stop blocking it; adding IAM or bucket-policy allows does not help because the block is not a permissions problem.

Server-side encryption means the AWS service receives your plaintext (over TLS) and encrypts it before writing to disk; client-side encryption means your application encrypts the data before sending it, so the service only ever stores ciphertext. Choose client-side encryption when a requirement says that AWS, the storage service, or people with full permissions on that service (for example DynamoDB or S3 administrators) must never see the plaintext.

Server-side encryption with a customer managed key does not hide data from a principal who can read the table or bucket: the service decrypts transparently for anyone it authorises. Only client-side encryption plus tight control of kms:Decrypt separates "can read the data store" from "can read the data". The AWS Database Encryption SDK encrypts and signs chosen attributes per item, leaves the others in plaintext (so keys and indexes keep working) and offers searchable encryption through beacons.

Don't build your own cryptography

The AWS Encryption SDK implements envelope encryption, authenticated encryption (AES-GCM), key commitment and a portable message format for you. Calling KMS Encrypt directly is limited to 4 KB of plaintext. Writing envelope logic yourself on top of a low-level AES library means owning nonce handling, key commitment and the stored format; the SDK exists so application teams do not write cryptographic code.

Data key caching and fewer KMS calls

By default the Encryption SDK asks KMS for a new data key for each message. At thousands of messages a second that means thousands of GenerateDataKey calls and KMS throttling. Data key caching (a caching cryptographic materials manager) reuses a data key within limits you set — maximum age, maximum number of messages and maximum bytes — trading a little key reuse for far fewer KMS calls. Newer SDK versions offer the AWS KMS Hierarchical keyring for the same purpose. The limits are the point of the feature: they bound how much data, and for how long, any single data key protects.

Envelope encryption encrypts the data with a data key locally and then protects the data key with a KMS key, storing only the encrypted data key beside the ciphertext. It exists because the KMS Encrypt operation accepts at most 4 KB of plaintext and because sending bulk data to KMS for every operation would be slow and expensive. The KMS key itself never leaves KMS.

The flow

1. Encrypt: call GenerateDataKey (for example KeySpec='AES_256'), encrypt the data locally with the plaintext key, store the ciphertext together with the CiphertextBlob, and drop the plaintext key from memory.
2. Decrypt: read the stored CiphertextBlob, call KMS Decrypt to recover the plaintext data key (this needs kms:Decrypt on the KMS key), then decrypt the data locally.

key = kms.generate_data_key(KeyId='alias/app-data',
                            KeySpec='AES_256')
sealed = seal_locally(key['Plaintext'], document)
s3.put_object(Bucket=BUCKET, Key=name, Body=sealed,
              Metadata={'wrapped-key':
                  b64encode(key['CiphertextBlob']).decode()})
del key  # the plaintext key lives only in memory

The security of the scheme rests on one rule: store the encrypted data key, never the plaintext one. If the plaintext data key sits next to the data, anyone who can read the store can decrypt it, and KMS access control no longer matters. The point of envelope encryption is that reading the data requires a kms:Decrypt call that the key policy and IAM can allow or deny, and that CloudTrail records.

Access to a KMS key is decided by its key policy first: no principal can use a key unless the key policy allows it, either directly or through the default statement that lets the account's IAM policies grant access. With that default statement in place, a role in the same account needs an IAM policy allowing the specific KMS action on the key's ARN.

The permissions SSE-KMS needs

S3 calls KMS with the caller's identity, so an S3 permission alone never reads or writes an SSE-KMS object: KMS authorises the caller separately against that object's key. SSE-S3 objects involve no KMS key and need no KMS permission. The same applies to DynamoDB, SQS, SNS and other services that call KMS on your behalf: the caller's identity needs the KMS permission too. The condition key kms:ViaService can limit a permission to requests that arrive through a named service (for example only through S3).

Cross-account use needs both sides

For a principal in account B to use a key in account A, both of these must allow it:

• the key policy in account A must allow account B (or the specific role) the action, for example kms:Decrypt; and
• an IAM policy in account B must allow the role that action on the key's full key ARN. KMS authorises cross-account use against the key ARN, not an alias.

Because an AWS managed key's policy cannot be edited, objects encrypted with aws/s3 cannot be decrypted from another account. To share them, re-encrypt them (for example with a copy-in-place) under a customer managed key whose key policy admits the other account. Grants are a third mechanism: programmatic, scoped permissions, often created by AWS services (for example EBS for an attached volume). A grant can also name a principal in another account as grantee (CreateGrant), and it lasts until it is retired or revoked.

Encryption context: binding ciphertext to its purpose

An encryption context is a set of non-secret key–value pairs (for example {"tenant": "t-42"}) passed with a symmetric KMS encryption request and cryptographically bound to the ciphertext as additional authenticated data. Decryption succeeds only when the caller supplies exactly the same encryption context; a missing or different pair makes Decrypt fail with InvalidCiphertextException, even when IAM allows kms:Decrypt. The fix is to pass the original pairs again, not to change permissions or the algorithm.

• The context is not secret: it appears in plaintext in CloudTrail logs, which makes it useful for auditing which record or tenant a decryption was for. Never put sensitive data in it.
• It works as an authorisation input through policy conditions:
  • kms:EncryptionContext:key — requires a specific value for a specific key, for example allowing a tenant's worker role to decrypt only when kms:EncryptionContext:tenant equals its own tenant ID. This gives tenant isolation on one shared KMS key.
  • kms:EncryptionContextKeys — only checks which keys are present, not their values, so it cannot stop one tenant decrypting another's data.

The AWS Encryption SDK and AWS Database Encryption SDK pass an encryption context for you and store it in their message format, so you get the binding without managing it by hand.

Applications should reference KMS keys by alias (for example alias/app-data) rather than by hard-coded key ID or ARN. An alias is account- and Region-scoped, so the same name can point at a different key in development and production, and UpdateAlias repoints it to a new key without a redeploy. Aliases work for cryptographic operations in your own account; cross-account callers must use the key ARN.

Rotation

Manual rotation (new key + alias) changes the key ARN, which breaks anything that references the ARN, such as key policies or grants in partner accounts.

Multi-Region keys

A multi-Region key is a set of related keys in different Regions that share the same key ID and key material, so ciphertext produced in one Region can be decrypted in another without a cross-Region call. They pay off for client-side encryption (Encryption SDK, Database Encryption SDK) of data that moves between Regions. They do not share a key across accounts, and S3 cross-Region replication treats them as single-Region keys and re-encrypts objects in the destination Region.

Each storage service has its own rule for when encryption at rest can be switched on, and that rule decides the steps a developer takes.

For DynamoDB, a customer managed key is the only option that lets your team write the key policy and disable the key to cut off access; aws/dynamodb is logged but its policy cannot be edited.

Encryption in transit

• Amazon S3: a bucket policy statement that denies all S3 actions when aws:SecureTransport is false rejects every plain-HTTP request from any client, without touching anyone's IAM permissions.
• Load balancers: an ALB HTTPS listener with an ACM certificate encrypts client → ALB. For encryption on every hop, set the target group protocol to HTTPS and serve TLS on the targets with a certificate installed on each instance (the ALB does not validate target certificates, so a self-signed or private certificate works there). A Network Load Balancer with TCP listeners passes TLS through to the targets, which then terminate it themselves.
• Databases: make the server require TLS — require_secure_transport=ON for RDS for MySQL/MariaDB, rds.force_ssl=1 for RDS for PostgreSQL, set in a custom DB parameter group — and make clients verify the server certificate against the RDS CA bundle (for example MySQL ssl-mode=VERIFY_IDENTITY, PostgreSQL sslmode=verify-full). REQUIRED/require encrypts without checking who answered, so a spoofed endpoint goes unnoticed.
• AWS APIs: SDKs and the CLI call HTTPS endpoints by default; FIPS endpoints are for FIPS 140-validated cryptography, not a substitute for client-side encryption.

AWS Certificate Manager (ACM) issues and renews TLS certificates for AWS services, and AWS Private CA runs a private certificate authority hierarchy you own. Pick by who must trust the certificate and where it is installed.

Rules that decide ACM questions

• ACM certificates are Regional. A load balancer or API uses a certificate in its own Region, but Amazon CloudFront only uses certificates from ACM in us-east-1 (N. Virginia); a certificate in any other Region never appears in the distribution's list. Request (or import) one in us-east-1.
• Public certificates need a publicly resolvable domain you can validate. Internal names that exist only in a private hosted zone (for example api.corp.internal) need a private certificate.
• To stop an imported certificate from causing an outage, replace it with an ACM-issued certificate that is DNS-validated, so renewal is automatic from then on.

AWS Private CA

Use AWS Private CA when you need your own CA hierarchy for internal names or mutual TLS (clients presenting certificates too) without running CA software. A general-purpose CA issues certificates of any validity and supports revocation through CRLs (published to S3) and OCSP. A CA in short-lived certificate mode issues certificates valid for at most seven days, at a lower monthly and per-certificate price; it suits workloads that request a fresh certificate per job and simply let it expire instead of revoking it.

Certificates and SSH keys for development

• EC2 key pairs: aws ec2 create-key-pair (RSA or ED25519) returns the private key once; AWS keeps only the public key, so a lost private key cannot be downloaded again. Create a new key pair (or generate one locally with ssh-keygen and register it with import-key-pair) and add its public key to the instance's ~/.ssh/authorized_keys, for example through EC2 Instance Connect, Session Manager or by attaching the root volume to another instance.
• Self-signed certificates: for a test HTTPS endpoint that has no public domain and is called only by clients you control, generate one with OpenSSL (for example openssl req -x509 -newkey rsa:2048 -nodes -days 30 …) and add it to those clients' trust stores. It costs nothing; it must never face real users.

AWS splits sensitive-data protection into three jobs: find it where it is stored, mask it where it flows (logs, messages), and limit who can read it. Personally identifiable information (PII) and protected health information (PHI) are the usual examples.

CloudWatch Logs masking rules

• Masking applies only to events ingested after the policy is in place. Older events keep their plaintext; if they must go, delete those log streams (or let retention expire them).
• Readers see masked values by default. Principals granted logs:Unmask on the log group can view the original values, so specific engineers can see real data during incidents while everyone else sees it masked.
• It is configuration on the log group, so applications keep logging as before. KMS encryption of a log group is a separate control: it protects the stored data, while a data protection policy controls what authorised readers see.

Acting on Macie findings

Macie publishes findings to Amazon EventBridge (and AWS Security Hub). An EventBridge rule that targets a Lambda function or ticketing workflow is the way to automate a response. To restrict access to objects Macie flags without moving them or editing every role, have that function tag each flagged object (for example classification=phi) and add a bucket policy that denies s3:GetObject when s3:ExistingObjectTag/classification is phi, except for the compliance role. s3:ExistingObjectTag checks tags already on the object; s3:RequestObjectTag checks tags sent in the current request, so it governs which tags a write such as PutObject or PutObjectTagging may set.

Beyond these services, limit access with least-privilege IAM, separate buckets or tables for PII/PHI stores, client-side encryption of the most sensitive attributes and kms:Decrypt granted only to the components that need plaintext. Redacting PII from prompts and responses of AI models is covered in the lesson on securing AI services (Task 2.3).

AWS Secrets Manager stores secrets encrypted with KMS and can rotate them on a schedule; Systems Manager Parameter Store stores configuration and SecureString parameters encrypted with KMS but has no built-in rotation. Credentials that must change automatically (database passwords, API keys with a rotation process) belong in Secrets Manager; plain configuration and static encrypted values fit Parameter Store. Plain environment variables in a task definition or function configuration are not a secret store.

Retrieving secrets efficiently

• Lambda: calling GetSecretValue on every invocation adds latency, cost and throttling. The AWS Parameters and Secrets Lambda Extension runs a local HTTP cache (on localhost:2773) with a configurable TTL, so the function reads from memory and picks up a rotated value when the TTL expires — no custom caching code.
• Amazon ECS: reference the secret or parameter ARN in the container definition's secrets section; the task execution role needs secretsmanager:GetSecretValue (or ssm:GetParameters) and kms:Decrypt for a customer managed key. The value is injected when the container starts, so a running task sees a rotated value only after it is restarted.
• Other code: the AWS SDK secret caching clients do the same job in-process.

Rotation

• Managed rotation needs no Lambda function. The simplest case: when creating or modifying an RDS DB instance or Aurora cluster, let RDS manage the master user password in Secrets Manager; RDS generates it, stores it encrypted and rotates it, and nobody types it anywhere. Managed external secrets extend managed rotation to credentials of onboarded third-party SaaS partners; other third-party secrets still need a custom rotation function.
• Rotation functions run four steps — createSecret, setSecret, testSecret, finishSecret — moving the staging labels AWSPENDING, AWSCURRENT and AWSPREVIOUS between versions. AWS provides templates for RDS, Aurora, Redshift and DocumentDB.
• Single user strategy changes the one user's password in place: clients still holding the old password fail to log in until they re-read the secret. Alternating users strategy keeps two users and switches between them, so the previous credentials stay valid until the next rotation; it needs a separate superuser (admin) secret that can update the other user. Choose it for long-lived connection pools and cached credentials where no login may fail.
• Networking: a rotation function in private subnets must reach both the Secrets Manager API — through a Secrets Manager interface VPC endpoint or a NAT gateway — and the database, which means the database's security group must allow inbound traffic on its port from the function's security group. A Lambda function attached to a VPC has no internet access of its own, so these paths must exist inside the VPC.