Data Encryption and Masking on AWS: KMS, TLS, and Anonymization

AWS KMS protects data through envelope encryption: KMS never encrypts your gigabytes of data directly with the KMS key. A service (or your code) calls GenerateDataKey, receives a unique data key in both plaintext and encrypted form, encrypts the object or block locally with the plaintext data key, discards the plaintext key and stores the encrypted data key alongside the ciphertext. To decrypt, it sends the encrypted data key back to KMS, which returns the plaintext data key so the data can be decrypted in memory.

The direct Encrypt API exists, but it accepts at most 4,096 bytes of plaintext — enough for a password or a small secret, never for a file or a table. Anything larger goes through a data key.

Two consequences matter. First, the KMS key never leaves KMS unencrypted — every use is mediated by the key policy and IAM and logged to AWS CloudTrail. Second, KMS is called once per data key rather than once per byte, and features such as S3 Bucket Keys cut that further. Revoking access is equally cheap: disable a customer managed key or remove a principal's kms:Decrypt permission and every data key wrapped by that KMS key becomes unreadable to that principal — nothing has to be re-encrypted.

Key rotation works the same way in reverse: rotating a KMS key (automatic or on demand) adds new key material for future encryptions while KMS keeps the old material to decrypt existing data keys. Rotation re-encrypts no data. If a requirement says existing objects must be protected by new material or a different key, you must rewrite or re-encrypt the data yourself.

Choose a customer managed key whenever a scenario demands control: editing the key policy, sharing the key across accounts, disabling or deleting it on your schedule, or proving auditable control over rotation. Choose the AWS managed key (aws/s3, aws/redshift, aws/rds and so on) when you want KMS-backed encryption with CloudTrail visibility but no management burden. AWS owned keys are used by AWS on your behalf without appearing in your account at all — DynamoDB uses one by default.

The most tested distinction: an AWS managed key can never be used by another AWS account. Its key policy is written by AWS, cannot be edited, and limits use to callers in the owning account through the owning service. Cross-account access to encrypted S3 objects, Redshift snapshots, RDS snapshots or EBS volumes therefore requires a customer managed key.

Where the key material lives

Customer managed keys offer three variations, and custody requirements decide between them:

• Imported key material (BYOK) — you generate the material in your own key manager and import it, so you control its origin and can delete it on demand. It is still stored and used inside AWS KMS, so it does not satisfy a rule that AWS must never store your keys.
• External key store (XKS) — the KMS key is backed by key material that stays in your own external key manager outside AWS; KMS calls out to it for every cryptographic operation. It keeps material outside AWS while still working with SSE-KMS, at the cost of running a highly available external key manager that AWS can reach.
• Multi-Region keys — related keys in different Regions that share key material and key ID, so ciphertext produced in one Region can be decrypted in another without a cross-Region call. A key must be created as multi-Region from the start: an existing single-Region key cannot be converted, and ReplicateKey works only on a multi-Region primary key. Multi-Region keys pay off with client-side encryption (AWS Encryption SDK, S3 Encryption Client), where an application in the second Region must decrypt the same ciphertext. S3 Cross-Region Replication treats them as single-Region keys and re-encrypts replicas under the destination key anyway.

Access to a customer managed key is decided by its key policy together with IAM policies. The key policy is the primary, resource-based control — a KMS key does not implicitly trust even its own account. The default key policy contains a statement allowing the account root principal, which delegates permission decisions to IAM; remove it and IAM policies alone can no longer grant access to the key.

Separate the two permission planes. Key administrators get kms:PutKeyPolicy, kms:ScheduleKeyDeletion, kms:EnableKey and similar actions — they manage the key but should not necessarily be able to decrypt with it. Key users get kms:Encrypt, kms:Decrypt, kms:GenerateDataKey* and kms:DescribeKey. Least-privilege designs keep these roles distinct.

Grants are the third mechanism: programmatic permissions for one grantee principal and a set of operations, which AWS services such as Amazon EBS and Amazon Redshift create to use your key asynchronously. A grant is an independent authorization path — it works even when the key policy does not enable IAM policies — and it is removed by RetireGrant (typically by the grantee or retiring principal when the work is done) or RevokeGrant (by a principal with permission on the key). Grants let you give and take away narrowly scoped key use without editing the key policy.

Condition keys that scope key use

So when a requirement says a key may be used only through S3 (or only through Redshift), the control is kms:ViaService. Encryption-context conditions are about binding ciphertext to its purpose and auditing, not about which service is calling.

Amazon S3 applies exactly one server-side encryption mode to each object, and since January 2023 every new object is encrypted by default with SSE-S3. Pick the mode by who must control the key and how many layers are required:

Modes cannot be stacked: an object is SSE-S3 or SSE-KMS, never both, and S3 Bucket Keys or S3 Versioning add no encryption layer. S3 Bucket Keys are a cost optimization for SSE-KMS — S3 uses a short-lived bucket-level key derived from your KMS key for many objects, cutting KMS request volume and cost on busy buckets, with fewer per-object KMS events in CloudTrail as the trade-off.

SSE-C needs one more step today: since April 2026, S3 blocks SSE-C by default on new general purpose buckets (and on existing buckets in accounts that hold no SSE-C objects), and SSE-C requests get an access-denied error until you allow the type through the bucket's default-encryption configuration (PutBucketEncryption with BlockedEncryptionTypes).

Changing the encryption of objects that already exist

Default encryption is forward-only: changing the bucket default affects new writes and leaves existing objects as they were. To change existing objects:

• SSE-S3 to SSE-KMS — the S3 UpdateObjectEncryption operation (also available as an S3 Batch Operations job) changes the encryption in place without copying the data. It fails on objects under an S3 Object Lock retention period or legal hold.
• Unencrypted, SSE-C or DSSE-KMS objects — UpdateObjectEncryption does not support them, so they must be rewritten, at scale with an S3 Batch Operations copy job that copies each object onto itself with the new encryption settings (a copy job also covers any SSE-S3 objects in the same inventory).

Requiring encryption on write is a bucket-policy deny on s3:PutObject when the s3:x-amz-server-side-encryption header is missing or not the expected value. Scope that deny to PutObject: GET requests do not send the encryption header, so a deny on s3:* or s3:GetObject conditioned on that header blocks every read.

Each data service integrates with KMS, but the knobs live in different places.

Amazon Redshift encryption is set at the cluster (or serverless namespace) level with KMS; data blocks, system metadata and snapshots are encrypted, and snapshots inherit the cluster's key. New provisioned clusters are encrypted by default since January 2025. Sharing an encrypted snapshot with another account requires a customer managed key that the other account may use. DynamoDB is always encrypted at rest; your only choice is the key — AWS owned (default, free, invisible), the AWS managed aws/dynamodb key, or a customer managed key when auditors require key control. Backups and streams are covered too.

AWS Glue security configurations set encryption for what a job produces besides its data: S3 output (SSE-S3 or SSE-KMS), CloudWatch Logs, and job bookmarks. You attach the security configuration to the job (or crawler) as a property — no script change. When CloudWatch Logs encryption is on, the KMS key policy must allow the CloudWatch Logs service principal (logs.<region>.amazonaws.com) to use the key, and the job role needs KMS permissions as well. Data Catalog metadata and connection passwords are encrypted through the separate Data Catalog encryption settings.

Amazon EMR security configurations hold at-rest settings (EMRFS data in S3, and local disks including EBS volumes) and in-transit TLS settings in one reusable object that clusters reference at launch. Account-level EBS encryption by default is not a substitute: it encrypts the EBS volumes attached to EMR nodes but not EMRFS data in S3, not instance store volumes, and not traffic between nodes. When a requirement spans storage and network, the EMR security configuration is the complete control. Amazon Kinesis Data Streams offers server-side encryption with KMS so records are encrypted as they are written to the stream.

Using a KMS key across account boundaries always takes two permissions: the key policy in the key's account must allow the external account (or specific external principals) to use the key, and an IAM policy in the consumer account must grant its principals the matching kms:Decrypt or kms:GenerateDataKey permissions on the key. Neither alone is sufficient — a key policy that trusts account B does nothing until an admin in account B delegates that access with IAM.

The canonical scenario: a central data lake account holds an SSE-KMS bucket under a customer managed key, and an analytics account must query it with Amazon Athena. You need a bucket policy allowing the analytics account to read the objects, a key policy statement allowing it to decrypt with the key, and IAM permissions in the analytics account for both. Miss the key policy and every GetObject fails with an access-denied error from KMS even though the S3 permissions look perfect.

Things that look like they grant key access but do not

• An alias ARN in an IAM policy's Resource element — KMS authorizes cryptographic operations against the key, not the alias, so the IAM statement must name the key ARN (or use the kms:ResourceAliases condition). An alias also resolves only within its own account and Region.
• A permissions boundary — it only caps what identity policies may grant; on its own it grants nothing.
• kms: actions in an S3 bucket policy — KMS ignores them; only the key policy, IAM policies and grants govern a KMS key.
• The aws/s3 key (or any AWS managed key) — it can never be used from another account; switch to a customer managed key.

Snapshots and replication

An RDS snapshot encrypted with the AWS managed aws/rds key cannot be shared. Copy the snapshot, re-encrypting it with a customer managed key, add the target account to that key's policy, then share the copy; the target account (with IAM permission on the key) can copy or restore it. Redshift follows the same rule: an encrypted snapshot can be shared only when it uses a customer managed key the target account may use. For S3 replication between accounts with SSE-KMS, the replication role must be able to decrypt with the source key and encrypt with the destination key, and the destination key policy must permit that role.

Encryption in transit on AWS means TLS. Every AWS API endpoint supports HTTPS, so the work is refusing everything that is not TLS.

Amazon S3: deny every action when aws:SecureTransport is false. Where you put the deny matters. In the bucket policy it applies to every request against the bucket — current, future and cross-account principals alike. In an identity policy it binds only the users and roles it is attached to, so a new role or a principal in another account is not covered. Enforcement that must hold for everyone belongs in the bucket policy.

Amazon Redshift: the client and the server each have a setting, and only one enforces anything.

Since January 2025, new and restored provisioned clusters use a default parameter group with require_ssl set to true. Older clusters on a custom parameter group keep whatever value it holds, so they may still accept unencrypted connections until you change it. For Amazon RDS and Aurora, enforcement is per engine (for example PostgreSQL's rds.force_ssl), and clients that must verify the server's identity validate its certificate against the AWS-published RDS certificate bundle.

Distributed frameworks also need TLS between their own nodes. An Amazon EMR security configuration enables in-transit encryption with TLS certificates you provide, covering node-to-node traffic such as Spark shuffle, and every cluster launched with that configuration gets it. Amazon MSK encrypts traffic between brokers and can require TLS from clients. AWS Glue connections to JDBC sources can require SSL in the connection definition.

Encrypting before transit is the stricter posture: the client encrypts data before it leaves the producing system, so the network and the service only ever carry ciphertext. The AWS Encryption SDK and the Amazon S3 Encryption Client implement this with KMS-backed data keys, and the AWS Database Encryption SDK does the same for DynamoDB. Choose it when a regulator or contract says AWS must never receive plaintext — every server-side mode, including SSE-C, encrypts data after S3 has received it.

Masking protects data that must stay queryable. Encryption is all-or-nothing per principal; masking lets one dataset serve users who may see everything and users who must never see raw PII. The choice comes down to where the data lives and whether a hidden value must disappear, appear transformed, or be removed before it lands.

The Detect Sensitive Data transform (formerly called Detect PII) finds managed or custom entity types in a Glue job and applies an action to each detection: REDACT, PARTIAL_REDACT or SHA256_HASH. Because it acts in the pipeline, raw values never reach the target store — the right fit when a rule says PII must not land in the lake at all. Lake Formation filters and Redshift masking act after landing, at read time, over the single stored copy. AWS Glue DataBrew offers similar redaction, substitution and hashing for data preparation. Amazon Macie is discovery, not masking: it finds where PII sits in S3 (covered with data privacy in Task 4.5).

De-identification techniques differ by one question: can anyone, including the data owner, get back to the person? If yes, the data is pseudonymized and remains personal data under laws such as GDPR; only data that can no longer be linked to an individual by any reasonably likely means counts as anonymized.

• Masking hides part of a value for display (****1234).
• Tokenization replaces a value with a surrogate token and keeps the mapping in a separate secure vault. It is reversible through the vault and tokens still join across tables — popular for payment data, and pseudonymization by design.
• Encryption is reversible by anyone with the key — also pseudonymization.
• Deterministic hashing (an unsalted or fixed-key hash of an email or customer ID) is one-way but repeatable. That makes it useful for joining datasets without exposing the identifier — and that same property is why it is pseudonymization, not anonymization: whoever holds a list of known identifiers can hash them and link every row back to a person.
• Generalization and suppression address quasi-identifiers — fields such as birth date, postcode and gender that are harmless alone but identify people in combination. Replace them with coarser values (birth year or age band, a postcode prefix) or drop them, so no small group of records stands out.
• Anonymization is the end state: direct identifiers removed (or replaced with random values whose mapping is destroyed) and quasi-identifiers generalized, so re-identification is no longer possible. Truly anonymized data generally falls outside GDPR, which is why regulators set a high bar for the label.

So when a requirement says the dataset must stay joinable or recoverable, you are choosing a pseudonymization technique and the data stays in scope; when it says nobody may ever re-identify individuals, hashing identifiers alone is not enough — remove them and generalize the quasi-identifiers.

Match the requirement phrase to the control. "Prove who used the key and when" or "revoke a partner's access immediately" points to a customer managed KMS key and its key policy. "Minimize cost and management overhead for encryption at rest" points to default SSE-S3 or an AWS owned key. "Share encrypted data with another account" means a customer managed key plus a two-sided permission setup. "Reduce KMS request charges on a high-throughput bucket" is S3 Bucket Keys. "Two layers of encryption" is DSSE-KMS. "AWS must never store the key" is SSE-C (or an external key store), not imported key material. "AWS must never see plaintext" is client-side encryption before transit. "Only through S3" is kms:ViaService.

For masking: "analysts see a partial value in Redshift" is dynamic data masking; "the column must be invisible to them" is a column-level GRANT or a Lake Formation column filter; "only their region's rows" is a Lake Formation row filter; "PII must never land in the lake" is Glue Detect Sensitive Data; "reversible surrogates that preserve joins" is tokenization; "nobody can re-identify anyone" is anonymization with generalized quasi-identifiers.

Watch for stems that mix layers. Encryption at rest does not protect data from an authorized user who can query it — that is masking's job. TLS does not protect stored data — that is at-rest encryption's job. A complete compliance answer usually stacks all three: KMS at rest, TLS in transit, and masking or filters at the access layer.