Preparing Logs for Audit: CloudTrail, CloudTrail Lake, and CloudWatch Logs

AWS CloudTrail answers the auditor's first question — who did what, when, from where — by recording API calls across your account as events containing the caller identity, source IP, timestamp, request parameters, and response elements. Every account gets 90 days of management events searchable for free in the CloudTrail console's Event history. Anything beyond that window, and anything beyond management events, requires you to configure a trail or a CloudTrail Lake event data store.

Know the event taxonomy cold. Management events (control plane) record operations on resources: creating a Glue job, modifying a Redshift cluster, attaching an IAM policy. Data events (data plane) record operations on the data inside resources: S3 object-level GetObject and PutObject, Lambda function invocations, DynamoDB item-level actions. Data events are high-volume, disabled by default, and billed — so when a question asks how to find out who read or deleted a specific S3 object, the answer is that S3 data events must have been enabled on a trail beforehand; Event history alone will never show it. Insights events are a third type: CloudTrail Insights baselines your normal API rate and flags unusual bursts of activity, such as a sudden spike in delete calls.

A trail delivers events as compressed JSON files to an S3 bucket you choose, optionally also to a CloudWatch Logs log group for real-time alerting. Best practice — and the assumed exam default — is a multi-Region trail, so activity in any Region, including Regions you never use, lands in one place. From the S3 archive you can retain events for as many years as your compliance regime demands, at S3 storage prices, with lifecycle policies tiering old logs to Glacier storage classes.

An audit-ready log is complete, centralized, and provably unaltered. Completeness and centralization come from an organization trail: created in the AWS Organizations management account (or a delegated administrator account), it automatically logs every member account — including accounts created later — into one S3 bucket. Member accounts can see the trail but cannot modify or delete it, which is exactly the separation an auditor wants: the people generating the activity cannot switch off its recording.

Provable integrity comes from CloudTrail log file validation. When enabled, CloudTrail delivers a digest file on a regular cadence containing cryptographic hashes of each log file, signed by CloudTrail. Running the validation with the AWS CLI proves mathematically whether any log file was modified, deleted, or forged since delivery. Enable it on every trail — there is no cost and no downside, and its absence is a classic exam wrong-configuration detail.

Harden the destination bucket itself: enable S3 Object Lock in compliance mode on the logging bucket (or a replicated archive bucket) so log objects cannot be deleted or overwritten by anyone, including the root user, until the retention period expires — a WORM (write once, read many) guarantee regulators frequently require. Combine that with a dedicated log-archive account that no workload principals can access, MFA delete, SSE-KMS encryption of the log files with a tightly scoped key, and an S3 bucket policy that only allows the CloudTrail service principal to write. The standard multi-account pattern puts trails, AWS Config, and application log archives all in this one locked-down account.

Amazon CloudWatch Logs is the default home for application logs — the output of your Lambda functions, Glue jobs, Amazon MWAA DAGs, Step Functions executions, EMR steps, ECS containers, and anything on EC2 running the CloudWatch agent. Logs are organized into log groups (one per application or job, typically) containing log streams (one per instance, container, or execution). Lambda and Glue write here automatically; for audit purposes your job is mostly configuration, not plumbing.

Retention is the first setting to fix: log groups default to never expiring, which silently accumulates cost forever. Set a retention period per log group to match your audit requirement, and for multi-year retention export to S3 — either with the one-off export task or, better for continuous archival, a subscription filter streaming into Amazon Data Firehose which batches into S3. Storage in S3 with lifecycle tiering is far cheaper than keeping years of logs hot in CloudWatch.

Two filter mechanisms turn stored logs into signals. Metric filters match patterns in incoming log events and increment a CloudWatch metric — the classic audit example is a metric filter on CloudTrail logs (delivered to CloudWatch Logs) counting unauthorized API calls or root-account usage, with an alarm notifying security. Subscription filters stream matching log events in near real time to a destination: a Lambda function, a Kinesis data stream, Amazon Data Firehose, or onward to Amazon OpenSearch Service. Subscription filters are how one central security account aggregates application logs from many workload accounts, and they are the standard answer for "analyze logs in near real time" stems.

AWS CloudTrail Lake is a managed audit data lake that removes the trail-to-S3-to-Athena pipeline entirely: you create an event data store, choose which event types it ingests (management events, data events, AWS Config configuration items, even audit events from outside AWS via its integration APIs), set a retention period measured in years, and query it directly with SQL from the CloudTrail console or API. There is no S3 bucket to secure, no Glue table to define, no partitioning scheme to maintain.

Its audit-specific properties are the selling point. Ingested events are stored immutably — you cannot edit or delete individual events — and an event data store can be organization-scoped, capturing all member accounts like an organization trail does. Retention is configurable to multi-year periods suitable for compliance regimes, and because the store is queryable in place, answering an auditor's question about activity from two years ago is a single SQL statement rather than a restore-and-catalog exercise. CloudTrail Lake charges for ingestion and for the data each query scans, so it is priced for audit and investigation workloads, not as a general-purpose analytics store.

Choose CloudTrail Lake over the trail-plus-Athena pattern when the requirement emphasizes minimal setup, centralized multi-account queries, immutability out of the box, or ingesting non-AWS audit events into the same queryable store. Choose a traditional trail to S3 when you need the raw log files themselves (some regulators want the artifacts), want the absolute cheapest long-term archive, or want to join CloudTrail data against other datasets already in your S3 data lake with Athena.

Amazon Athena is the query engine for logs archived in S3. For CloudTrail, the console can generate the Athena table definition for your trail bucket in one click; from there you write standard SQL to answer questions like which principals called DeleteObject last quarter, or which source IPs used a given access key. The cost model is per data scanned, so the exam-relevant optimizations are the usual S3 analytics ones: partition the table (by account, Region, and date — partition projection avoids crawling thousands of prefixes), compress, and convert high-volume logs to a columnar format like Parquet with a Glue job if you query them heavily. Athena is equally the tool for VPC Flow Logs, ALB access logs, and S3 server access logs sitting in S3.

CloudWatch Logs Insights is the interactive query tool for data already in CloudWatch Logs — no table definitions, no export. Its purpose-built query language pipes commands together: filter events, parse fields, aggregate with stats, sort, and limit. It automatically discovers fields in JSON log events, can query many log groups at once, and is priced per data scanned. It is the right answer for operational and troubleshooting queries: find the error spike in last night's Glue job logs, count Lambda timeouts per function this week, trace a request ID across services.

The dividing line for the exam: where do the logs live, and how old are they? Logs in CloudWatch Logs within retention — Logs Insights. Logs archived to S3, or questions that join multiple datasets, or heavy scheduled reporting — Athena. API-activity questions with zero infrastructure setup — CloudTrail Lake. If a stem mentions building dashboards or full-text search rather than ad hoc SQL, that is the cue for OpenSearch, covered next.

Amazon OpenSearch Service fits when the requirement is near-real-time, interactive log exploration: full-text search across log lines, filterable dashboards in OpenSearch Dashboards, and alerting on patterns — the live security operations view, as opposed to Athena's retrospective SQL. Logs typically arrive via a CloudWatch Logs subscription filter or through Amazon Data Firehose, which batches, optionally transforms with Lambda, and delivers into the OpenSearch domain (with failed records diverted to S3 so nothing is silently lost). Because a hot OpenSearch cluster is comparatively expensive, real architectures keep a short window hot for interactive use and rely on the S3 archive plus Athena for history; UltraWarm and cold storage tiers extend retention on the OpenSearch side more cheaply.

Amazon EMR is the exam's stated answer for processing very large volumes of log data. When years of multi-terabyte log archives need sessionization, enrichment, joining against reference data, or reformatting into Parquet for cheaper downstream querying, a Spark job on EMR (or EMR Serverless) reads the raw logs from S3, transforms them at cluster scale, and writes curated output back to S3 — after which Athena queries become fast and cheap. EMR is a batch transformation engine in this story, not an interactive query tool; picking it for a quick one-off investigation is over-engineering, which the exam punishes.

A complete audit-logging architecture therefore layers the tools rather than choosing one: CloudTrail and application logs stream to durable S3 archives (immutable, validated), a recent slice flows to OpenSearch for live monitoring, CloudWatch metric filters raise alarms on security patterns, and Athena or CloudTrail Lake serve the deep historical queries — with EMR batch-processing the archive when scale demands it.

Most exam questions in this task reduce to matching a phrase in the scenario to one service. Use this table as the tie-breaker.

Worked scenario: an auditor asks who deleted objects from a regulated S3 bucket fourteen months ago, and your compliance rules require proving the logs were never altered. The correct chain is: S3 data events were enabled on an organization trail with log file validation, delivered to an Object Lock-protected bucket in the log-archive account; you either query the archive with Athena or, if an organization event data store exists, run one SQL query in CloudTrail Lake; and you run the CLI validation to demonstrate integrity. Any answer relying on Event history fails twice — it only spans 90 days and never included data events.