Logging solutions: CloudTrail, CloudWatch Logs, Security Lake and network logs

Log sources and where they go

Day 1 and Day 2 used services that read logs: GuardDuty, Security Hub and Config. Today is about the logs themselves. Skill 1.2.1 asks you to pick the right log source for a requirement and the right place to store it. A detection is only as good as the log behind it, and you can’t turn on a log after an incident to see what happened before.

Most AWS logs land in one of three targets: CloudWatch Logs, Amazon S3 or Amazon Data Firehose. In a multi-account setup, they then flow to a central Log Archive account that workload teams can’t change.

$ aws cloudtrail describe-trails --query "trailList[].[Name,IsMultiRegionTrail,IsOrganizationTrail]"
[["org-trail", true, true]]

$ aws ec2 describe-flow-logs --query "FlowLogs[].[ResourceId,TrafficType,LogDestinationType]"
[]

$ aws route53resolver list-resolver-query-log-configs --query "ResolverQueryLogConfigs[].Name"
[]

CloudTrail event types and trails

CloudTrail records API activity. It has four event types, and the exam loves to ask which one is needed and which are on by default. Only management events are logged by default; data, network activity and Insights events must be turned on and cost extra.

Advanced event selectors

Advanced event selectors are the modern way to choose what a trail or event data store records. Each selector is a list of field conditions such as eventCategory, resources.type, resources.ARN, eventName and readOnly. They let you log, for example, only write data events on one bucket, which keeps cost low. A multi-Region trail (the console default) records all enabled Regions; a single-Region trail can only be made with the CLI or API.

$ aws cloudtrail put-event-selectors --trail-name org-trail \
  --advanced-event-selectors '[
   { "Name": "Mgmt events",
     "FieldSelectors": [ {"Field":"eventCategory","Equals":["Management"]} ] },
   { "Name": "S3 VPCE denials",
     "FieldSelectors": [
       {"Field":"eventCategory","Equals":["NetworkActivity"]},
       {"Field":"eventSource","Equals":["s3.amazonaws.com"]},
       {"Field":"errorCode","Equals":["VpceAccessDenied"]} ] } ]'

Organization trail, Log Archive and integrity

Skill 1.2.2 names the first big setup step: an organization trail. It logs every member account into one S3 bucket, usually in a dedicated Log Archive account (the AWS Security Reference Architecture and Control Tower both use this pattern). Then you protect that bucket so logs can’t be changed or deleted, and you prove they weren’t with log file validation.

Protecting the Log Archive bucket with Object Lock

S3 Object Lock stores objects in a write-once-read-many (WORM) way and needs versioning. In governance mode, users with s3:BypassGovernanceRetention can still delete or shorten retention, which is good for testing. In compliance mode, no one, not even the root user, can delete a locked version or shorten its retention until it ends. A legal hold blocks deletion with no end date until someone with s3:PutObjectLegalHold removes it. Add an SCP that denies cloudtrail:StopLogging and cloudtrail:DeleteTrail as a second guard.

$ aws cloudtrail validate-logs \
    --trail-arn arn:aws:cloudtrail:us-east-1:111122223333:trail/org-trail \
    --start-time 2026-10-01T00:00:00Z --end-time 2026-10-02T00:00:00Z

Validating log files for trail arn:aws:cloudtrail:us-east-1:111122223333:trail/org-trail
Results requested for 2026-10-01T00:00:00Z to 2026-10-02T00:00:00Z
Results found for 2026-10-01T00:00:00Z to 2026-10-02T00:00:00Z:

24/24 digest files valid
1340/1341 log files valid, 1/1341 log files INVALID

CloudWatch Logs: agent, central account and subscriptions

The second example in skill 1.2.2 is a dedicated CloudWatch logging account, and the third is the CloudWatch Logs agent. CloudWatch Logs is where you search and alarm on logs quickly. The exam asks how logs get in (agent, service integrations) and how they get to one central account.

Three ways to centralize logs in a logging account

1. Subscription filters. A log group can have up to 5 subscription filters (older material says 2). A filter streams matching events in real time to Kinesis Data Streams, Firehose, Lambda or OpenSearch in the same account. For cross-account, the central account creates a CloudWatch Logs destination that points to a Kinesis stream or Firehose stream, plus a destination access policy that lets source accounts call logs:PutSubscriptionFilter. The log group and the destination must be in the same Region. An account-level subscription filter policy covers many log groups at once.

2. Organization log centralization. A newer CloudWatch feature: the management account or a delegated administrator creates centralization rules (with trusted access turned on) that copy new log events from chosen accounts, OUs and Regions into one destination account, with an optional backup Region. Only data that arrives after the rule is created is copied. Name patterns such as ${source.accountId} keep accounts apart.

3. Cross-account observability. A monitoring account links to source accounts so people can view and query their logs, metrics and traces from one console. The data is not copied; it stays in each source account.

$ aws logs put-destination --destination-name central-sec \
    --target-arn arn:aws:firehose:us-east-1:111122223333:deliverystream/to-s3 \
    --role-arn arn:aws:iam::111122223333:role/CWLtoFirehose

$ aws logs put-destination-policy --destination-name central-sec --access-policy '{
  "Version":"2012-10-17","Statement":[{"Effect":"Allow",
   "Principal":{"AWS":"444455556666"},"Action":"logs:PutSubscriptionFilter",
   "Resource":"arn:aws:logs:us-east-1:111122223333:destination:central-sec"}]}'

# in the source account, same Region
$ aws logs put-subscription-filter --log-group-name /app/prod \
    --filter-name to-central --filter-pattern "" \
    --destination-arn arn:aws:logs:us-east-1:111122223333:destination:central-sec

Security Lake, CloudTrail Lake and third-party tools

Skill 1.2.3 asks you to build log storage and data lakes and connect third-party security tools. The AWS answer is Amazon Security Lake: it collects security logs from many accounts and Regions, converts them to one schema (OCSF) and stores them in an S3 data lake that you own. Partners and SIEMs then read from it as subscribers.

CloudTrail Lake event data stores

A CloudTrail Lake event data store keeps events in a managed, immutable store and lets you run SQL (Trino) queries over them, join several stores, and use dashboards. One store holds one kind of data: CloudTrail events, Insights events, AWS Config items, Audit Manager evidence or events from outside AWS sent with PutAuditEvents. An organization event data store covers all member accounts. Retention is up to 3,653 days with one-year extendable pricing or 2,557 days with seven-year pricing. You can federate a store with the Glue Data Catalog to query it in Athena. If you add your own KMS key, you can’t change it later. New: CloudTrail Lake is no longer open to new customers from 31 May 2026; existing customers keep using it. AWS’s own migration advice for CloudTrail Lake users is Amazon CloudWatch: since December 2025 CloudWatch can ingest CloudTrail organization-wide (also VPC and WAF logs), normalize to OCSF with pipelines, and expose the data as Apache Iceberg tables in S3 Tables for Athena; existing event data stores can be exported with aws logs create-import-task.

$ aws securitylake create-subscriber \
    --subscriber-name siem-analytics \
    --access-types LAKEFORMATION \
    --subscriber-identity principal=444455556666,externalId=siem-ext-123 \
    --sources '[{"awsLogSource":{"sourceName":"VPC_FLOW","sourceVersion":"2.0"}},
                {"awsLogSource":{"sourceName":"CLOUD_TRAIL_MGMT","sourceVersion":"2.0"}}]'

Network and edge log sources

Skill 1.2.6 asks you to choose logs from the network design and the threat. Ask where the attack travels. Traffic between instances shows up in VPC Flow Logs. Traffic between VPCs and on-premises through a transit gateway shows up in Transit Gateway flow logs. DNS-based threats show up in Resolver query logs. Web attacks show up in WAF, ALB and CloudFront logs.

The other network and edge logs

Transit Gateway flow logs capture traffic through a transit gateway, with fields like tgw-attachment-id and packets-lost-blackhole, to CloudWatch Logs, S3 or Firehose. Only the transit gateway owner can create them. Route 53 Resolver query logs record DNS queries from your VPCs, from on-premises through inbound endpoints, through outbound endpoints, and the action of DNS Firewall rules. Answers served from the Resolver cache are not logged again. You can share a query logging configuration with other accounts through AWS RAM. ELB access logs are off by default, delivered every 5 minutes on a best-effort basis. CloudFront has standard logs and real-time logs (to Kinesis Data Streams within seconds, with a sampling rate). AWS WAF logs go to CloudWatch Logs, S3 or Firehose, and the destination name must start with aws-waf-logs-.

S3 server access logs vs CloudTrail data events. Server access logs are free to deliver (you pay storage), arrive within a few hours, and also record authentication failures and lifecycle actions. CloudTrail data events arrive in minutes, support prefix filtering and integrity validation, and are the recommended choice for auditing. Many teams use both.

2 111122223333 eni-0a1b2c3d 203.0.113.12 10.0.1.25 51544 22 6 3 180 1791622800 1791622860 REJECT OK

$ aws ec2 create-flow-logs --resource-type VPC --resource-ids vpc-0abc \
    --traffic-type ALL --max-aggregation-interval 60 \
    --log-destination-type s3 --log-destination arn:aws:s3:::central-flow-logs \
    --log-format '${version} ${vpc-id} ${subnet-id} ${instance-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${tcp-flags} ${action} ${flow-direction}'

Cheatsheet extras: protecting, keeping and troubleshooting CloudTrail logs

The cheat sheet has a full chapter on securing CloudTrail log files: stop unauthorized access, stop deletion, keep logs for X years, read them as an auditor, and fix delivery. Today’s lecture covered trails, the organization trail and validation; this section joins them into one protection design and adds the troubleshooting checklist and the four-service comparison the cheat sheet says to learn.

{ "Sid": "AWSCloudTrailWrite",
  "Effect": "Allow",
  "Principal": { "Service": "cloudtrail.amazonaws.com" },
  "Action": "s3:PutObject",
  "Resource": "arn:aws:s3:::org-ct-logs/AWSLogs/o-abc123/*",
  "Condition": { "StringEquals": {
      "s3:x-amz-acl": "bucket-owner-full-control",
      "aws:SourceArn": "arn:aws:cloudtrail:us-east-1:111122223333:trail/org-trail" } } }

$ aws s3api put-bucket-lifecycle-configuration --bucket org-ct-logs \
  --lifecycle-configuration '{"Rules":[{"ID":"ct-retention","Status":"Enabled",
    "Filter":{"Prefix":"AWSLogs/"},
    "Transitions":[{"Days":90,"StorageClass":"DEEP_ARCHIVE"}],
    "Expiration":{"Days":2557}}]}'

Facts the cheat sheet tests about CloudTrail

• CloudTrail records API calls only. It does not record an SSH or RDP login to an instance or the commands typed there; use the CloudWatch agent (OS logs) or Session Manager session logging (Day 4).
• Log files are delivered about every 5 minutes on average; this isn’t guaranteed, so use EventBridge when you need seconds.
• A trail that applies to all Regions is automatically added to new Regions. Event times are in UTC.
• You can have up to 5 trails per Region; use separate trails or advanced event selectors to send data events and management events to different buckets.
• Data events (S3 object, Lambda invoke, DynamoDB item calls) are off by default and are the usual reason for a surprise CloudTrail bill.

When logs stop arriving: a checklist

The four logging services side by side

Lecture summary map

Day 3 on one screen.

1. Mind map

2. Match the requirement to the log

3. “If the question says… pick…”

Must-remember cheat sheet

Core quiz

Check the basics first. Pick an answer, then press Check.

Hard quiz

Harder than most real exam questions on purpose, with one “choose two” and one matching question. Aim for 9 out of 12 before Day 4.

Practice test questions (from your mock exams)

These 9 questions match today’s topics: 8 come from your own SCS-C03 mock exams and 1 was added to cover a recent AWS change. Wording was modernized where services were renamed (for example CloudWatch Events is now Amazon EventBridge), and each answer was checked against current AWS behavior; where the mock’s answer key was wrong, the explanation says so. The source (mock and question number) is at the end of each explanation.

Hands-on lab (30 minutes, low cost)

1. Open CloudTrail → Event history and filter by Event name = ConsoleLogin. Note that this works with no trail at all.
2. Create a trail (or reuse one) with log file validation on, writing to a new S3 bucket. Wait about an hour, then run aws cloudtrail validate-logs --trail-arn ... --start-time .... Delete one log object and run it again to see the INVALID result.
3. In a test VPC, create a VPC flow log to a CloudWatch Logs log group with a 1-minute aggregation interval and REJECT traffic only. Try to SSH to an instance from an address the security group doesn’t allow and find the REJECT record.
4. Turn on Route 53 Resolver query logging for the same VPC to the same log group. From the instance run dig example.com twice and check that the second, cached answer isn’t logged again.
5. Set the log group retention to 1 day, then open Logs Insights and run fields @timestamp, @message | sort @timestamp desc | limit 20 to preview Day 4.
6. Clean up: delete the flow log, the Resolver query logging configuration and the log group; stop and delete the extra trail (a second trail’s management events are charged); empty and delete the bucket; terminate the instance. Don’t enable Security Lake or CloudTrail data events for all buckets in a personal account: both can grow costs quickly.