Security+ SY0-801 Investigation Data: Logs, Data Sources, System Images and Log Parsing

Access and accounting logs record what an identity opened and how it used its access, authentication logs record who signed in, and audit logs record who changed a setting. All three involve accounts, which is why they are the easiest to confuse; the difference is the event each one captures.

• Logical access logs record access to systems and data: which account read, changed, exported or deleted which file, database record or object, and when. A database or file server writes them after the user is already signed in.
• Physical access logs come from badge readers and door controllers: which badge or credential opened which door, at what time, and whether entry was granted or refused. They prove a credential was used, not who was holding it.
• Accounting (the third A of AAA) records how access was used once granted: session start and stop, duration and data volume. With RADIUS, a network access server sends accounting start, interim and stop records carrying the session time and the bytes in and out for each user session.
• Authentication logs record each sign-in attempt: account, result (success or failure), time, source address, method and the MFA outcome. They answer whether an account signed in, from where and when, including sign-ins an attacker made from another device with a stolen password.
• Audit logs record administrative and configuration actions: which privileged account changed which policy, permission or setting, when, and usually the old and new values.

Endpoint logs show what happened on a host, while server and application logs show what a service did with the requests it received. They hold detail no network source can see, because it exists only inside the system.

Endpoint logs

Endpoint logs come from workstations and servers: operating system event logs (for example Windows Security and System logs, Linux auth and syslog files) and the telemetry of an endpoint detection and response (EDR) agent. The most valuable records for an investigation are process creation events, which name the new process, its parent process and its command line. That is how an analyst learns that a spreadsheet macro spawned cmd.exe and which download command it passed. Endpoint logs also record file and registry changes, USB device connections, local and network logons, and service installations. A destination server's logon events record the source address of an incoming network logon, which makes them useful for tracing movement between internal hosts that no perimeter sensor saw.

Server and application logs

A web server's access log records one line per request: client address, time, method, URL with query string, status code and bytes returned. Because it is written after TLS has been decrypted, it holds the request details that an encrypted packet capture taken in front of the server cannot show, which makes it the primary source for web attacks such as SQL injection or directory traversal. Applications also log their own events: sign-ins to the application, errors, transactions and business actions.

One trap concerns proxies and load balancers. A load balancer or reverse proxy accepts the client's connection and opens its own connection to the web server, so every record written on the server side shows the load balancer's address as the client. The real client address survives only in the load balancer's own request logs or in a forwarded header such as X-Forwarded-For, if the server is configured to log it.

Device logs record the events of network infrastructure itself, network logs record the decisions it makes about traffic, and communication logs record who exchanged messages or calls with whom.

• Device logs come from routers, switches, firewalls, wireless controllers and other appliances, usually forwarded by syslog. They record restarts, power and hardware faults, interface up/down changes and similar operational events, so they confirm, for example, that a core router logged a power-supply fault before an outage.
• Network logs record traffic decisions: firewall allow and deny entries (source, destination, ports, rule), web proxy logs (which user or host requested which URL), DNS query logs and NAT translation logs.
• DNS query logs record each lookup with the requesting client address, the name and the time. Malware usually resolves its command-and-control domain before connecting, so searching these logs for a malicious domain lists every internal host that tried to reach it. These logs cover name resolution only; the traffic itself is recorded elsewhere, by flow data, firewalls or proxies.
• DHCP lease logs record which MAC address and hostname held which IP address and for how long. They map an address to a device; they do not record lookups or traffic.
• Communication logs describe messages and calls rather than packets: an email gateway's message trace (sender, recipients, subject, time, delivery status), chat platform logs and VoIP call detail records. They usually show who communicated with whom and when, often without the content. A message trace lists every mailbox that received a phishing message, while proxy logs show only who clicked its link.

Metadata is data about data: descriptive information stored with a file or message, such as its author, timestamps, location or routing, which can establish origin and timing without relying on any server log.

• Document properties in office files record the author, the account that last saved the file and the creation and modification times. Because they travel inside the file, investigators can read them from a leaked copy found anywhere.
• Image metadata (EXIF) can hold the camera or phone model, the time taken and GPS coordinates.
• Email headers travel with the message. Every mail server that relays it adds a Received line with its name, address and time, and each new line is added above the previous ones, so reading them from the bottom up shows the claimed route wherever the message was found. Only lines added by servers you trust or can verify are reliable: anything below the first trusted relay could have been written by the sender. The Authentication-Results header, which records SPF, DKIM and DMARC outcomes, helps confirm the sending domain. The From display name is set by the sender and is trivially forged, so it proves nothing about origin.
• File system metadata records file creation, modification and access times on the volume.

Metadata is different from a hash: a hash identifies the exact file and proves it is unchanged, but holds no author, time or location information. Metadata can also be edited by someone who controls the file, so it is corroborating evidence, strongest when it agrees with independent logs.

Use flow data to learn who talked to whom, when and how much; use a packet capture when you need the content of the traffic. The two sources trade detail for retention.

NetFlow (Cisco's format) and IPFIX (Internet Protocol Flow Information Export, the IETF standard based on NetFlow version 9) export one summary record per conversation: source and destination addresses and ports, protocol, start and end times, packet and byte counts. They carry no payload. Records are small, so organizations keep them for weeks or months, which makes them the source for questions about destinations and volumes over a period longer than packet retention covers.

A packet capture (PCAP) stores full frames including the payload. For unencrypted protocols (FTP, HTTP, Telnet, SMB without encryption) an analyst can reassemble the session and recover the file transferred or the commands typed. Captures are very large, so they are kept for a short time, often only days, and older traffic is gone. Encryption hides the payload: a capture of TLS traffic shows addresses, ports, timing and handshake details, but not the URLs, files or commands inside.

Both share one limit: a sensor only records traffic that passes it. A capture sensor or flow exporter on the internet edge never sees east-west traffic between two servers on the same internal switch, however long its retention. For that traffic, use the hosts' own logs (such as logon events on the destination) or an internal sensor.

Beyond logs and network data, investigators draw on vulnerability scans, automated reports, dashboards, security tool data and surveillance footage, each answering a different kind of question.

• Vulnerability scans record which weaknesses a host had when it was scanned. Archived scan results are evidence of past exposure: they show what a host was exposed to before an incident, which only a scan from that time can record once remediation has changed the host. They also show whether a finding was reported and missed or not yet remediated.
• Automated reports are scheduled, dated summaries of a past period (for example a compliance report generated at each month-end). Kept as sent, they are a historical record of status at those points, after the tool's own detailed data has aged out.
• Dashboards present live, continuously updated status on one screen: open alerts, most-affected hosts, trends. They show the present, not last year.
• Security tools contribute their alerts and telemetry: EDR, intrusion detection and prevention systems, data loss prevention, antivirus, the SIEM and email security tools.
• Surveillance footage shows who was physically present. A badge log proves which credential opened a door; video of the door at that time shows the person who walked through, which matters whenever the credential and the person may not match, such as a lent or cloned badge.

A memory dump captures what is running, a bit-level copy captures every sector of a drive, and a snapshot captures a virtual machine or cloud volume from the hypervisor or provider; choose by where the evidence lives and what access you have. (The acquisition procedure itself, order of volatility, write blockers and chain of custody, belongs to the incident response lesson for objective 4.7.)

Why each one matters

Memory dumps preserve state that never reaches disk. Fileless malware that receives commands over TLS leaves nothing on the drive and nothing readable on the wire, but while it runs, its code and the decrypted commands and session keys are in memory. The same applies to full-disk encryption: while a laptop runs unlocked, the volume key is in memory, so dumping memory before imaging preserves a way to decrypt the drive image.

Bit-level (bit-by-bit) copies duplicate the whole drive, not just its files. Deleting a file usually removes only its directory entry; the data remains in unallocated space until overwritten. On SSDs, TRIM and garbage collection can erase deleted blocks soon after deletion, so recovery is less certain. File-level copies and backups follow the file system, so they collect only files that still exist.

Snapshots are taken through the hypervisor or cloud provider. In a public cloud, where nobody can reach the physical disk and autoscaling may terminate the instance, a volume snapshot preserves the disk immediately and survives termination; it can be attached to a forensic workstation later. A hypervisor snapshot of a running VM that includes memory state captures both memory and disk from outside the guest, so nothing runs inside it for anti-forensic malware to detect.

Hash evidence when you collect it, and send logs off the host as they are written to storage the host cannot change. Integrity means being able to show that evidence is exactly what it was when collected.

• Hashing. A cryptographic hash such as SHA-256 is a fingerprint of a file's exact contents. Record it when a log export or image is made; anyone can recompute it later, and any change, even one bit, produces a different value. Only a hash recorded at collection and compared later proves a file is unchanged; other controls protect or restrict a file without proving anything about it.
• Central, write-once log storage. An attacker with administrator rights on a server controls everything stored on that host, including any integrity data kept beside the logs, and commonly deletes or edits logs to hide activity. Logs stay trustworthy only if events are forwarded off the host as they occur (for example by syslog or an agent to a SIEM or collector) and land on append-only or WORM (write once, read many) storage that the server's administrators have no rights to change.
• Comparing copies. When a host's local log and the copy it forwarded in real time disagree, the forwarded copy is the more trustworthy one. Events present centrally but missing locally point to tampering with the local log after the events were sent.

Accurate time underpins integrity as well: NTP keeps clocks synchronized so events from different systems can be ordered reliably.

Log parsing turns raw lines into answers by filtering for the relevant events, extracting the fields you need, counting and sorting them, and normalizing timestamps so different sources line up. SIEMs do this with parsers and queries; on the command line the classic Unix tools do the same job.

• Filtering: grep "Failed password" auth.log keeps matching lines; grep -v inverts the match and keeps every line that does not match; grep -i ignores case.
• Regular expressions match values by their shape rather than their position, so a pattern for an IPv4 address (four numbers separated by dots) finds addresses wherever they appear in mixed log formats, with tools such as grep -E or a SIEM's extraction rules.
• Field extraction: awk '{print $1}' prints a field by position ($NF is the last field, $(NF-3) the fourth from last); cut does the same with a delimiter.
• Counting and ranking: sort | uniq -c | sort -rn groups identical values, counts them and lists the highest count first. uniq collapses only adjacent duplicates, so it must receive sorted input or the same value appears several times with partial counts.

Put together, grep "Failed password" auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn ranks the source addresses of failed SSH logins, because each failure line ends with "from <IP> port <n> ssh2".

Normalizing time

Sources often record time in different zones or formats. Before building a timeline, convert every timestamp to one reference zone, usually UTC. A gap between related events that exactly matches a time-zone offset (two hours between UTC and UTC+2 logs, for example) is a normalization problem, not a real delay. Re-syncing clocks with NTP fixes future drift but does not rewrite entries already logged.

An IP address identifies a device only at a specific time, and a device is not a person, so attribution takes two steps: address to device using the logs that assigned or translated the address at that moment, then device to user using authentication logs.

• Dynamic internal addresses move between devices. Any current-state view answers who holds the address now, not who held it at the time of the incident. The DHCP lease history records which MAC address and hostname held the address at the time; logon events on that device then show who was signed in.
• Port address translation (PAT) puts hundreds of internal hosts behind one public address. The public address alone cannot single one out; the NAT device's translation log matched on the exact time and the public source port maps to one internal address and port. Confirm the log's time zone and clock accuracy first.
• Behind a load balancer, the real client address is in the load balancer's logs or the X-Forwarded-For header, not the web server's logs.

Stakeholders

Investigations reach beyond the security team, and objective 4.8 names three stakeholders:

The deciding question is what the investigation needs from them: discipline goes to HR, legal decisions to Legal, the money trail to Accounts.