Log data refers to the systematically recorded information generated by software
applications, operating systems, or hardware devices. This data captures various
events, transactions, or activities within a system or network, serving as a chrono-
logical record that can be analyzed for troubleshooting, security monitoring, and
compliance purposes. Log data is often crucial for investigations, as it can provide
an immutable trail of system and user activities, helping to identify what happened,
when it happened, and who was involved. The log data you collect from your sys-
tems and devices might seem pretty mundane; however, these logs could contain the
precise evidence needed to investigate and successfully prosecute a crime. For log
data to stand up in court as admissible evidence, you must put into place and operate
with chain of custody in mind and take special precautions in collecting, handling,
and storing the data. Many regulations, such as PCI DSS, HIPAA, and SOX, require
the use of logs and log management. When set up properly and with due care, logs
can provide an immutable fingerprint of system and user activity. In many cases,
the logs tell a story about what really happened in an incident. They can tell what
systems were involved, how the systems and people behaved, what information was
accessed, who accessed it, and precisely when these activities took place. Due to
the substantial volumes of log data generated in modern network environments,
storage practices have evolved beyond traditional methods. While optical media
like DVD-R was once standard, it is now an outdated approach due to its limited
capacity and durability. Modern standards favor the use of high-capacity hard drives
and cloud-based storage solutions. These methods offer enhanced scalability and
resilience, which are crucial for handling extensive log records. Furthermore, cloud
storage provides the benefits of remote accessibility and the potential for integration
with advanced analytical tools.
Syslog is a standardized protocol used by various devices and systems to send event
notification messages across an IP network to a syslog server. This protocol plays a
critical role in the management and security of networks by enabling the central-
ized collection and storage of log data. These logs are essential for many operational
tasks, including auditing, monitoring, troubleshooting, and analyzing to maintain
system health and security.
Figure 22-1 illustrates syslog messages transmitted from network devices and servers
to a centralized logging system. For instance, it shows how a mail server, specifi-
cally running postfix/smpd—indicative of a Linux host—has rejected a connection
attempt because the host’s identity information was unattainable. This is a practical
example of how syslog can provide insights into network activities. Several visual
syslog servers are available at no cost, with SolarWinds offering one such option.
Firewall Logs
Firewall logs are essential for monitoring the traffic allowed or denied through a
network’s firewall. These logs can reveal unauthorized attempts to access the net-
work and help in understanding the traffic patterns that are typical for the orga-
nization. For example, a firewall log might show that an IP address from a foreign
country attempted to connect to your server multiple times within a short period,
which could be a sign of a potential attack.
Here’s an example of part of a firewall log:
2023-09-23 14:32:10 ALLOW TCP 192.168.1.2 8.8.8.8 443 80
2023-09-23 14:32:11 DENY TCP 203.0.113.42 192.168.1.2 22 6000
The first entry in this example indicates that a device within the network
(192.168.1.2) was allowed to connect to an external server (8.8.8.8) over HTTPS
(port 443). The second entry shows that an external IP address (203.0.113.42)
attempted to connect to an internal device on SSH port 22 and was denied. If port
22 is not supposed to be publicly accessible, this entry serves as an alert for a pos-
sible security issue that requires immediate action, such as blocking the suspicious
IP address.
Application Logs
Application logs contain records of services, events, and systems within an application,
providing near end-to-end visibility. The processes for implementing, managing, and
reporting application logging need to be configured and reviewed before, during, and
after an incident. You should parse, partition, and analyze all logs and data generated
by an application.
Logged critical process information about user, system, and web application behav-
ior can help incident responders build a better understanding of the breadth and
depth of an attack. Application logs contain information related to applications
run on the local system, how they are performing, and any attempts to utilize the
applications. Depending on the level of logging, the logs can help in determining
whether users exposed an unknown flaw in application programming or whether
they attempted to escalate privileges or modify data they were not intended to have
access to.
In the left sidebar, you see several categories under Windows Logs, indicating the
different types of logs that can be reviewed, including Application, Security, Setup,
System, and Forwarded Events. Each of these sections holds logs related to a specific
area of the Windows operating system.
The main pane displays a list of individual events under the Application category.
Each entry shows the level of the event (for example, Information), the date and
time when the event was logged, the source (that is, the program or component that
logged the event), the event ID (which is a unique identifier for each type of event),
and the task category (which can sometimes provide more details about the event).
At the bottom of the screen is a detailed view for a selected event; for example, in
Figure 22-2, it’s for Event ID 16384 from the Security-SPP source. The description
provides specific information about the event—in this case, “Successfully scheduled
Software Protection service for restart at 2123-10-16T16:42:58Z. Reason: Rules-
Engine.” This description gives you insight into what occurred (in this case, that a
service was scheduled to restart at a given time).
Endpoint logs provide detailed information about activities on individual devices,
such as computers and smartphones, that are connected to a network. These logs
can help you identify suspicious or unauthorized behavior, such as the installation
of unauthorized software or attempts to access restricted files. For instance, an
endpoint log might reveal that a user attempted to install software that’s not on the
approved list, triggering a security alert.
Here’s an example of part of an endpoint log:
2023-09-23 14:35:00 User "JohnDoe" initiated "New_Outbound_
Connection" to IP "203.0.113.42" on Port "8080"
2023-09-23 14:36:00 User "JohnDoe" executed "Unknown_
Application.exe"
The first entry in this example indicates that a user named JohnDoe initiated an
outbound connection to an external IP address on port 8080. The second entry
shows that the same user executed an application that was not recognized by the
system’s security software. Such entries can be invaluable for tracing unauthorized or
suspicious activities. For instance, if Unknown_Application.exe is not a sanctioned
application, this could be a sign of malware or unauthorized software installation
that needs to be investigated further.
operating system (OS)–specific security logs are all about the capabilities of the
specific system you are trying to obtain logs from. Each system that has logs also
refers to specific system messages that contain log records for the operating system
events. Many of them show how system processes and drivers were loaded, errors
that occurred during loading, and any failures that have occurred. In Windows, there
are a few areas to obtain logs. For example, you can use the Event Viewer to collect
.evt and .evtx files from other Windows-based machines. You can even sort by event
ID and perform filtering and exporting. The Windows Admin Center can provide
events from Event Viewer as well and allows you to export them.
On Linux hosts, log files are mostly contained in the /var/log directory. You should
make it a regular process to collect these logs and store them as you would with
other devices. Application, system, and security logs should be stored off-host when
possible. Monitoring system logs provides proactive monitoring and helps you build
a complete picture when an incident occurs; it helps you determine which systems
were accessed and when, what files were accessed, which applications were installed,
and whether the integrity of the system was compromised and how.
Figure 22-3 shows the Windows Event Viewer, where you can clearly see the event
ID, level of logging, and type of message. Here, four types of messages are displayed:
Warning, Critical, Information, and Error. Information messages are typically from
the operating system to let you know that it performed some action like allowing a
process to start. Errors are generated when processes fail to start. Critical messages
indicate some serious failure, such as a system not shutting down properly. Warning
messages are normal; Windows handles all of these events and recovers without any
user intervention.
Figure 22-3 also shows a view of the Security section of the Windows Event Viewer
interface, which is pivotal for security monitoring and incident response activities.
The panel in this example is populated with security events and indicates Audit
Success for each of them, which signifies the successful execution of monitored
activities, such as user authentications or policy access checks. These logs are crucial
for identifying patterns pointing to security threats or verifying compliance with
security policies.
Each entry details the timestamped date and time when the event was recorded,
the source of the event, which typically references the security component of Win-
dows, the unique event ID for the particular security event, and the task Category,
which includes subcategories such as Logon, Special Logon, and User Account
Management.
Highlighted at the bottom is an event with the ID 4799, which details that security-
enabled local group membership was enumerated. This action is typically performed
to review group policies or access rights, which could be routine or part of a security
audit. The timestamp and the fact that it’s an audit success suggest that the action
was expected and permitted.
This portion of the Event Viewer serves as an essential audit trail, capturing each
security-related event, enabling system administrators to maintain oversight over
security-relevant operations, and ensuring that any anomalous activities can be
detected and investigated promptly.
The system log records error messages, warnings, and other information generated
from the Windows operating system. It provides a wealth of information on what is
going on with the system. System log records can also help you determine if, when,
or where a user attempted to log in and failed, what user ID was used, and what IP
address it came from.
Intrusion prevention system (IPS) and intrusion detection system (IDS) logs are
specialized records that identify potentially harmful activities within a network.
These logs are generated by an IPS or IDS solutions, such as Snort or Suricata,
and can alert you to various attacks, such as SQL injections or cross-site scripting.
Let’s review what a potential IPS/IDS log entry would look like and break down the
information.
Here’s an example of part of an IPS/IDS log:
2023-09-23 13:45:32 ALERT SQL Injection detected from
192.168.1.4
2023-09-23 13:46:10 ALERT Brute-force attempt from 203.0.113.7
The first entry in this example indicates that an SQL injection attack originating from
the IP address 192.168.1.4 was detected. This serious alert warrants immediate inves-
tigation, as SQL injections can lead to unauthorized access to sensitive data.
The second entry shows a brute-force login attempt from the IP address 203.0.113.7.
This is another red flag that could indicate an attacker is trying to gain unauthorized
access, and it should be promptly addressed to prevent a potential breach.
Network Logs
Network logs capture data traffic that traverses the network infrastructure. These
logs, which can be generated by network monitoring tools like Wireshark or Solar-
Winds, provide insights into bandwidth usage, connection times, and the types of
protocols being used. Let’s take a look at a network log example.
Here’s an example of part of a network log:
2023-09-23 14:20:15 INFO TCP Connection from 192.168.1.2 to
8.8.4.4
2023-09-23 14:21:00 WARNING Large data transfer to 203.0.113.8
The first entry in this example documents a standard TCP connection from an
internal IP address, 192.168.1.2, to an external IP address, 8.8.4.4, which is often
used for DNS resolution. This is generally considered normal traffic and may not
require immediate attention. However, the second entry warns about a large data
transfer to an external IP address, 203.0.113.8. This could be a sign of data exfiltra-
tion and should be investigated immediately to ensure that sensitive information is
not leaked.
Metadata
Metadata is created from every activity you perform, whether it’s on a personal com-
puter or online, including every email, web search, and social or public application
and interaction. Metadata is data that provides information about other data. There
are many distinct types of metadata. On its own, it may have little or no relevance
to an investigation. However, when metadata is used with other types of data and
sources, the result can help you conduct an investigation.
Metadata security involves practice and policies designed to protect an organization
from security risks posed by unauthorized access to the organization’s metadata.
Even metadata from Microsoft Word documents contains the names of authors
and modifiers, dates of creation, and changes and file sizes. Metadata poses a risk of
disclosure of private information, usually unknowingly, because metadata is hidden
from plain view, and users may be unaware of it. It’s likely that those outside your
company shouldn’t have access to this information.
There are several types of metadata you should be aware of. The first is descriptive
metadata, which refers to elements like titles, dates, and keywords. For instance,
when you download a video file, it contains metadata describing what the video is
about as well as the name and date. With books, descriptive metadata includes book
titles, author names, dates, pages, and key data locations.
Another form of metadata, structural metadata, provides information concerning a
specific object or resource. Structural metadata records information on how a par-
ticular object or resource might be sorted. For example, with digital media (film on
DVD, for example), each section has a certain film running time length.
Preservation metadata provides information that strengthens the entire process
of making a digital file (object). This includes vital details required for a system to
communicate or interact with a specific file and upholds the integrity of a digital file
or object. Preservation Metadata: Implementation Strategies (PREMIS) mode is a
preservation metadata standard for common factors such as actions taken on a digi-
tal file or the rights attached to it.
Use metadata is data that is sorted each time a user accesses and uses a specific piece
of digital data. Use metadata is gathered in a clear and direct attempt to make poten-
tially helpful predictions about a user’s future behavior. This type of metadata can be
used to understand fluctuations in data that have no pattern when there is really a
pattern beneath the data.
There are even more types of metadata, including provenance metadata. This type
of metadata is most relevant when something changes or is duplicated frequently,
such as in the digital realm, where there are frequent changes. Administrative
metadata informs users what types of instructions, rules, and restrictions are placed
on a file. This information helps administrators limit access to files based on user
qualifications.
Cell phones hold a ton of metadata. For example, the pictures you take provide GPS
coordinates, the date and time the photo was taken, the language, the camera type,
the flash setting, and more. From the coordinates of an image you post to Instagram,
to metadata from emails, and public and private social media applications you use, to
airline tickets and rental car details, your phone is a treasure trove of information.
Figure 22-4 shows metadata that can be accessed for a Microsoft Word file. By
right-clicking the file and looking at the properties under Details, you can see the
title, authors, subject, tags, and comments. Notice the Remove Properties and Per-
sonal Information link at the bottom; by clicking it, you can remove certain personal
information.
Data Sources
Data sources are the various tools and methods used to collect, analyze, and
present information that supports an investigation. These sources can range from
vulnerability scans that identify weak points in a network to dashboards that provide
real-time analytics. Understanding how to leverage these data sources effectively is
crucial in a comprehensive approach to cybersecurity.
Vulnerability Scans
If there were no vulnerabilities in a network or computer system, there would be
nothing to exploit, and the network attack surface would be greatly reduced. How-
ever, software vulnerabilities always exist and will continue to exist because applica-
tions are developed by people, and people make mistakes, all of which can allow
attackers to compromise networks. Vulnerability scans are systematic examinations
of networks, systems, or applications to identify and evaluate security weaknesses or
flaws that attackers could exploit. These scans utilize specialized software tools to
probe systems for known vulnerabilities, such as unpatched software, insecure con-
figurations, and unprotected systems; they provide insights into potential security
gaps that require remediation to strengthen the overall security posture. Running
regular vulnerability scans is just the first step in your defensive posture. A network
scan should include any devices that has an IP address—each workstation, laptop,
printer, IoT device, router, switch, hub, IDS/IPS, server, wireless network, and
firewall—and all the software running on each device. You should run both authen-
ticated and unauthenticated scans because each type of scan provides insight into
vulnerabilities found in services running on the network, open ports on devices that
could allow malicious apps to run or communicate on them, and configurations or
issues that affect security.
Vulnerability scan reports should be saved for at least 24 months, and certain regu-
lations and industries require longer retention times. Some systems enable you to
save these reports and compare them from month to month to get a better picture
of your cybersecurity posture and improvements. Historical vulnerability scans can
also provide significant insights after an incident. By comparing previous scans with
the most recent ones, you can also look for variances in devices and systems that
may have been changed. A change can be anything from additional TCP/UDP ports
being shown as open to the detection of unauthorized software, scheduled events,
or unrecognized outbound communications. When you find problems, you should
make reimaging these devices a top priority.
Figure 22-6 shows basic vulnerability scan results from my internal network,
1.1.100.0/24, showing two certificate issues as medium-level threats. Vulnerability
reports are perfect for engaging several departments in your organization, getting
them involved and responsible for the overall posture.
Automated Reports
Automated reports serve as your eyes and ears within a network, offering a synthe-
sized view of various security metrics and events. These reports are often generated
by security information and event management (SIEM) systems such as Splunk or
IBM QRadar. They can be configured to run on a daily or weekly schedule or can
be triggered by specific events. Imagine receiving an automated report highlighting
15 failed login attempts from a single IP address within a 10-minute window. This
could be a red flag indicating a brute-force attack attempt. Another report might
alert you to three large data transfers to an external IP address within a short time
frame, raising suspicions of potential data exfiltration. Automated reports like these
enable you to spot irregularities and take corrective action swiftly, thereby minimiz-
ing the risk and impact of security incidents.
Dashboards
A dashboard is a user interface that organizes and presents information in an easy-
to-read format, often using graphs, charts, and gauges. This centralized display
aggregates data from multiple sources, providing a real-time overview of an orga-
nization’s network and security status, allowing for prompt responses to threats
and efficient monitoring of system health and performance. Dashboards are pivotal
in managing the complex flow of information in cybersecurity operations. SIEM
software works by collecting logs and event data generated by an organization’s
applications, security devices, and host systems and bringing it together into a single
centralized platform, allowing companies to identify threats in real time. A SIEM
system gathers data from antivirus events, firewall logs, and other locations and
sorts this data into categories. When the SIEM system identifies a threat through
network security monitoring, it generates an alert and defines a threat level based on
predetermined rules.
The SIEM dashboard in Figure 22-7 provides a quick reference to the top threats
that require security analyst attention. The section Alarms by Day Past 30 Days
enables the analyst to trend network performance on threats. Top Classification
shows types of active attacks in the last nine hours. Top Host (Origin) shows who is
attacking and from where, as well as hosts impacted by attacks.
Packet Captures
Packet captures offer a granular view of network activity. Tools like Wireshark are
commonly used to capture detailed data packets transmitted over a network. Say
that you’re reviewing a Wireshark capture, and you see a series of packets like this:
No. Time Source Destination Protocol Info
1 0.000000 192.168.1.2 8.8.4.4 TCP 80 > 443
[SYN] Seq=0 Win=5840 Len=0 MSS=1460
2 0.000200 8.8.4.4 192.168.1.2 TCP 443 > 80
[SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460
3 0.000400 192.168.1.2 8.8.4.4 TCP 80 > 443
[ACK] Seq=1 Ack=1 Win=5840 Len=0
4 0.001000 192.168.1.3 203.0.113.8 UDP Source port:
5060 Destination port: 5060
The first three packets in this example show a typical TCP handshake between the
source IP address 192.168.1.2 and the destination IP address 8.8.4.4. This is gener-
ally considered normal traffic. However, the fourth packet shows UDP traffic from
the source IP address 192.168.1.3 to the external IP address 203.0.113.8. This UDP
traffic could be especially interesting, as UDP is often used for streaming and might
also indicate data exfiltration. Understanding the nuances of packet capture, as in
this Wireshark example, can help you detect anomalies or malicious activities that
might otherwise go unnoticed.