With the increasing threat of hackers, identity thieves, and corporate data theft, you need to be able to analyze the security of your network at the packet level.
The topics that will be covered in this chapter include:
- Security analysis methodology
- Scans and sweeps
- OS fingerprinting
- Malformed packets
- Phone home traffic
- Password cracking traffic
- Unusual traffic
Security analysis at the packet level is based on detecting and analyzing suspect traffic, that is, the traffic that does not match normal patterns because of the presence of unusual protocol types or ports, or unusual requests, responses, or packet frequency. Suspicious traffic may include reconnaissance (discovery) sweeps, phone home behavior, denial of service attacks, botnet commands, or other types of behavior from direct attacks or virus- or botnet-based agents.
Wireshark captures strategic points in the network to investigate suspicious packets from specific hosts or on network segments and egress points can also complement any Intrusion Detection System (IDS) systems that may be in place to alert the IT staff about the suspicious traffic.
The ability to identify abnormal traffic patterns that bear investigation versus traffic caused by poorly behaving applications, misconfigurations, or faulty devices can be made much easier if you have a baseline of what is normal. A baseline is a snapshot capture of typical conversations with your primary applications and servers and the background traffic on the network segments that they reside on. In a potential security breach situation, you can compare the normal protocols, traffic patterns, and user sessions from a baseline with a current capture, filter out the normal traffic, and then inspect the differences.
To allow the comparison of baselines in your security analysis, you need to periodically capture and store packet trace files that cover a sufficient period of time to provide a good sample of typical user and background traffic patterns while keeping the file sizes manageable for use within Wireshark, for example, 100 MB to 1 GB per file. You can configure the Ring Buffer option within Wireshark's Capture Options window to save a series of reasonably sized files for longer captures or busier network segments.
Although your baselining needs and practices will depend on your environment, some of the traffic aspects that you should inspect include:
- Broadcast and multicast types and rates:
- What devices and applications are using broadcasts and multicasts?
- What are the typical broadcast and multicast packet rates?
- Applications and protocols:
- What applications are running over the network?
- What protocols and ports are they using?
- Application launch sequences and typical tasks
- Are application sessions encrypted?
- Are all users forced to use encryption? Any exceptions?
- What are the login/logout sequences and dependencies?
- Routing protocol(s) and routing updates
- ICMP traffic
- Boot-up sequences
- Name resolution sessions
- Wireless connectivity includes normal management, control, and data frame contents
- VoIP and video communications
- Idle time traffic is the host communicating with other hosts when there are no users logged in
- What backup processes are running at night and for how long?
- Are there any suspect protocols or broadcasts/scans taking place?
As you inspect your baseline captures, it is helpful to view a summary of the protocols being used by selecting Protocol Hierarchy from the Wireshark's Statistics menu. In the following screenshot, for example, you can see that there is some Internet Relay Chat (IRC) traffic, as well as the Trivial File Transfer Protocol (TFTP) traffic, neither of which might be normal on your network and could be an indication of rogue communications with outside entities:
Analyzing baselines of normal traffic levels and patterns is also an excellent way of getting familiar with your network environment and its typical packet flows and protocols, which better prepares you to spot abnormal traffic.
There are several popular tools that are used by security professionals to perform security assessment and vulnerability testing. As these tools can generate the same types of scans, fingerprinting, and other exploitive activities, as might be used by hackers and malicious agents, they can be useful to a packet analyst to analyze the packets that they generate with Wireshark to build familiarity with how different types of activities appear in a packet trace and also to build display filters to detect them.
One of the most popular tools is Network Mapper (Nmap), a free and open source utility for network discovery and security auditing. Nmap runs on all major computer operating systems and offers a command-line and GUI version (Zenmap).
Note
You can find more information about Nmap at http://nmap.org and information on other top security tools can be found at http://sectools.org.
Wireshark can be used to identify unusual patterns or packet contents in the network traffic including network scans, malformed packets, and unusual protocols, applications, and or conversations that should not be running on your network. The following is a general list of traffic types that may not be acceptable and/or warrant investigation to validate their legitimacy in your environment:
IP address and port scans can be generated from network management applications to build or maintain their list of devices and applications to monitor/manage, but that's usually the only legitimate source of these types of traffic.
- Clear text passwords: These are passwords that you can see in the Wireshark's Packet Details or Packet Bytes fields. These are typical for File Transfer Protocol (FTP) logins, but not typical or acceptable elsewhere.
- Clear text data: This is the data in packet payloads that can be read. This is typical for HTTP requests and responses and commonly seen in application server to database requests and responses, but these database exchanges should be between hosts on isolated, nonpublic network segments and otherwise physically secure environments.
- Password cracking attempts: These are repeated, systematic attempts to discover a working password, usually from a single device.
- Maliciously formed packets: These are packets with intentionally invalid or improperly formatted data in protocol fields that are intended to exploit vulnerabilities in applications.
- Phone home traffic: This is the traffic from a rogue agent that may be resident on a server or workstation that periodically checks in with a remote (usually off-network) host.
- Flooding or Denial of Service (DOS) attacks: This is the traffic that is intentionally sent at a very high packet-per-second rate to one or more hosts in an attempt to flood the host(s) or network with so much traffic that no one else can access their services.
- Subversive activities: These include a number of techniques to prepare for and facilitate the man-in-the-middle attacks where a device is tricked into sending packets to a malicious host for the purpose of intercepting data.
This is only a sampling of types of malicious traffic that you might see on your network; network security is an ever evolving exchange of increasingly sophisticated attacks and subsequent countermeasures.
As you develop your security analysis skills, you might want to build a special security profile in Wireshark that includes packet coloring rules based on display filters to help identify suspicious or malformed packets, as well as a set of Filter Expression Buttons that isolate and display various types of questionable traffic you might be looking for.
Some examples of display filters to isolate and inspect suspicious packets include:
Malicious programs and rogue processes might investigate a network environment for available ports and hosts using various scanning processes before launching an exploit. Identifying the presence of these reconnaissance processes may allow thwarting the attack before it is launched, as well as tracking down and/or blocking the source of the malicious activity—especially if that source is inside the company as some of them are.
ARP scans, also called as ARP sweeps, are used to discover active localhosts on a network segment. An ARP sweep can be difficult to detect unless you apply a display filter and observe a steady, incremental sweep from the same device, as seen in the following screenshot:
As ARP packets cannot pass through a router, the source device conducting the ARP sweep must be on the same network segment that the ARP packets are seen on.
ICMP ping sweeps are used to discover active hosts on local or remote network segments (since ICMP uses IP and is routable) using ICMP Type 8 Echo Requests and Type 0 Echo Replies for a range of IP addresses. You can easily detect ping sweeps by using a display filter icmp.type == 8 || icmp.type == 0.
TCP port scans allow a malicious agent to discover which TCP ports are open on a target host. Network ports are the entry points to a server or workstation; a service that listens on a given port is able to service requests from a client. Malicious agents can sometimes exploit vulnerabilities in server code to gain access to sensitive data or execute malicious code on the machine, which is why testing all active ports is necessary for a complete coverage of any security validation.
Some of the most common ports used for TCP-based services include:
- 80 HTTP
- 443 HTTPS
- 8080 HTTP proxy
- 8000 HTTP alternate
- 21 FTP
- 22 SSH
- 23 Telnet
- 3389 Microsoft Remote Desktop
- 5900 VNC
- 25 SMTP
- 110 POP3
- 143 IMAP
- 3306 MySQL
- 1433 Microsoft SQL Server
- 1720 H.323
- 5060 SIP
A TCP port scan device will send a TCP SYN packet to a port on a target host, which will respond with either SYN, or ACK if the port is open, or RST if the port is closed. Similar to an ARP scan, a TCP scan can be detected by a series of SYN packets from a single IP address to a target IP address over a range of port numbers. A display filter can make detecting these types of scans easier:
ip.dest == <IP Address of target host> && tcp.flags.syn
UDP port scans are like TCP scans, but they are run against typical UDP-based services, the most common of which include:
- 53 DNS
- 161/162 SNMP
- 67/68 DHCP
- 5060 SIP
- 135 Microsoft Endpoint Mapper
- 137/139 NetBIOS Name Service
The preceding topics cover just a sampling of the most common scans used by malicious agents. Security analysts should research this topic further to identify all the types of scans that may be used to exploit their particular environment's vulnerabilities.
ARP scans
ARP scans, also called as ARP sweeps, are used to discover active localhosts on a network segment. An ARP sweep can be difficult to detect unless you apply a display filter and observe a steady, incremental sweep from the same device, as seen in the following screenshot:
As ARP packets cannot pass through a router, the source device conducting the ARP sweep must be on the same network segment that the ARP packets are seen on.
ICMP ping sweeps are used to discover active hosts on local or remote network segments (since ICMP uses IP and is routable) using ICMP Type 8 Echo Requests and Type 0 Echo Replies for a range of IP addresses. You can easily detect ping sweeps by using a display filter icmp.type == 8 || icmp.type == 0.
TCP port scans allow a malicious agent to discover which TCP ports are open on a target host. Network ports are the entry points to a server or workstation; a service that listens on a given port is able to service requests from a client. Malicious agents can sometimes exploit vulnerabilities in server code to gain access to sensitive data or execute malicious code on the machine, which is why testing all active ports is necessary for a complete coverage of any security validation.
Some of the most common ports used for TCP-based services include:
- 80 HTTP
- 443 HTTPS
- 8080 HTTP proxy
- 8000 HTTP alternate
- 21 FTP
- 22 SSH
- 23 Telnet
- 3389 Microsoft Remote Desktop
- 5900 VNC
- 25 SMTP
- 110 POP3
- 143 IMAP
- 3306 MySQL
- 1433 Microsoft SQL Server
- 1720 H.323
- 5060 SIP
A TCP port scan device will send a TCP SYN packet to a port on a target host, which will respond with either SYN, or ACK if the port is open, or RST if the port is closed. Similar to an ARP scan, a TCP scan can be detected by a series of SYN packets from a single IP address to a target IP address over a range of port numbers. A display filter can make detecting these types of scans easier:
ip.dest == <IP Address of target host> && tcp.flags.syn
UDP port scans are like TCP scans, but they are run against typical UDP-based services, the most common of which include:
- 53 DNS
- 161/162 SNMP
- 67/68 DHCP
- 5060 SIP
- 135 Microsoft Endpoint Mapper
- 137/139 NetBIOS Name Service
The preceding topics cover just a sampling of the most common scans used by malicious agents. Security analysts should research this topic further to identify all the types of scans that may be used to exploit their particular environment's vulnerabilities.
ICMP ping sweeps
ICMP ping sweeps are used to discover active hosts on local or remote network segments (since ICMP uses IP and is routable) using ICMP Type 8 Echo Requests and Type 0 Echo Replies for a range of IP addresses. You can easily detect ping sweeps by using a display filter icmp.type == 8 || icmp.type == 0.
TCP port scans allow a malicious agent to discover which TCP ports are open on a target host. Network ports are the entry points to a server or workstation; a service that listens on a given port is able to service requests from a client. Malicious agents can sometimes exploit vulnerabilities in server code to gain access to sensitive data or execute malicious code on the machine, which is why testing all active ports is necessary for a complete coverage of any security validation.
Some of the most common ports used for TCP-based services include:
- 80 HTTP
- 443 HTTPS
- 8080 HTTP proxy
- 8000 HTTP alternate
- 21 FTP
- 22 SSH
- 23 Telnet
- 3389 Microsoft Remote Desktop
- 5900 VNC
- 25 SMTP
- 110 POP3
- 143 IMAP
- 3306 MySQL
- 1433 Microsoft SQL Server
- 1720 H.323
- 5060 SIP
A TCP port scan device will send a TCP SYN packet to a port on a target host, which will respond with either SYN, or ACK if the port is open, or RST if the port is closed. Similar to an ARP scan, a TCP scan can be detected by a series of SYN packets from a single IP address to a target IP address over a range of port numbers. A display filter can make detecting these types of scans easier:
ip.dest == <IP Address of target host> && tcp.flags.syn
UDP port scans are like TCP scans, but they are run against typical UDP-based services, the most common of which include:
- 53 DNS
- 161/162 SNMP
- 67/68 DHCP
- 5060 SIP
- 135 Microsoft Endpoint Mapper
- 137/139 NetBIOS Name Service
The preceding topics cover just a sampling of the most common scans used by malicious agents. Security analysts should research this topic further to identify all the types of scans that may be used to exploit their particular environment's vulnerabilities.
TCP port scans
TCP port scans allow a malicious agent to discover which TCP ports are open on a target host. Network ports are the entry points to a server or workstation; a service that listens on a given port is able to service requests from a client. Malicious agents can sometimes exploit vulnerabilities in server code to gain access to sensitive data or execute malicious code on the machine, which is why testing all active ports is necessary for a complete coverage of any security validation.
Some of the most common ports used for TCP-based services include:
- 80 HTTP
- 443 HTTPS
- 8080 HTTP proxy
- 8000 HTTP alternate
- 21 FTP
- 22 SSH
- 23 Telnet
- 3389 Microsoft Remote Desktop
- 5900 VNC
- 25 SMTP
- 110 POP3
- 143 IMAP
- 3306 MySQL
- 1433 Microsoft SQL Server
- 1720 H.323
- 5060 SIP
A TCP port scan device will send a TCP SYN packet to a port on a target host, which will respond with either SYN, or ACK if the port is open, or RST if the port is closed. Similar to an ARP scan, a TCP scan can be detected by a series of SYN packets from a single IP address to a target IP address over a range of port numbers. A display filter can make detecting these types of scans easier:
ip.dest == <IP Address of target host> && tcp.flags.syn
UDP port scans are like TCP scans, but they are run against typical UDP-based services, the most common of which include:
- 53 DNS
- 161/162 SNMP
- 67/68 DHCP
- 5060 SIP
- 135 Microsoft Endpoint Mapper
- 137/139 NetBIOS Name Service
The preceding topics cover just a sampling of the most common scans used by malicious agents. Security analysts should research this topic further to identify all the types of scans that may be used to exploit their particular environment's vulnerabilities.
UDP port scans
UDP port scans are like TCP scans, but they are run against typical UDP-based services, the most common of which include:
- 53 DNS
- 161/162 SNMP
- 67/68 DHCP
- 5060 SIP
- 135 Microsoft Endpoint Mapper
- 137/139 NetBIOS Name Service
The preceding topics cover just a sampling of the most common scans used by malicious agents. Security analysts should research this topic further to identify all the types of scans that may be used to exploit their particular environment's vulnerabilities.
OS fingerprinting is a technique wherein a remote machine sends various types of commands to a target device and analyzes the responses to attempt to identify the target devices' operating system and version. Knowing which operating system a device is running makes it possible to use exploits specific to that operating system.
Nmap detects operating systems based on a series of port scans, ICMP pings, and numerous other tests, and then runs a set of follow-up tests based on the results to further define the OS version running.
In the following screenshot, you can see the test results verbiage from the GUI version of Nmap (Zenmap) as it completes an OS detection scan, as well as its best estimate of the operating system and version:
A Wireshark capture of the OS detection activity described earlier included as an example of one of the OS fingerprinting scripts that are run, a bogus HTTP request to the target device (172.20.0.1) for /nice%20ports%2C/Tri%6Eity.txt%2ebak to see exactly what kind of error response was generated, which is used to help pinpoint the OS version:
The exact format of the HTML response from the preceding request could be used to identify the OS and/or web server version, as seen in the following Wireshark packet details screenshot:
Analyzing packet captures of these kinds of OS fingerprinting requests and responses will make it much easier to spot similar activities from malicious entities.
Maliciously malformed packets take advantage of vulnerabilities in operating systems and applications by intentionally altering the content of data fields in network protocols. These vulnerabilities may include causing a system crash (a form of denial of service) or forcing the system to execute the arbitrary code.
An example of malformed packet vulnerability is Cisco Security Advisory cisco-sa-20140611-ipv6, wherein vulnerability in parsing malformed IPv6 packets in a certain series of routers could cause a reload (reboot) of a certain card that carries network traffic, which could intermittently cause service outages.
Another example of this kind of vulnerability is in some unpatched Windows or Linux systems that will crash if they receive a series of fragmented packets where the fragments overlap each other.
The types and possibilities of malformed packets are endless, but vulnerabilities are usually announced as they are discovered and some may provide packet details. You can build display filters and/or build coloring rules in Wireshark to detect these packets. It also helps to study and understand what range of values the different protocol fields normally and legally contain, and what TCP and other protocol sequences normally look like so you can spot suspicious contents in packet flows.
Phone home traffic originates from a rogue application on a device that periodically connects to a remote (usually off-network) host to receive updates or commands or deliver data collected from the infected host. The majority of phone home traffic will be the operating system and virus protection updates, Dropbox or other external services, and similar authorized and appropriate services, so it will take some effort to identify malicious traffic out of this mix.
It is important to understand the risk that phone home traffic can represent: many botnet Distributed Denial of Service (DDoS) attacks are supported by a "zombie army" of hijacked computers running software that may lie undetected for some period of time except for periodic communications with their Command and Control (C&C) servers awaiting instructions to attack a target. In a similar fashion, keylogging traffic will send periodic reports of video screenshots and keystroke data to the collecting host.
One way to identify potentially malicious phone home traffic is to capture and inspect the DNS queries as these sessions start up, looking at two distinct areas:
- The hostname(s) of legitimate services are often reasonably recognizable.
- DNS queries for illegitimate applications contacting C&C servers will often return a long list of aliases with IP addresses that are not all in the same general range (that is, from all over the world). A display filter that helps identify DNS responses with long response lists is
dns.count.answers > 5.
It also helps to have a baseline that includes the idle period traffic and a sample of known updates/services dialogs to compare a questionable capture to.
Password-cracking traffic can be detected by observing numerous error messages from a target host directed to a client that repeatedly and unsuccessfully attempts to log in. There are two general types of password cracking attempts:
- Dictionary attacks work from a list of common words, names, and numbers
- Brute force attacks use a sequence of characters, numbers, and key values
Both of these types are often thwarted by login security measures that lock out an account after a short number of failed login attempts.
While it is difficult to anticipate what methods a hacker may use in an attempt to infiltrate a network or host, there are a few things that should probably never happen on a normal, healthy network. Due to their usefulness in testing and conveying error conditions, ICMP packets are a likely target for malicious redirection. Since TCP is the predominant transport protocol in use for most applications, you should look out for abnormalities in TCP headers or payloads that could be a sign of malicious intent.
Some examples of abnormalities to look out for are discussed in the following table:
The topics covered in this chapter on security analysis included detecting scans and sweeps to identify targets for planned attacks, operating system fingerprinting, detecting malformed packets, and packets that are suspiciously fragmented or sent out of order, phone home traffic from malicious agents, identifying password cracking attempts, and identifying other abnormal packets and payloads.
In the next chapter, we'll review several key command-line utilities provided in a Wireshark installation, as well as a few additional packet analysis tools that can complement your toolset.