Book Image

Instant OSSEC Host-based Intrusion Detection System

By : Brad Lhotsky
Book Image

Instant OSSEC Host-based Intrusion Detection System

By: Brad Lhotsky

Overview of this book

Security software is often expensive, restricting, burdensome, and noisy. OSSEC-HIDS was designed to avoid getting in your way and to allow you to take control of and extract real value from industry security requirements. OSSEC-HIDS is a comprehensive, robust solution to many common security problems faced in organizations of all sizes. "Instant OSSEC-HIDS" is a practical guide to take you from beginner to power user through recipes designed based on real- world experiences. Recipes are designed to provide instant impact while containing enough detail to allow the reader to further explore the possibilities. Using real world examples, this book will take you from installing a simple, local OSSEC-HIDS service to commanding a network of servers running OSSEC-HIDS with customized checks, alerts, and automatic responses. You will learn how to maximise the accuracy, effectiveness, and performance of OSSEC-HIDS' analyser, file integrity monitor, and malware detection module. You will flip the table on security software and put OSSEC-HIDS to work validating its own alerts before escalating them. You will also learn how to write your own rules, decoders, and active responses. You will rest easy knowing your servers can protect themselves from most attacks while being intelligent enough to notify you when they need help! You will learn how to use OSSEC-HIDS to save time, meet security requirements, provide insight into your network, and protect your assets.
Table of Contents (7 chapters)

Detecting SSH brute-force attacks (Intermediate)


The rules engine tracks its state; it knows what's been happening in the recent past. We can leverage that information to alert or squelch alarms based on frequency of events. The most common example of this is to detect and defend against SSH brute-force password attacks.

Getting ready

This is such a common problem the SSH brute-force detection and active response is built into OSSEC and enabled by default. We're going to take a look at the rules that interact to provide this functionality and provide an explanation of those rules. This will give us insight into how to write something similar for a web application log.

How to do it...

The decoder provides the analyzer with interesting elements out of the log messages, normally using regular expressions. Like the rules engine, the decoder is extensible and configurable. We can easily add decoders for our in-house application logs using the same techniques.

  1. Let's look at a decoder to extract the user and source IP from SSHD log messages involving login failures.

    The following snippet comes from OSSEC's default decoders in /var/ossec/etc/decoders.xml:

    <decoder name="sshd">
      <program_name>^sshd</program_name>
    </decoder>
    
    <decoder name="ssh-invfailed">
      <parent>sshd</parent>
      <prematch>^Failed \S+ for invalid user|^Failed \S+ for illegal user</prematch>
      <regex offset="after_prematch">from (\S+) port \d+ \w+$</regex>
      <order>srcip</order>
    </decoder>
    
    <decoder name="ssh-failed">
      <parent>sshd</parent>
      <prematch>^Failed \S+ </prematch>
      <regex offset="after_prematch">^for (\S+) from (\S+) port \d+ \w+$</regex>
      <order>user, srcip</order>
    </decoder>
    
    <decoder name="ssh-error">
      <parent>sshd</parent>
      <prematch>^error: PAM: Authentication \w+ </prematch>
      <regex offset="after_prematch">^for (\S+) from (\S+)$</regex>
      <order>user, srcip</order>
    </decoder>

    The following rules were selected from the distribution's rules/sshd_rules.xml file:

    <rule id="5700" level="0" noalert="1">
      <decoded_as>sshd</decoded_as>
      <description>SSHD messages grouped.</description>
    </rule>
    
    <rule id="5716" level="5">
      <if_sid>5700</if_sid>
      <match>^Failed|^error: PAM: Authentication</match>
      <description>SSHD authentication failed.</description>
      <group>authentication_failed,</group>
    </rule>
    
    <rule id="5720" level="10" frequency="6">
      <if_matched_sid>5716</if_matched_sid>
      <same_source_ip />
      <description>Multiple SSHD authentication failures.</description>
      <group>authentication_failures,</group>
    </rule>
  2. Let's examine the interaction between the decoder and rules so we can start using this for our own logs.

How it works...

Understanding how these messages are processed requires that we understand the path a log entry takes through the OSSEC analyzer. First, let's take a quick look at a flowchart, and then get into the specifics at each step. We'll look at an SSH-failed login message:

The log message starts by passing through the predecoder to extract the program name, the hostname, timestamp, and deconstructs the log message without the metadata. The next step is to pass through the decoder to add context to the event. The rules are then parsed and a matching rule adds its data to the event and converts it to an alert. If there is an action to be taken (e-mail, log, or active response) for this event, it is dispatched. The details of the event are then tabulated in the OSSEC server for statistical reporting and aggregated analysis. The event has now finished processing. We need to understand how the selected decoders and rules interact to understand how OSSEC can protect our servers from SSH brute-force attacks.

The SSH decoder starts off by anchoring itself to log messages with a program_name attribute starting with sshd. The ssh-invfailed, ssh-failed, and ssh-error decoders all anchor off the parent decoder; they're only loaded and evaluated if the SSHD decoder has matched. Each child rule uses a prematch operative that requires a simple regular expression to match before extraction is attempted.

Once that prematch is satisfied, the regex is called with an offset attribute set to after_prematch. Using this offset attribute, the string after the prematch string is handed to the regex attribute for extraction. Using the offset attribute shortens the string sent to the regular expression, eliminating duplication of matching operations.

For example, using our sample log message:

May  5 15:38:40 ether sshd[5857]: Failed password for root from 220.161.148.178 port 41037 ssh2

After the predecoder extracts the metadata, the decoders process the message as follows:

  1. The sshd decoder matches the program_name attribute to the value sshd.

  2. The ssh-invfailed decoder fails its prematch.

  3. The ssh-failed decoder succeeds its prematch.

  4. The string after the prematch string is sent through the decoder for extraction:

      root from 220.161.148.178 port 41037 ssh2
    
  5. The regex element extracts two tokens through the use of grouping parentheses:

     'root', '220.161.148.178' 
    
  6. The order element tells the decoder about user and srcip respectively.

  7. The log message is handed over to the rule engine for analysis as the decoder extracted data.

At this point, OSSEC is building an alert data structure for this message. The decoder uses its output to set the alert data for srcip and user. The decoded_as element is set to the parent of the decoder (sshd here).

The first rule (5700) serves as an anchor for the rest of the SSHD-based alerts. OSSEC uses this rule to form a tree of dependent rules. A parent rule referenced by other related rules helps optimize the path through the rules set. The 5700 rule matches successfully and the alert is set to level 0 and the rule_id attribute is set to 5700.

The rule 5716 anchors to the parent rule using an if_sid attribute with a value 5700. Since our sample message is from SSHD, the rule 5700 has matched and the next condition of the rule 5716 is evaluated. The message is checked to see if it starts with Failed or error: PAM Authentication. If the log entry matches, so that the alert level is updated to 5, the rule_id attribute is updated to 5716 and the group is set to authentication_failed. After this point, no other rules looking for a rule ID of 5700 will match on this message.

The rule 5720 has a frequency attribute set to 6. By setting the frequency, the analyzer will keep tabs on the number of times the rule is triggered in the next 6 minutes. We can use the timeframe attribute to change how long the analyzer tracks the number of times this rule is fired depending on our requirements.

The if_matched_sid attribute checks back 6 minutes to see if the rule_id value 5716 has been triggered 6 times. Remember, the rules are still being evaluated, so even though this rule is currently 5716, there's no guarantee it will stay that way through the rules tree. During the first evaluation, the if_matched_sid attribute checks on the rule_id value 5716 and returns zero results.

We have an additional constraint of this frequency/timeframe attribute to check that the alert is from the same source IP using the same_source_ip attribute. This first message does not match rule_id="5720" this time, but it creates our counter for rule_id="5720" and source IP 220.161.148.178 at 1. Assuming these are the only rules we're using, the rule engine finishes, and an alert with the following data is created:

{
  'srcip'      : '220.161.148.178',
  'rule_id'    : 5716,
  'level'      : 5,
  'dstuser'    : 'root',
  'group'      : 'authentication_failed,',
  'decoded_as' : 'sshd'
}

You may notice that the way the log is parsed it will actually take seven failed login attempts in 6 minutes to trigger the rule_id value 5720. On the seventh failed login attempt from 220.161.148.178, as any user, we'll finally hit the threshold on the rule_id value 5720, and the alert will be rewritten as follows:

{
  'srcip'      : '220.161.148.178',
  'rule_id'    : 5720,
  'level'      : 10,
  'dstuser'    : 'mysql',
  'group'      : 'authentication_failed,authentication_failures,',
  'decoded_as' : 'sshd'
}

The important steps are that the rule_id attribute is replaced, as is the level attribute, and the group attribute has been appended.