Book Image

Instant OSSEC Host-based Intrusion Detection System

By : Brad Lhotsky
Book Image

Instant OSSEC Host-based Intrusion Detection System

By: Brad Lhotsky

Overview of this book

Security software is often expensive, restricting, burdensome, and noisy. OSSEC-HIDS was designed to avoid getting in your way and to allow you to take control of and extract real value from industry security requirements. OSSEC-HIDS is a comprehensive, robust solution to many common security problems faced in organizations of all sizes. "Instant OSSEC-HIDS" is a practical guide to take you from beginner to power user through recipes designed based on real- world experiences. Recipes are designed to provide instant impact while containing enough detail to allow the reader to further explore the possibilities. Using real world examples, this book will take you from installing a simple, local OSSEC-HIDS service to commanding a network of servers running OSSEC-HIDS with customized checks, alerts, and automatic responses. You will learn how to maximise the accuracy, effectiveness, and performance of OSSEC-HIDS' analyser, file integrity monitor, and malware detection module. You will flip the table on security software and put OSSEC-HIDS to work validating its own alerts before escalating them. You will also learn how to write your own rules, decoders, and active responses. You will rest easy knowing your servers can protect themselves from most attacks while being intelligent enough to notify you when they need help! You will learn how to use OSSEC-HIDS to save time, meet security requirements, provide insight into your network, and protect your assets.
Table of Contents (7 chapters)

Writing your own rules (Simple)


So we have a running OSSEC server. It's configured to send us e-mails with alerts and we're getting a lot of e-mails. Not every alert is actionable or interesting in our environment. We can fine-tune alerts by overriding, supplementing, and enhancing the base rule set with our rules/local_rules.xml file.

By leveraging OSSEC's rules, we can tune rules based on the username, IP address, source hostname, URL, filename, time of the day, day of the week, rules matched, frequency, and time since last alert. The rules provide a powerful way to tweak the alerts we receive and are a great starting point for customization as no coding is required.

Getting ready

Before we start writing rules, we should be aware of some rules to be followed:

  • The first rule of writing custom rules is to never modify the existing rule files in the /var/ossec/rules directory except local_rules.xml. Changes to those rules may modify the behavior of entire chains of rules and complicate troubleshooting.

  • The second rule of writing custom rules is to use IDs above 100000 as IDs below it are reserved. Interfering with those IDs is the same as tampering with the distributed rules files themselves. You risk an update of OSSEC clobbering all your hard work.

  • The third rule is to maintain order in your rules. As the rules parser is loading the rules at startup, it validates the existence of referenced rules and groups. If you reference a rule that hasn't been loaded, the parser will fail.

Minding these three rules helps to ensure that our installations won't break with upgrades and that we can always get back to a "stock" OSSEC by removing the local_rules.xml file from our configuration.

Every rule must have an ID, a level, a description, and a match condition. The IDs must be unique, and our rules must have an ID over 100000. It's important to note that re-using or reordering rule IDs can cause confusion or inaccuracy in historic data.

Rules in OSSEC have a level from 0 to 15. The higher the level, more certain the analyzer is of an attack. Level 0 is a special level to tell OSSEC to ignore the alerts where no log will be generated and OSSEC will discard the alert and data silently. By default, OSSEC considers anything at or exceeding level 7 to be e-mail worthy, but it is also configurable.

Rules also require a description field to explain what the rule does. This description will be used as the event identifier in the e-mails and log messages that OSSEC generates. As it will be a part of the reporting, it's best to explain the rule professionally and format it consistently. Descriptions like "alerting for this thing" won't be helpful to your colleagues, whereas "Ignore failed login attempts from vulnerability scanners between 4:00 and 7:00" will be clear and informative.

Now that we're well versed with the protocols of the rules, let's process some data from our custom application logging via syslog as follows:

May 4 19:12:03 server custom-app: Startup initiated.
May 4 19:12:07 server custom-app: No error detected during startup!
May 4 19:12:08 server custom-app: Startup completed, processing data.
May 4 19:12:08 server custom-app: Failed login from '4.5.6.7' as testuser

We're receiving an alert about unknown errors and authentication failures from our custom application. We would prefer to silence these unknown error messages and ensure that we don't provide alerts for failed logins from 4.5.6.7, our vulnerability scanner.

How to do it...

In order to figure out the first step, we need to understand what's happening to generate the alerts:

  1. Use the ossec-logtest tool provided by OSSEC. It works by accepting log messages on STDIN (your terminal input) and explaining the path through the rules. Here's how we can run it:

    $ sudo /var/ossec/bin/ossec-logtest
    
  2. Then we can paste in log lines to see which ones are generating alerts:

    ossec-testrule: Type one log per line.
    
    May  4 19:12:03 server custom-app: Startup initiated.
    
    **Phase 1: Completed pre-decoding.
         full event: 'May  4 19:12:03 server custom-app: Startup initiated.'
         hostname: 'server'
         program_name: 'custom-app'
         log: 'Startup initiated.'
    
    **Phase 2: Completed decoding.
         No decoder matched.
    
  3. The first log message completed the parsing of the line and no alert was generated. So we try the next log message:

    **Phase 1: Completed pre-decoding.
         full event: 'May  4 19:12:07 server custom-app: No error detected during startup!'
         hostname: 'server'
         program_name: 'custom-app'
         log: 'No error detected during startup!'
    
    **Phase 2: Completed decoding.
         No decoder matched.
    
    **Phase 3: Completed filtering (rules).
         Rule id: '1002'
         Level: '2'
         Description: 'Unknown problem somewhere in the system.'
    **Alert to be generated.
    
  4. We can see from this output that our unknown problem is being generated by this log line. We get the rule ID and the level being generated and using this information, we can write a rule to ignore it using OSSEC's level="0".

    <!-- Local Rules for Example.com -->
    <group name="local,syslog,">
       <rule id="100000" level="0">
         <if_sid>1002</if_sid>
         <program_name>custom-app</program_name>
         <description>Ignore errors for custom-app</description>
       </rule>
    </group>
  5. Once we've saved the local_rules.xml file, we can restart ossec-logtest and try the event again:

    **Phase 3: Completed filtering (rules).
         Rule id: '100000'
         Level: '0'
         Description: 'Ignore unknown errors for custom-app'
    
  6. Now that we've moved this event to level 0, we can look at the failed login events:

    May  4 19:12:08 server custom-app: Failed login from '4.5.6.7' as testuser'
    **Phase 3: Completed filtering (rules).
         Rule id: '2501'
         Level: '5'
         Description: 'User authentication failure.'
    **Alert to be generated.
    
  7. We'll use a simple match with this data to silence this alert from 4.5.6.7:

    <rule id="100001" level="0">
      <if_sid>2501</if_sid>
      <program_name>custom-app</program_name>
      <match>4.5.6.7</match>
      <description>Ignore failed logins from scanner</description>
    </rule>
  8. And now re-running ossec-logtest, we see:

    May  4 19:12:08 server custom-app: Failed login from '4.5.6.7' as testuser'
    
    **Phase 3: Completed filtering (rules).
         Rule id: '100001'
         Level: '1'
         Description: 'Ignore failed logins from our security scanner'
    **Alert to be generated.
    

Using these two rules, we've been able to silence the noisiest log entries in our sample environment.

How it works...

OSSEC rules are processed sequentially. Each rule has a number of conditions and a logical AND is applied to the conditions. The more specific we make the rule, the more accurate it will be. In our example, we filtered events using the program_name attribute for the string, custom-app.

Each rule also specifies an if_sid element, which requires the log message to be flagged with the rule ID we specify. The if_sid parameter can take a comma-separated list of rule IDs, where the rule will match if any of those IDs are matched. Consider that multiple instances of the same element appear in a rule; refer to the following example:

<match>illegal user</match>
<match>unknown user</match>
<match>invalid user</match>

That grouping is surrounded in logical OR. It's important to note that after our rules match, the ID is changed and other rules looking for those lines with the same if_sid parameter, which was originally set, will fail to match. When a new rule matches, it replaces the attributes of the alert with its own values, replacing the ID and level.

While our first rule stopped at eliminating the match based on the rule ID and program name only, the second rule used the match attribute to find a string in the log message itself. In addition to match, there is also a regex attribute to allow more flexible matching of strings.

To see a complete list of fields available, check out the OSSEC documentation on rules' syntax available at the following link:

http://www.ossec.net/doc/syntax/head_rules.html

There's more...

OSSEC rules are quite capable. We saw how to modify an alert based on the if_sid parameter, which is the rule ID. We saw that we can adjust the rule level using the level of the new rule. Most cases will involve this type of rule-level promotion or demotion depending on the context. We can evaluate events based on a number of fields. We used ossec-logtest to see some of those fields, but we're missing data. In our examples, we saw the following message:

**Phase 2: Completed decoding.
       No decoder matched.

Without a decoder, our event data is limited to simple string matches. Let's take a look at what decoders are and how they work.

Decoding event data

Our custom application does not have a valid decoder because we didn't build one. Not every program needs a decoder and we were able to be effective without it. Decoders provide a way to extract field data from the message to be used in context, but not every rule needs a decoder. We saw some of these fields in the pre-decoding phase where common data are extracted:

**Phase 1: Completed pre-decoding.
     full event: 'May  4 19:12:08 server custom-app: Failed login from '1.2.3.4' as testuser''
     hostname: 'server'
     program_name: 'custom-app'
     log: 'Failed login from '1.2.3.4' as 'testuser'

In our rule, we even used the program_name attribute to match our custom application from the log stream.

OSSEC ships with an array of decoders out of the box, consider the output of this default install using ossec-logtest to read an SSH log message:

May  4 19:11:01 ether sshd[3747]: Accepted publickey for brad from 1.2.3.4 port 48801 ssh2

**Phase 1: Completed pre-decoding.
     full event: 'May  4 19:11:01 ether sshd[3747]: Accepted publickey for brad from 1.2.3.4 port 48801 ssh2'
     hostname: 'ether'
     program_name: 'sshd'
     log: 'Accepted publickey for brad from 1.2.3.4 port 48801 ssh2'

**Phase 2: Completed decoding.
     decoder: 'sshd'
     dstuser: 'brad'
     srcip: '1.2.3.4'

**Phase 3: Completed filtering (rules).
     Rule id: '5715'
     Level: '3'
     Description: 'SSHD authentication success.'
**Alert to be generated.

To learn more about decoders, including writing custom decoders, refer to the OSSEC documentation on decoders:

http://www.ossec.net/doc/manual/rules-decoders/create-custom.html