Book Image

Practical Threat Intelligence and Data-Driven Threat Hunting

By : Valentina Costa-Gazcón
Book Image

Practical Threat Intelligence and Data-Driven Threat Hunting

By: Valentina Costa-Gazcón

Overview of this book

Threat hunting (TH) provides cybersecurity analysts and enterprises with the opportunity to proactively defend themselves by getting ahead of threats before they can cause major damage to their business. This book is not only an introduction for those who don’t know much about the cyber threat intelligence (CTI) and TH world, but also a guide for those with more advanced knowledge of other cybersecurity fields who are looking to implement a TH program from scratch. You will start by exploring what threat intelligence is and how it can be used to detect and prevent cyber threats. As you progress, you’ll learn how to collect data, along with understanding it by developing data models. The book will also show you how to set up an environment for TH using open source tools. Later, you will focus on how to plan a hunt with practical examples, before going on to explore the MITRE ATT&CK framework. By the end of this book, you’ll have the skills you need to be able to carry out effective hunts in your own environment.
Table of Contents (21 chapters)
Section 1: Cyber Threat Intelligence
Section 2: Understanding the Adversary
Section 3: Working with a Research Environment
Section 4: Communicating to Succeed
Appendix – The State of the Hunt

The intelligence cycle

Before we dive into the theory of the intelligence cycle, I believe it is worth showing the relationship between data, knowledge, and intelligence practice through what is known as a knowledge pyramid. In it, we can see how the facts, through measurement, are transformed into data that we can extract information from when processing it. When analyzed together, it can be transformed into knowledge. This knowledge interacts with our own experience and forms the basis of what we call wisdom. It is this ultimate wisdom that we rely on for decision-making.

As shown in the following pyramid, we can intertwine this knowledge pyramid with the processes that are part of what is widely known as the intelligence cycle:

Figure 1.2 – DIKW pyramid

Figure 1.2 – DIKW pyramid

In short, here, we can deduce that an intelligence analyst must process data to transform it into wisdom (intelligence), which in the last instance will lead to an action (decision).

Traditionally, the intelligence process is understood as a six-phase cycle: planning and targeting, preparation and collection, processing and exploitation, analysis and production, dissemination and integration, and evaluation and feedback. Each of these phases presents its own particularities and challenges:

Figure 1.3 – The intelligence cycle

Figure 1.3 – The intelligence cycle

We will now look at each of these phases in detail.

Planning and targeting

The first step is to identify the IR (IRs). Any information that the decision makers need and don't know enough about falls under this category.

In this stage of the process, it is important to identify the key assets of the organization, why the organization might be an interesting target, and what the security concerns of those in charge of making decisions are.

It's also important to identify the potential threats that exist and what mitigations can be prioritized (through a process known as threat modeling), as well as establishing a collection framework and collection priorities.

Preparation and collection

This stage refers to defining and developing collection methods to obtain information regarding the requirements that were established in the previous phase.

It is important to keep in mind that it's impossible to answer all the questions we may have and meet all our IR.

Processing and exploitation

Once the planned data has been collected, the next step is to process it to generate information. The processing method is usually not perfect, and the amount of data that the intelligence team is able to process is always lower than the amount of data that has been gathered. All data that does not get processed is the same as data not collected at all. It's lost intelligence.

Analysis and production

The information that's been gathered so far must be analyzed in order to generate intelligence. There are several techniques that are used for intelligence analysis and to prevent the analyst's bias. The cyber threat intelligence analyst must learn how to filter their personal views and opinions to carry out the analysis.

Dissemination and integration

In this stage, the intelligence that's been produced is distributed to the necessary sectors. Before distribution, the analysts have to consider a variety of things, such as what the most pressing issues are among the intelligence that's been collected, who should receive the report, how urgent the intelligence is or how much detail the recipient needs, if the report should include preventive recommendations, and so on. Sometimes, different reports may need to be created and directed to different audiences.

Evaluation and feedback

This is the final stage of the process and probably the most difficult to achieve, mainly due to the usual lack of feedback from intelligence recipients. Establishing good mechanisms to get feedback helps intelligence producers evaluate the effectiveness of the intelligence that's been generated before they repeat the process over and over, without making the necessary adjustments that will make the intelligence that's produced more relevant to the recipients. As intelligence producers, we want our intelligence to be relevant – we want our intelligence to help the decision makers to make informed decisions. Without gathering the appropriate feedback, we won't know if we are achieving our goal, and we won't know which steps to take to improve our product.

This model has been widely accepted and adopted, especially in the United States of America and among those who follow their academic discussions in an attempt to replicate its methods. Despite this wide acceptance, there have been some vocal criticisms against this model.

Some have pointed out that the current model depends excessively on the data that's been collected, and also that technological advances have allowed us to collect massive amounts of it. This endless harvesting process and the capacity to better represent the data that's been collected leads us to believe that this process is enough for us to understand what is happening.

There have been alternative proposals for the intelligence cycle. For anyone interested in studying more on this matter, there is a particularly interesting contribution that's been published by Davies, Gustafson and Ridgen (2013) titled The Intelligence Cycle is Dead, Long Live the Intelligence Cycle: Rethinking Intelligence Fundamentals for a New Intelligence Doctrine (, in which what has been labeled the UK Intelligence Cycle is described in detail:

Figure 1.4 – The Core Functions of Intelligence (JDP 2-00) (3rd Edition)

Figure 1.4 – The Core Functions of Intelligence (JDP 2-00) (Third Edition)

Now, let's learn how to define and identify our IR.