Analysis and production
Analysis and production can be thought of as the interpretation step where the processed data is converted into indicators of compromise, alerts, and alarms, with the capability to notify all the relevant parties of any potential threats. The results should be presented in perfect harmony with the objectives and requirements that were collected in the first phase (planning and direction). There is no one specific output format for presenting the analysis of an intelligence project. It is essential to understand the consumers before providing the results. This step is the livelihood of the intelligence project; that is, the main reason for its existence. Hence, the analyst or CTI team needs to pay attention to it.
Although collecting and processing intelligence data is automated, interpreting the results requires human expertise. And this is where human errors cause disruptions. This is known as bias and needs to be avoided when analyzing the processed data. Bias is causally linked to personal views, opinions, and interpretation of the intelligence result. CTI is an evidence-based product and process. Hence, every analysis should be supported by clear evidence – for example, an analyst who supports a specific theory without evidence based on experience or their gut feeling. The analyst then looks for evidence that supports the idea and rejects any other evidence that doesn't support the theory. This kind of analysis will result in a higher bias toward supportive facts.
One of the most commonly used methods is structured analytic techniques (SAT), created by the United States Government. It is used to implement an unbiased solution and improve intelligence analysis. SAT will be covered in detail in Chapter 3, Cyber Threat Intelligence Frameworks, as a form of tradecraft. SAT is used by several private sectors and intelligence analysts, including the CIA. Its primary objective is to minimize judgment and control uncertainties that can happen during analysis. This method uses three different techniques, grouped by their purpose:
- Diagnostic techniques: These techniques focus on transparency. As approached by SATs, diagnostic techniques use arguments and assumptions to support decisions or threat analysis output. The idea behind this method is to ensure that intelligence analysts do not discard any relevant hypotheses. Some of the techniques in this category are as follows:
a. Quality of information check: This is where the comprehensiveness of the data that analysis is or needs to be performed on is benchmarked. This category provides grounds for confidence in the analytic evaluation and results in a precise assessment of what is provided by the intelligence platform.
b. Indicators of change: While exploring and analyzing the intelligence output, it is imperative to observe indicators regarding sudden data changes. This method is useful when the CTI team or an analyst wants to track activities specific to a target or an adversary. This method avoids bias by adding credibility to the analytics result.
c. Analysis of competing hypothesis: Suppose that the CTI team collected and processed a large amount of data. In this method, every CTI analyst provides an interpretation of the analysis. Cross-evaluation is then done in the form of a challenge, where hypotheses are compared based on their efficacity and the evidence that supports them. The best approach to using the competing hypothesis is to create a matrix of analysis.
- Contrarian techniques: These techniques challenge a specific hypothesis. The idea is to eliminate bias through contradiction. The analysts contradict even the most founded intelligence analysis interpretation to collect more evidence to support it. Some of the popular methods that are used in this category of techniques include the following:
a. The devil's advocate: As the name implies, this method challenges a strong interpretation of the result by developing and supporting alternative interpretations. Suppose that after intelligence analysis is performed, indicators showing threats from Chinese IP addresses emerge. The entire team concludes that Chinese IP addresses are trying to communicate with a certain system application.
Using the devil's advocate, a brave analyst challenges this conclusion by saying that those IP addresses belong to another country and that proxychains and VPNs were used to mask the adversary's real origin. Now, the team uses the contradicting hypothesis to prove that the threats originate from China. This method removes bias by showing how confident the team is in their interpretation.
b. AB team: This is one of the most prominent methods. The manager or the CTI team leader divides the group into two teams: A and B. The two teams challenge each other by competing when it comes to interpreting the intelligence result. Moreover, it is essential to draw a line between the AB team and the devil's advocate approach. The former is used when there is more than one interpretation of the same analysis. The objective should remain the same: discussing how to eradicate everyone's bias mindset by making them defend an interpretation they do not agree upon.
c. What-if analysis: In the example provided for the devil's advocate, instead of confirming the team's opposing thoughts, an analyst should ask, what if the IP addresses are not from China? The focus is on how is it possible to have China's IP addresses as a threat? The team can then focus on parameters that might have enabled the presence of Chinese IP addresses in the system.
- Creative thinking techniques: These techniques produce new interpretations or insights regarding the analysis. This allows analysts to create further analysis angles and produce alternative results to the primarily completed study. Imaginative thinking includes several popular methods, such as the following:
a. Brainstorming: Brainstorming involves generating new concepts, ideas, theories, and hypotheses around the analysis results. The CTI team must use brainstorming to promote creativity and push analysts to think outside the box. It is used to reduce bias as analysts are likely to step away from their clouded opinions to develop fresh new ideas – every concept matters. The CTI team leader should consider all analysts' views and understand the triggering points of those ideas.
b. Red team analysis: The most technical approach to intelligence analysis is when the analyst wears the adversary's dress. In red team analysis, the CTI analyst tries to replicate the adversary's threat method (how an adversary attacks, how they think, and so on). When performing threat intelligence analysis, it is vital to take a red team approach because it assumes the worst scenario, and it also helps the team prepare a defense mechanism that can resist the most potent of threats. The analyst becomes a white adversary. Note that this kind of analysis is complicated, time-consuming, and resource-intensive. This is because an exceptional team of analysts needs to be implemented to simulate the adversary.
c. Outside-in thinking: The CTI team must always look at the external factors that can easily influence the analysis. The intelligence analyst should be able to identify the forces that impact the analysis. For example, what are the key elements that might push China to be a cyber threat? Factors such as politics, socioeconomics, and technology should be considered when doing critical thinking regarding an analyzed threat.
In most cases, the CTI team uses the three techniques described here to perform an approximate complete and unbiased analysis. Each technique has several key components that need to be checked to validate their application (more details will be covered in Chapter 3, Cyber Threat Intelligence Frameworks).
The analyst should also establish or identify relationships between different threats and adversaries during the analysis step. This helps with finding a correlation, patterns, or unique characteristics between different threat actors (for example, a current threat might have the same properties as a past threat). The diamond model is one of the universally used models for clustering and correlating threats and adversaries.
With that, we have explained what needs to be done during the analysis and exploration step, as well as what methodologies a CTI team can use to yield a useful analysis and interpretation. More details on how this can be done, along with examples, will be provided later in this book. We will also include a short overview of the biases that can mislead a threat intelligence operation.