What is good CTI?
Almost anyone can generate threat intelligence. However, not everyone can generate good threat intelligence. In order to generate threat intelligence that is considered good and is useful, there are five key traits to consider in combination with the Admiralty, source, and data credibility ratings. When combining all of these key concepts together, the end result should generate timely, accurate, and useful threat intelligence.
Let's look at the traits of good CTI.
The five traits of good CTI
When thinking of CTI in general, there are five key traits that can be distilled down to illustrate what constitutes good CTI.
Those five traits include the following:
- Accuracy: Is the intelligence correct in every detail? This is a key concept ensuring that only accurate intelligence is retained.
- Completeness: How comprehensive is the intelligence? Completeness helps ensure all related intelligence is gathered and collected.
- Reliability: Does this intelligence contradict other trusted sources? Reliability means that a piece of information is reliable and doesn't conflict with another piece of information or data in a different source or system. When data or intelligence conflicts from two sources, that intelligence then risks becoming untrustworthy.
- Relevance: Do you really need this intelligence, that is, in terms of the geographical location and/or nature of the business your organization is in? Looking at relevance establishes a need for intelligence. If irrelevant intelligence is being gathered, time is being wasted along with the possible pollution of current or future collected intelligence.
- Timeliness: Is the intelligence up to date? Simply put, intelligence that isn't timely can lead to analysts making the wrong decisions based on historical or incorrect intelligence. Timeliness ensures decisions aren't made with stale information.
There are many methods available to ensure the accuracy, completeness, reliability, relevance, and timeliness of intelligence. However, one tried and true method for ensuring those are met is a framework called Admiralty.
Admiralty ratings
The Admiralty System or NATO System is a method for evaluating and rating collected intelligence. It consists of a two-character notation that evaluates the reliability of the source and the assessed level of data credibility of the intelligence. Employing Admiralty ratings to collect intelligence is an important data quality and source reliability assessment tool.
Source ratings
Understanding the reliability of an intelligence source (automated, semi-automated, or human) is paramount when considering onboarding an intelligence source. A source rating should be applied to intelligence that is collected and analyzed.
Applying a source rating is an important process in CTI as it serves as a historical ledger of activity of the source of the intelligence, making it easier for perusal in the future. When examining source ratings, sources are classified in order of decreasing reliability, with A being the most reliable:
Table 1.4 – Data and intelligence source reliability scale
Source ratings play an important part in any CTI program. Source ratings help establish a baseline trust rating for any source – whether that is data or human in scope. In the following section, we're going to discuss an additional part of CTI: data credibility ratings.
Data credibility ratings
Within CTI, it's important to trust but verify the data sources of threat intelligence. Assigning a credibility rating to threat intelligence helps to establish the fundamental accuracy of an organization's CTI program. Additionally, when employed, credibility ratings help establish a profile of the intelligence that is being collected. And finally, data credibility, while somewhat subjective, helps eliminate confirmation bias by seeking independent source validation.
Data credibility ratings measure the levels of corroboration by other sources. When examining source ratings, the credibility is classified in order of decreasing credibility, with 1 being confirmed by independent sources:
Table 1.5 – Data credibility ratings
Data credibility ratings help a CTI organization judge the credibility of the data they are ingesting. While data credibility ratings play a crucial role in CTI, fusing the data credibility rating with source ratings makes for a great combination to assess data and intelligence accurateness, reliability, and trustworthiness.
Putting it together
In principle, it should be easy to apply Admiralty codes to threat intelligence, but in practice, it's more difficult. The question that often arises is, ultimately, what data and intelligence can we trust?
While that answer will vary, one method to consider employing is from a paper titled The Admiralty Code: A Cognitive Tool for Self-Directed Learning, written by James M. Hanson at the University of New South Wales (2015; https://www.ijlter.org/index.php/ijlter/article/download/494/234).
Using Table 1.5, it's easy to start applying source and credibility ratings to collected CTI:
Table 1.6 – The Admiralty code for evaluating data credibility
Using the preceding table as an example in which to apply to threat intelligence, an information security industry threat intelligence blog would be considered B1, which is usually reliable and confirmed and can, thus, be considered credible.
A second example would be intelligence from a little-known independent researcher on their personal blog with no independent confirmations. This intelligence could be rated F3, or the source cannot be judged, and the credibility of it would be possibly true, requiring additional investigation.
Employing Admiralty ratings in conjunction with intelligence life cycles in a CTI program is a generally accepted mechanism to enable a CTI program. Let's move on to threat intelligence life cycles next.