Tag: threat intelligence

The juggernaut known as the “threat intelligence sharing imperative.” Security and industry conferences fill their time with “sharing.” How many sharing groups and platforms do we require? Too many exist. Alien Vault recently reported that 76% of survey respondents reported a “moral obligation to share threat intelligence.” McAfee says sharing threat intelligence “is the only way we win” (that isn’t even remotely true). However, it’s not working.

According to Robert Lemos in eWeek, even with the most recent US cyber security legislation providing legal immunity organizations are not rushing to share. The reason is simple. That was only one component of a complicated problem. While the legislation addressed one policy element, it didn’t address that sharing has never been proven (with data) to benefit sharing organizations.

We must move beyond these “religious” arguments and provide clear incentives for defenders to share.

In January, President Obama signed the Cybersecurity Act of 2015, but companies remain in a holding pattern, waiting for legal clarity and demonstrable benefits before sharing sensitive information.

– Robert Lemos, eWeek “Cyber-Threat Data Sharing Off to Slow Start Despite U.S. Legislation” [2016-10-02]

The Loudest in the Room

There is one thing I notice – security vendors yell the loudest about sharing. I don’t claim their sharing narrative is FUD, but the sharing narrative is a net positive for them. The more data and intelligence they receive strengthen their products and services adding value to their organization. Security vendors have strong incentives to promote threat intelligence sharing. But, what is the case that the cost of sharing to defenders is a net benefit to them?

Security vendors have strong incentives to promote threat intelligence sharing. But, what is the case that the cost of sharing to defenders is a net benefit to them?

Sharing is Costly

I’ve been involved in threat intelligence sharing for a long time. I am the first to support the notion of sharing. I have story up on story which supports the sharing narrative. But, I qualify my support: the value of sharing must exceed the cost.

Most network defenders will agree: sharing is costly.

It requires significant cost to integrate externally shared threat intelligence effectively.
Once you consume that threat intelligence you quickly discover it may consume your security team with poor quality – and requires significant tuning. There is risk.
Establishing a sharing mechanism, program, and process is costly. It usually requires engineering effort.
Management support for sharing usually requires political capital from network defense leaders. They must prove that the resources spent on sharing are more important than the 20 other components competing for resources. Also, let’s not forget about the legal support.

An Incentive-Based Approach

Sharing must go beyond a “religious” argument. Instead, we must take an incentive-based approach. We must create and promote incentives for defenders to share – with demonstrable results. Therefore, those promoting sharing must provide a coherent and consistent data-driven case that sharing overcomes these costs to defending organizations. “Share because it is good for you” is not enough.

So, next time you advocate for sharing – enumerate why network defenders should share. Make it meaningful. Make it data-driven.

Indicators and Security Analytics: Their Place in Detection and Response

By Sergio Caltagirone

On September 21, 2016

In Detection

Indicators for research and response; analytics for detection

Indicators of Compromise (IOCs), the lingua franca of threat intelligence. Almost every intel sharing conversation begins and ends with indicators; commercial intelligence platforms revolve around them; most intelligence consumers end their interest there. Does a better way exist? Security analytics!

The Problem with Indicators in Detection

For all the focus given to indicators we know that they have the shortest lifespan of all intelligence exhaust (see the Pyramid of Pain by David J. Bianco). In many cases, we see single use or victim specific indicators making sharing of these useless. In general, adversaries tend towards shortening the indicator lifespan – or removing them; for instance Locky recently transitioned to hardcoded RSA keys to remove the vulnerability of connecting to a command and control (C2) server.

Broad based indicator sharing is fraught with problems. First, it assumes that the same indicators will be leveraged against multiple victims. This is certainly the case for some threats. But not all. Second, quality will likely be a problem. For instance, DHS Automated Indicator Sharing (AIS) states:

Indicators are not validated by DHS as the emphasis is on velocity and volume: our partners tell us they will vet the indicators they receive through AIS, so the Department’s goal is to share as many indicators as possible as quickly as possible. However, when the government has useful information about an indicator, we will assign a reputation score. – DHS Automated Information Sharing

Further, AIS contributors can choose to remain anonymous. Think about the problems of blindly consuming thousands of non-validated anonymously sourced indicators. How exactly do you effectively validate an anonymously contributed indicator? Previously, I wrote on the cost of poor intelligence. Just one instance of 8.8.8.8 by an anonymous contributor could cause massive issues.

Indicators of Compromise are only threat exhaust – the necessary by-product of malicious activity. Short-lived and increasingly single use, indicators pose a poor basis for detection – and it’s getting worse. I’m not advocating for throwing indicators out entirely – they serve their purpose, but should not form the entire basis of threat intelligence detection.

Analytics For Detection

As the Pyramid of Pain suggests, we must move towards behavioral based detection focusing on whole classes of threats. I’d much rather rely on an analytic detecting overwriting Windows registry keys for a “sticky keys” attack than hoping someone shares an IP address of a random hop point used before to remote desktop (RDP) into a host. In the analytic case I catch every adversary using sticky keys, in the case of the indicator I catch only one adversary – with the hope they use the same infrastructure again.

Where do you find analytics?

The best place is your red team – ask them to describe their techniques and procedures. Read their reports! (I know – a stretch for some)
Read threat intelligence reports on adversary behaviors.
Ask your threat intelligence provider! (Who you already abuse with information requests anyways – right?)
Check out MITRE’s Cyber Analtyics Repository.

The Place for Indicators – Research and Response

Indicator sharing works within a small group of organizations that share a “victim space” (as the Diamond Model refers to victims with shared threats). This greatly increases the value of shared indicators because the likelihood of attackers reusing indicators increases. However, indicator sharing outside the “shared victim space” reduces their value and increases their cost. Research and response receive the greatest value from shared indicators as it allows a method of communicating observables discovered in attacks allowing analysts to pivot deeper into malicious activity seen by others.

Your Own Intelligence is the Best

In the end, to achieve greater detection capability organizations must invest in security analytics and reduce their reliance (and insistence) on indicators from externals. The best indicators in the world are those from your organization’s own telemetry – your own threat intelligence is the most relevant. Otherwise, look suspiciously at indicators from others and instead ask to share analytics!

Note: Security analytics are a dirty word – overused and often misused. To be clear, I define analytics in this post as indicator-independent behavioral detection derived from the knowledge of bad stuff (i.e. Threat Intelligence)

13 Principles of Threat Intelligence Communication

By Sergio Caltagirone

On March 9, 2016

In Best Practices

I have written at length about bad threat intelligence. However, I think it is time that I spend the effort communicating my key principles to making great threat intelligence. One aspect of great threat intelligence is great communication. As I have said before, you may be the greatest analyst in the world, but if you can’t effectively communicate your knowledge then it is of little use.

I’ve found these principles apply to all modes of my communication when discussing threat intelligence with others. They’ve guided me well and I hope they do the same for you.

Answer the Three Questions

All threat intelligence communication should work towards answering three critical questions, if you clearly articulate the answer to these questions your communication will be generally successful.

What is it? (give me the information)
Why should I care? (tell me about the threat and its relevance to me)
What am I going to do? (enable my decision and action)

Maintain Your Focus

Focus is key to your communication – understand your audience and your objective and maintain that throughout. Here are some elements which help me:

Remember the four qualities of good intelligence (CART): completeness, accuracy, relevance, and timeliness. Fulfill them as best you can.
Remember the purpose of threat intelligence to inform and enable effective decision-making, whether that be tactical/technical, operational, or strategic. You don’t need to provide EVERYTHING, only that which will support and enhance the intelligence.
Length matters: your communication should be as long as it needs to be but never longer than it should be. Here’s a secret: it’s okay to not communicate everything in one vehicle – sometimes separating the material makes the threat intelligence more effective.
Don’t derail your audience. After reading your 30 page report, make sure I know the value of the information and that you’ve addressed the key questions. For example: don’t all of a sudden drop an unrelated element in your conclusion just because you want to make a point.

Analytic Integrity is All You Have

Intelligence is about trust. When people can’t independently verify your findings and conclusions (which most won’t/can’t) then they must trust you. You must create, support, and encourage that trust by practicing analytic integrity in your communications. If you break that trust you lose your integrity and nobody will listen to you. Here are some of my rules to creating and encouraging trust with your audience:

Don’t lie – if you don’t know, just say that
Don’t embellish – don’t use hyperbole or language which might cause an over-reaction
Don’t plagiarize – never intentionally (and avoid accidents) copy the work of another
Practice humility – hubris infers overcompensation for weakness, be bold but not stupid

Be a Storyteller

Threat intelligence is a story – tell it as one. Threat intelligence should have a beginning, middle, and an end. Engage your audience.

The Summary IS the Communication

I know it sounds weird, but your summary is the most important part of your communication. This is what people will remember and what they’ll rely on most afterwards. For many, this is the only part to which they’ll pay attention. The summary (or key points, etc.) should be par excellence. I instruct analysts to spend at least 20% of their time on their summary and conclusion – it is that important.

As the old adage goes: “tell them what you’re going to tell them, tell them, tell them what you told them.” This is CRITICAL advice and not often heeded by technical analysts.

However, I want to caution you. Others suggest that following this old adage only bores an audience. I agree that it is a pitfall for most, only because many follow the guidance without understanding it. Avoid the summary and conclusion containing the same bullet points or phrasing – that is boring. However, your summary/introduction/key points/etc. and your conclusion should carry your key message and information, but in different ways.

Language Matters

The language you use greatly determines the effectiveness of your communication.

Use Active Voice – this isn’t some joke or regurgitation of high-school English. It matters. Active voice has been proven to decrease ambiguity and increase comprehension. It improves your intelligence.
- Science: “Certain syntactic constructions are known to cause the processor to work harder than others. Sentences with passive verbs are more difficult to comprehend than those with active verbs (Gough 1966; Slobin 1966; Olson and Filby 1972; Ferreira 2003) since they not only reverse the standard subject-verb-object order of the participants but are often used without a byphrase , which omits one participant altogether and can obscure the grammatical relations.”
Use Estimative Probability – judgements, hypotheses, and conclusions are never 100% certain; use words of estimative probability to clarify your certainty to your audience.
Clarity wins over all – don’t use complex language when simple will do.
Minimize subjective qualifications – avoid words/phrases like (sophisticated adversary) or (complex encryption) unless you can measure them either objectively or in comparison with others. These phrases only add ambiguity.
Words mean things – don’t dilute your language or create a phrase when one already exists.
Analysis is not a religion – don’t use the word believe; hold measured judgements expressed in language differentiating fact and hypothesis.

Value Your Audience

Value their intelligence and their time. They are not fish caught by click-bait or hyperbole but respected for their interest in your work. Your audience is spending time with you because they think you have something valuable to communicate and they have come to learn something new – GIVE IT TO THEM! Or, they will leave you.

Images are Powerful

Use images strategically to tell your story, reinforce critical concepts, and increase accessibility and understanding. Images should not become overwhelming, distracting, or superfluous.

Write for Your Future Self

Communicating intelligence and analysis is HARD. It’s hard because you’re trying to take a very complex cognitive process and share that with others. I’m not the only one who has read something they wrote a year ago only to scratch my head and wonder what I was smoking. I’ve found that to make this easy I simply imagine that I’m communicating to my future self – say 1, 2, or 3 years from now. This helps ensure that I include important details which are obvious now but will be lost later. Further, it ensures that I make my logic chains clear and easily followed by others.

Don’t be an Island

Be part of the community. Respect the community. Expand on the work of others and fill in knowledge gaps. Confirm others’ findings and add support to their conclusions or hypotheses. Add exculpatory evidence and provide alternative hypotheses. And here’s a secret: it’s okay to point to the analysis of others in your communication – you don’t always have to self-reference. This actually adds value for your audience and makes you more valuable to them because they trust you’re going to tell them the whole story – not just your story.

Respect Your Adversaries

Don’t belittle adversaries in your threat intelligence. Don’t give them undue credit, but also don’t take away from their effectiveness. This will only lead to hubris – and hubris is deadly. We all know of an analyst who called a threat “unsophisticated” or “simple” only to later report a massive compromise.

Be Bold, Be Honest, Be Right, But Always Be Willing to be Wrong

I’ve said it before, I like my analysis like I enjoy my coffee, bold. I want analysts to be analysts – not reporters. I want to hear ideas, conjecture, assessment, opinions. I want those clearly separated from the facts.

Separate Fact From Everything Else

This is a pretty simple rule. But harder to follow in practice while working through a complex analysis. Strive to use language, format, font, etc. to separate fact from hypothesis. Because threat intelligence enables decision-making, decision makers (whether a SOC analyst, a CIO, or whoever) should make their own judgement based on your analysis. If your facts and hypotheses are indistinguishable it is highly likely they’ll make poor decisions based on misinterpreted analysis.

Cyber Threat Language Dilution

By Sergio Caltagirone

On March 7, 2016

In Analysis

A “trojanized document” hides malware inside itself, but rarely do we call a webpage doing the same a “trojanized webpage”. The word Trojan, derived from Homer’s epic poem, intended to describe a seemingly innocuous object containing damaging material, now describes almost all cyber threat delivery vectors. The term “Trojan” in cybersecurity has become diluted to the point of nonsense.

Trojan is just one example in a diluted language space now including other terms like virus, rootkit, targeted, etc. As the community grows in both terms of depth and breadth, it will carry with it historical baggage and loose terminology. Poor phraseology will infect those writing on the topic not familiar with nuances further contributing to the problem. Lastly, as cyber threats grow and change the language must evolve as well causing further issues. For example, increased modularization of capabilities challenge attempts to clearly categorize with existing language.

This is a problem for effective threat intelligence communication. Good threat intelligence accurately communicates the context of the threat relativizing it to a risk environment. A reliance on diluted language increases ambiguity therefore decreasing accuracy and effectiveness.

My message to those responsible for communicating cyber threats: consider language dilution, both your own actions contributing to dilution but also leveraging diluted language and its effect on your customers. Language dilution is a fact-of-life for any discipline, but how it’s addressed makes the difference.

Names…Names Everywhere! The Problem, and Non-Problem, of Name Pollution

By Sergio Caltagirone

On March 3, 2016

In Best Practices

Naming pollution is real. It’s a real problem. First anti-malware/AV malware detection names, now APT group names – and their campaigns – and their malware. Analysts are in love with names – and marketing is in love with their names.

You see, naming is powerful. It’s why we agonize over a child’s name. It’s why (in the Judeo-Christian tradition) God’s name was truncated and not to be uttered. At about 2 years old we start learning the names of things and are able to start uttering them back. This gives us power, because when the 2-year-old is able communicate a thing’s name – we give it to them! It’s powerful to a 2-year-old and that same power follows us throughout life – see “name dropping” – or the honor of naming a new geographic/astrological feature.

It’s followed us into the information security space – for both good and bad. You see, we need names. Names are important. It’s part of how we organize cognitive information and make sense of our world – through abstraction. It’s important to how we communicate. But, like any power, it can be misused and misappropriated. Every organization now loves to name “adversaries,” “actors,” “activity groups,” or whatever you call them. They can blog about it, tweet about it, produce nice glossy materials and presentations. It gives them power – because that’s what names do.

The problem isn’t names, it’s the power we attribute to them and their use in our analysis. When ThreatToe calls something BRUCESPRINGSTEEN and CyberCoffin identifies a similar activity and names it PEARLJAM, everyone else starts updating their “Rosetta Stone” and makes the association BRUCESPRINGSTEEN = PEARLJAM. Everyone else now starts attributing their intelligence to these two named groups. But, nobody actually knows what the heck these things are aside from a few properties (e.g. IPs/domains/capabilities/etc). That is not enough to understand.

I can’t tell you how many time’s I’ve heard: “Did you see the recent report from CyberVendor – can you believe they attributed that activity to PEARLJAM?! That is clearly STEVIEWONDER – those guys don’t know what they’re talking about.” The problem with that statement is that assumes: (1) you actually know what you’re talking about (you’ve correct correlated activity) and (2) you understand their definition of PEARLJAM. Within their own analytic definition the correlation could be absolutely correct. It’s that we’ve made unfounded assumptions and assigned too much power to the names.

But, WHY CAN’T WE JUST ALL AGREE ON NAMES!!!!! (as this is usually said in an elevated tone and usually while slightly-intoxicated) Because we can’t. That’s why. It’s not about the names. The names are just crutches – simple monikers for what is very complex activity and analytic associations which we still don’t know how to define properly. To understand this, you need to understand how we’re actually defining, correlating, and classifying these into groups – read the Diamond Model section 9 for this information.

The simple answer: it’s hard enough to correlate activity consistently within a 10 person team let alone across a variety of organizations. The complex answer: correlation and classification is a complex analytic problem which requires us to share the same grouping function and feature vector.

What we shouldn’t do is to start using each other’s names – because, again, it’s not about the names. If you begin to use the names of others you start to take on their “analytic baggage” as well since you are now intimately associating your analysis with theirs. This means you may also take on their errors and mis-associations. Further, it may mean that you agree with their attribution. Its highly unlikely that you’ll want intertwine your analysis with that of others whose you don’t really understand.

Instead, we need to rely on definitions. We need to openly share our correlation and classification logic and the feature vectors which we’re applying. But to those who are now saying, “Finally! An answer! Let’s just share this!” sorry, it’s not a silver bullet. Because, the feature vector is highly dependent on visibility. For instance, some organizations have excellent network visibility, some have outstanding host visibility, others may have great capability/malware visibility, etc. It means that generally, I need the same visibility as another organization to effectively use the shared functions to produce accurate output.

So, reader, here I am, telling you about this problem forcing poor analytic practices on daily basis causing us all these issues but without a real solution in sight. Yes, I think that sharing our definitions will get a LONG way towards improving correlation across organizations and giving those names real value – but it is by no means a silver bullet. I’m a proponent of this approach (over pure name/Rosetta stone work) but I know we’ll still spend hours on the phone or in a side conversation at a conference hashing all of this out anyways. But maybe, just maybe, it will reduce some analytic errors – and if that is the case it is better than what we have today.

Questions for Evaluating an External Threat Intelligence Source

By Sergio Caltagirone

On March 2, 2016

In Best Practices

I’ve spoken before on the cost of poor threat intelligence and its risk to an organization. I’ve also spoken about the 4 qualities of good intelligence: relevance, timeliness, accuracy, and completeness. To better evaluate threat intelligence sources – DRIVE FOR TRANSPARENCY! If you treat threat intelligence like a black box you’re going to lose.

Here are questions to use when evaluating an external source. These are just a starting point or additions to your own list based on your unique needs.

[Relevance] Why do I need threat intelligence?

Before you go out evaluating threat intelligence sources, you need to know what you’re looking for. This is best done using a threat model for your organization and asking where threat intelligence supports visibility and decision making within that model. Remember, your own threat intelligence is almost ALWAYS better than that produced by an external source. External intelligence should complement your own visibility and reduce gaps.

Kudos: Thanks to Stephen Ramage for his comment highlighting the exclusion of such a critical question.

[Relevance] What types of intelligence are available?

Strategic country-level reporting? Cyber threats mixed with political threats? Technical indicators? Campaign behaviors? Written context? These all determine how useful, actionable, and relevant the intelligence will be for your organization.

[Relevance] Give me your context!

Make sure you understand the context provided with any data. There is a difference between threat data and threat intelligence. Intelligence helps drive effective decision-making. Context makes data relevant.

[Relevance] Which threat types?

Is it limited to botnet C2 nodes? Commodity threats in general? Does it cover targeted threats? Does the threat intelligence provide insight into your threat model?

Related Questions: How many unique threats are distinguishable in the intelligence?

[Relevance] How many direct threats to my organization or those in my industry has your intelligence identified?

Has the source ever shown direct success in highlighting threats in your industry?

[Relevance] How is the intelligence made available to consumers?

If the intelligence is not provided in a usable form, it will not be successful.

[Relevance] What types of use-cases produce the best experience/feedback? In which use cases has your intelligence failed?

This is a soft-ball question but one which should provoke a good question-answer session. The answers will illuminate their decisions developing the intelligence and highlight where the intelligence may fit best (or not fit at all).

Related question: What threat model is this intelligence attempting to address?

[Completeness/Relevance] What is the source of the intelligence?

Is this intelligence derived from human sources crawling the dark-web? Global network apertures? VirusTotal diving? This question should frame their visibility into threats and inform the types of intelligence expected. This also highlights any natural biases in the collection. Look for sources of external intelligence which complement your own internal threat intelligence capabilities.

[Completeness] What phases of the kill-chain does the intelligence illuminate?

Understand how wide, against any single threat, the intelligence goes. Does it only show C2, or will it also illuminate pre-exploitation activities as well. The wider the intelligence, the greater the likelihood of it being useful.

[Completeness] What is the volume and velocity of the intelligence?

“How much” intelligence is actually produced? Numbers don’t matter that much – but if the number is ridiculously small or ridiculously large, it is an indicator of possible issues.

[Accuracy] How is the intelligence classified and curated?

Drive for transparency in their process which helps improve your evaluation on accuracy. Be wary of “silver bullet” buzz-word answers such as “machine learning” or “cloud.”

[Accuracy] How is the intelligence validated?

Do you want to track down false positives all day? No! Do you want to rely on poor analysis? No! Make sure this question gets enough attention.

Related questions: How often is it re-validated? How are false positives handled? How can customers report false positives? What is your false positive rate? How many times in the last month have you had to recall or revise an intelligence report?

[Accuracy] Does the intelligence expire?

Expiration of intelligence is key. Is there a process which continuously validates the intelligence?

[Timeliness] How quickly is the intelligence made available to customers after detection?

Related questions: What part of your process delays intelligence availability? What is the slowest time to availability from initial detection?

CART: The 4 Qualities of Good Threat Intelligence

By Sergio Caltagirone

On July 1, 2015

In Analysis

I write often of poor quality threat intelligence which pervades the security community. Poor quality threat intelligence not only has a heavy cost on its consumers, it also threatens the confidence threat intelligence consumers place in their providers. Confidence is the cornerstone of threat intelligence. Nobody will take intelligence from an untrustworthy source and act – at least they shouldn’t. It is important that the producer and consumer trust each other. That trust needs to be based on transparency and verification.

However, how does one appropriately assess threat intelligence? The first step must be to identify the qualities which define “good” threat intelligence. However, these are not binary qualities – there is a clear gradient based on use case. Timeliness is a good example of this gradient as some intelligence (likely more strategic) has a more fluid timeliness requirement while tactical threat intelligence has stricter requirements.

Further, one single threat intelligence source will not likely be able to satisfy all qualities simultaneously. For instance, it is unlikely any one provider will have complete visibility across Diamond elements or Kill Chain phases and consumers will have to rely on more than one to achieve satisfactory completeness.

The four qualities are (CART): Completeness, Accuracy, Relevance, and Timeliness.

Completeness

Threat intelligence must be sufficiently complete to provide effective detection and (hopefully) prevention. For instance, providing a domain indicator used in the exploitation of only one victim is not sufficient for other victims and therefore the intelligence is effectively incomplete and unhelpful.

Accuracy

Threat intelligence must save organizations more in success than it costs them in errors and mistakes.

Relevance

Threat intelligence must address a threat to the organization in a method that allows for effective action. Intelligence addressing threats not faced by the organization is of no value. Further, intelligence delivered in a type or method not usable by the organization is also unhelpful.

Timeliness

Threat intelligence must be received and operationalized fast enough to make an impact more valuable than the cost of the threat intelligence itself.

15 Things Wrong with Today’s Threat Intelligence Reporting

By Sergio Caltagirone

On February 19, 2014

In Best Practices

What I think when I read most threat intelligence reporting

As I have written before, intrusion analysis is equal parts knowing the technical elements of an intrusion and being an analyst. However, most in this domain spend an inordinate amount of time studying technical details compared to honing any analytic skills.

How long has it been since you’ve taken a highly technical course? (probably within the last year or two) How about an analysis course? (probably in the last 5 years, 10 years, never?)

I read several threat intelligence reports daily. It is painfully obvious how the lack of analytic skill is harming the discipline. Many folks come from technical degree backgrounds and analyze packets and binaries well enough but can’t seem to tell the difference between inductive, deductive, or abductive reasoning. Furthermore, their managers and mentors never recognize a problem, they just send them to more technical courses.

What is the risk? Threat intelligence provides insight and context to improve decision making. The risk of bad intelligence is high. Bad decisions can easily be made from poor intelligence – potentially doing more harm than good. Good analytic practices improve analysis thereby decreasing the risk of poor intelligence. You could have the best packet analysis skills in the world, but if you cannot communicate your conclusions effectively to those who need to act on your information those skills are effectively useless in threat intelligence.

We need to do better. I started this post about a month ago and wrote down a “lesson” whenever I saw an example of poor analysis. Needless to say, I saw some of these several times. (Contrary to the recommendation of others, I will not cite/quote specific examples – I believe that would only name and shame others)

Trend – the word actually means something

How many times per week must I read about a new “trend” from threat intelligence? One or two events does not constitute a trend. Even three or more events, depending on the universe of events, may not constitute a trend. Trends are serious. True trends in adversary activity and methodologies inferred by threat intelligence should drive data collection, analytic tradecraft, and defensive decisions. Before you start throwing out the word trend just because you’ve seen something a few times, consider the millions of other events you’re not seeing and consider if they’re just an anomaly rather than a trend.

Analysts and conclusions are like horses: sometimes you need to lead them to water

In many cases I can follow the logical progression of hypotheses and facts to the conclusion. In some cases I cannot. Either because an analyst failed to include the appropriate evidence/fact on which now an assumption must rest or because of convoluted logical reasoning. Ensure evidence supports your conclusions and the logical reasoning is clear. Don’t assume that what is clear in your mind will be clear in mine.

You can’t be completely confident all of the time – use words of estimative probability

Do you know how often I see the effective use of estimative probability in recent threat intelligence reporting? Almost never. This is a problem. Not everything presented is irrefutable fact; in fact, a good analysis will have a proper mix of data/fact, hypotheses and conclusions. The confidence values of these conclusions vary. When you don’t effectively apply estimative probability and variable measures of confidence it removes value from the analysis and increases the risk of poor decision making by consumers. First, if you don’t know what estimative probability is, LEARN about it. Then learn how and when to apply it properly. Importantly, also know what words/phrases to avoid (i.e. weasel words).

Never be afraid to include contrary evidence

Do you know how many times I saw evidence contrary to the conclusion presented in a threat intelligence report this month? Never. Practice analytic honesty. If there is exculpatory evidence, contrary evidence, or an alternative hypothesis – show it. As long as you’re following some of the other lessons here (e.g., separating fact and hypothesis, using words of estimative probability) it will strengthen your analysis and provide more value to the consumer.

Just because you’ve seen something for the first time doesn’t mean it’s the first time it happened

We all love finding something awesome and telling the world. It’s cool because we all want to know what you’ve found! But, please don’t assume it is the first time it has happened or even the first time it has been seen. Having confidence is critical, but hubris is deadly to analysis.

Don’t operate on an island

You are not alone! Don’t act like it. Share and consume, enrich and enhance. Go ahead and build on the analysis of others (citing appropriately). Whatever your observation point or data sources, they’re not omnipresent. I rarely see analysis reference other (obviously) related pieces. How is that? The power of defenders lies in our community and our ability to work together against an adversary.

Be bold, but don’t be stupid

I like my analysis like I like my coffee: bold. But, there is a line between taking facts to their logical conclusion and taking facts to crazy-land. The difference is logic. Ensure your conclusions and hypotheses follow logically from the facts through induction, deduction, or abduction. If your conclusions cannot be logically traced or tested, then they’re likely living in crazy-land.

Don’t mix hypotheses, conclusions, and facts

Hypotheses, conclusions, and facts are not the same. Your intelligence reports should not treat them as such. Ensure that your readers can effectively separate the three through your use of language, formatting, etc. When the three are confused it can lead to erroneous assumptions by consumers and lead to decisions made on weak conclusions rather than facts.

Save hyperbole for the glossy promotion material

Hyperbole has its place. It doesn’t have a place in threat intelligence. Save that for the glossies. Be precise, honest, and accurate. Don’t embellish or exaggerate. Trust me when I say we have enough people running around like chickens with their heads cut off in this field.

Logical fallacies are just that, get to know them

Enough said. I’m sorry I have to say this, but please understand the differences and applicability of deductive, inductive, and abductive reasoning BEFORE writing your first threat intelligence report. Or, at the very least, have an editor who knows the difference.

Don’t create new words when existing words suffice

I’m not going to name-call here. You know who you are. There are times when words/phrases have multiple meanings. I understand that. But, aside from that….stop it.

Tell a story!

Your analysis is a story. You’re effectively documenting history – studying the past – in the hopes of making conclusions and judgments which will help the present and future. While you are documenting the activity of computers you are ultimately describing the actions caused by adversaries. Just like any story your report should have a beginning, middle, and an end.

Answer my questions

Write as if you are in a conversation. Think about what somebody else may ask of what you’re saying, and address those questions in the text. Any questions left unanswered have the ability to form into assumptions on the part of the consumer/customer. If you don’t have an answer, feel free to write: no further information.

Be concise, be accurate

Practice analytic honesty and respect the time of your reader. The report you’re considering may actually need to be three different reports – one describing all of the malware reverse engineering, one describing all of the network activity, and another describing the threat itself which references the other report. Putting everything in one report does not make it more consumable, it makes it less consumable and allows analysts to muddle up various lines of analysis.

Describe diagrams, charts, and tables both in the narrative text but also in a caption

This is just a pet-peeve of mine, but one which I find increases the readability of threat intelligence reports. Make sure you describe your diagrams, charts, and tables in both the narrative text (as part of the story) and also in a caption. I find this necessary because as I move backwards and forwards through a report reading and re-reading, forming and re-forming logical chains, I don’t want to hunt for the description in the text every time. I also don’t want to jump to the caption in the middle of text if not necessary which breaks my concentration.