Author: Sergio Caltagirone Page 2 of 5

Sergio is the head of threat intelligence analysis at Microsoft and operates a global threat intelligence mission to discover, understand, track, and disrupt malicious activity against Microsoft and its customers. He is passionate about empowering defenders with timely and accurate information and moving information security from a reactive to a proactive posture.

Before Microsoft he worked for the United States Government for 8+ years and built and led several threat intelligence missions.

Sergio grew up in Western Washington State and attended the University of Portland where he received his Bachelor of Science in Computer Science and also a degree in Theology with a strong liberal arts background. He went on to the University of Idaho in 2005 where he received his Master of Science in Computer Science. At Idaho Sergio expanded his education by becoming the first computer science student allowed to take Law classes where he focused on legal topics connected to computer security.

Sergio has been very active in research and innovation receiving his first patent working with cognitive psychologists on graphical passwords (US20100169958) and had over 12 publications and a thesis on the topic of Active Response. He has gone on to work in several organizations doing computer and network security, forensics, and intrusion analysis.

Sergio is also the Chief Scientist of The Center for Cyber Intelligence Analysis and Threat Research working towards the goal of moving cyber from an art to a science.

Sergio Caltagirone+

Indicators and Security Analytics: Their Place in Detection and Response

By Sergio Caltagirone

On September 21, 2016

In Detection

Indicators for research and response; analytics for detection

Indicators of Compromise (IOCs), the lingua franca of threat intelligence. Almost every intel sharing conversation begins and ends with indicators; commercial intelligence platforms revolve around them; most intelligence consumers end their interest there. Does a better way exist? Security analytics!

The Problem with Indicators in Detection

For all the focus given to indicators we know that they have the shortest lifespan of all intelligence exhaust (see the Pyramid of Pain by David J. Bianco). In many cases, we see single use or victim specific indicators making sharing of these useless. In general, adversaries tend towards shortening the indicator lifespan – or removing them; for instance Locky recently transitioned to hardcoded RSA keys to remove the vulnerability of connecting to a command and control (C2) server.

Broad based indicator sharing is fraught with problems. First, it assumes that the same indicators will be leveraged against multiple victims. This is certainly the case for some threats. But not all. Second, quality will likely be a problem. For instance, DHS Automated Indicator Sharing (AIS) states:

Indicators are not validated by DHS as the emphasis is on velocity and volume: our partners tell us they will vet the indicators they receive through AIS, so the Department’s goal is to share as many indicators as possible as quickly as possible. However, when the government has useful information about an indicator, we will assign a reputation score. – DHS Automated Information Sharing

Further, AIS contributors can choose to remain anonymous. Think about the problems of blindly consuming thousands of non-validated anonymously sourced indicators. How exactly do you effectively validate an anonymously contributed indicator? Previously, I wrote on the cost of poor intelligence. Just one instance of 8.8.8.8 by an anonymous contributor could cause massive issues.

Indicators of Compromise are only threat exhaust – the necessary by-product of malicious activity. Short-lived and increasingly single use, indicators pose a poor basis for detection – and it’s getting worse. I’m not advocating for throwing indicators out entirely – they serve their purpose, but should not form the entire basis of threat intelligence detection.

Analytics For Detection

As the Pyramid of Pain suggests, we must move towards behavioral based detection focusing on whole classes of threats. I’d much rather rely on an analytic detecting overwriting Windows registry keys for a “sticky keys” attack than hoping someone shares an IP address of a random hop point used before to remote desktop (RDP) into a host. In the analytic case I catch every adversary using sticky keys, in the case of the indicator I catch only one adversary – with the hope they use the same infrastructure again.

Where do you find analytics?

The best place is your red team – ask them to describe their techniques and procedures. Read their reports! (I know – a stretch for some)
Read threat intelligence reports on adversary behaviors.
Ask your threat intelligence provider! (Who you already abuse with information requests anyways – right?)
Check out MITRE’s Cyber Analtyics Repository.

The Place for Indicators – Research and Response

Indicator sharing works within a small group of organizations that share a “victim space” (as the Diamond Model refers to victims with shared threats). This greatly increases the value of shared indicators because the likelihood of attackers reusing indicators increases. However, indicator sharing outside the “shared victim space” reduces their value and increases their cost. Research and response receive the greatest value from shared indicators as it allows a method of communicating observables discovered in attacks allowing analysts to pivot deeper into malicious activity seen by others.

Your Own Intelligence is the Best

In the end, to achieve greater detection capability organizations must invest in security analytics and reduce their reliance (and insistence) on indicators from externals. The best indicators in the world are those from your organization’s own telemetry – your own threat intelligence is the most relevant. Otherwise, look suspiciously at indicators from others and instead ask to share analytics!

Note: Security analytics are a dirty word – overused and often misused. To be clear, I define analytics in this post as indicator-independent behavioral detection derived from the knowledge of bad stuff (i.e. Threat Intelligence)

The Laws of Cyber Threat: Diamond Model Axioms

By Sergio Caltagirone

On August 29, 2016

In Best Practices

Many confuse the purpose of the Diamond Model. Most believe the Diamond Model exists for analysts, but that is an ancillary benefit. Instead, think of the Diamond Model like a model airplane used to study the principles of aerodynamics. It is not an exact copy but rather a good approximation of the full-scale airplane being studied. The model exposes elements to test and study in a controlled environment improving the performance of the plane in an operational environment. The Diamond Model does the same, except for cyber threat analysis.

When describing the Diamond Model to others, I usually start with, “we didn’t create the Diamond Model, we simply expressed some fundamental elements which always existed.” Surprisingly, I learned while writing the Diamond Model how exposing this fundamental nature improved cyber threat intelligence.

The Diamond Model captures this fundamental nature about threats in seven axioms and one corollary. This post will highlight those axioms.

Axiom 1

For every intrusion event there exists an adversary taking a step towards an intended goal by using a capability over infrastructure against a victim to produce a result.

What it means: every malicious event contains four necessary elements: an adversary, a victim, a capability, and infrastructure. Using this fundamental nature we can create analytic and detective strategies for finding, following, and mitigating malicious activity.

Axiom 2

There exists a set of adversaries (insiders, outsiders, individuals, groups, and organizations) which seek to compromise computer systems or networks to further their intent and satisfy their needs.

What it means: there are bad actors working to compromise computers and networks – and they do it for a reason. Understanding the intent of an adversary helps developing analytic and detective strategies which can create more effective mitigation. For example, if we know that an adversary is driven by financial data, maybe we should focus our efforts on assets that control and hold financial data instead of other places.

Axiom 3

Every system, and by extension every victim asset, has vulnerabilities and exposures.

What it means: vulnerabilities and exposures exist in every computer and every network. We must assume assets can (and will) be breached – other express this notion as “assume breach.”

Axiom 4

Every malicious activity contains two or more phases which must be successfully executed in succession to achieve the desired result.

What it means: malicious activity takes place in multiple steps (at least two), and each step must be successful for the next to be successful. One popular implementation of this axiom is the Kill Chain. But, the Kill Chain was not the first to express this notion – another popular phase-based expression is from the classic, Hacking Exposed.

Axiom 5

Every intrusion event requires one or more external resources to be satisfied prior to success.

What it means: adversaries don’t exist in a vacuum, they require facilities, network connectivity, access to victim, software, hardware, etc. These resources can also be their vulnerability when exploring mitigation options.

Axiom 6

A relationship always exists between the Adversary and their Victim(s) even if distant, fleeting, or indirect.

What it means: exploitation and compromise takes time and effort – adversaries don’t do it for no reason. An adversary targeted and compromised a victim for a reason – maybe they were vulnerable to a botnet port scan because the adversary looks to compromise resources to enlarge the botnet, maybe the victim owns very specific intellectual property of interest to the adversary’s business requirements. There is always a reason and a purpose.

Axiom 7

There exists a sub-set of the set of adversaries which have the motivation, resources, and capabilities to sustain malicious effects for a significant length of time against one or more victims while resisting mitigation efforts. Adversary-Victim relationships in this sub-set are called persistent adversary relationships.

What it means: what we call “persistence” (such as in Advanced Persistent Threat) is really an expression of the victim-adversary relationship. Some adversaries need long-term access and sustained operations against a set of victims to achieve their intent. Importantly, just because an adversary is persistent against one victim doesn’t mean they will be against all victims! There is no universal “persistent” adversary. It depends entirely on each relationship at that time.

Corollary

There exists varying degrees of adversary persistence predicated on the fundamentals of the Adversary-Victim relationship.

What it means: not all persistence is created equal. Some adversary-victim relationships are more persistent than others. Sometime a victim will mitigate a years long intrusion only to be compromised again by the adversary that same week; at other times the adversary will never return.

Diamond Model or Kill Chain?

By Sergio Caltagirone

On August 27, 2016

In Best Practices

Rob MacGregor at PwC in “Diamonds or chains” asked , do you choose the Diamond Model or Kill Chain? I get asked this question often. The question assumes that the models are mutually exclusive when, in fact, they are not only complementary but interconnected. Both models express fundamental elements of network exploitation in methods usable by network defenders. You can’t expect complete intelligence or network defense without using both the Diamond Model and the Kill Chain.

Most understand that the Diamond Model expresses the first axiom encompassing the basic components of any malicious event: “For every intrusion event there exists an adversary taking a step towards an intended goal by using a capability over infrastructure against a victim to produce a result.” However, most readers stop there, at page 15 – only 25% of the model.

Adversaries don’t just conduct one activity and move on – no, they must conduct several in a phased approach each successfully completing before the next. As expressed on page 15 via Axiom 4: “Every malicious activity contains two or more phases which must be successfully executed in succession to achieve the desired result.” Axiom 4 effectively describes the Intrusion Kill Chain (section 3.2 of the Kill Chain). Therefore, Events interconnect via Activity Threads which describe campaigns.

One may notice a great similarity between the figure describing key campaign indicators (Kill Chain pg. 8) and the Activity Threads illustration (Diamond Model pg. 31). The two approaches interconnect at this point!

Diamond Model Activity Threads; The Diamond Model of Intrusion Analysis pg. 31

Campaign Key Indicators; Intelligence-Driven Computer Network Defense
Informed by Analysis of Adversary Campaigns and
Intrusion Kill Chains pg. 8

Dependent Events (composed of a victim, adversary, capability, victim) create Activity Threads across the Kill Chain. These threads compose (using key campaign indicator analysis) adversary campaigns. Ta Da! The first interconnection between the two models.

The Diamond Model and Kill Chain analysis are highly complementary. Kill Chain analysis allows an analyst “to target and engage an adversary to create desired effects.” (Kill Chain pg. 4) The Diamond allows analysts to develop tradecraft and understanding to build and organize the knowledge necessary to execute the Kill Chain analysis.

Once an analyst develops an activity thread, courses of action for each event along the thread can be identified using the Kill Chain’s course of action matrix. As illustrated in the figures, courses of action for each of the Kill Chain stages are identified for activity threads. The power of the Diamond Model is that courses of action can be designed to span multiple victims and across the activity of an adversary making the actions even more powerful as they further reduce the capacity of the adversary.
Activity groups clustered by same likely adversary (i.e., clustering by attribution) with analysis of the largest common feature set amongst the events in a group can provide the Kill Chain’s required key campaign indicators necessary to focus and prioritize courses of actions.

In the end, don’t ask: do we use the Diamond Model or the Kill Chain. Instead ask: are you using them both effectively?

Threat Intelligence Definition: What is Old is New Again

By Sergio Caltagirone

On August 24, 2016

In Uncategorized

Michael Cloppert, whom I hold in great esteem and friendship, argues for a new and unconventional definition of “cyber threat intelligence.” His post is excellent and well-done. His argument is simple: that the existing definitions of intelligence and cyber threat intelligence are lacking based on his professional experience of the domain and fail to capture its unique elements. He offers several definitions:

Cyber threat operations as actions taken in cyberspace to compromise and defend protected information and capabilities available in that domain

Cyber Threat Intelligence Analysis as the analysis of those actions and the actors, tools, and techniques behind them so as to support Operations

I define the Cyber Threat Intelligence domain as the union of Cyber Threat Intelligence Operations and Analysis.

Michael Cloppert, Defining Cyber Threat Intelligence (2016)

I agree with his assessment that existing cyber threat intelligence definitions lack accuracy. But, Mike’s definitions are too constrained by operations and lack inclusion of the key element of intelligence in any discipline: that intelligence serves to inform decision-making (whether that decision-making is of the technical/tactical nature such as in firewalls, or strategic at the executive level). Intelligence doesn’t serve operations, intelligence serves decision-making which in turn drives operations to achieve policy outcomes.

Mike references some key CIA thought-pieces on their definitions of intelligence, namely by Martin T. Bimfort in A Definition of Intelligence. Mike is correct that taken at face value, Bimfort’s definition is too constrained with concern about national security to be of much value to cyber threat intelligence.

Intelligence is the collecting and processing of that information about foreign countries and their agents which is needed by a government for its foreign policy and for national security, the conduct of non-attributable activities abroad to facilitate the implementation of foreign policy, and the protection of both process and product, as well as persons and organizations concerned with these, against unauthorized disclosure.

Martin T. Bimfort’s definition of intelligence in A Definition of Intelligence

However, instead of taking Bimfort’s definition at face value, let’s instead look at its essence by removing the domain-specific (state-only) language. By doing so, I arrive at the following revised definition:

Intelligence is the collecting and processing of that information about threats and their agents which is needed by an organization for its policy and for security, the conduct of non-attributable activities outside the organization’s boundaries to facilitate the implementation of policy, and the protection of both process and product, as well as persons and organizations concerned with these, against unauthorized disclosure.

This definition fits well what we do in cyber threat intelligence: we uncover the hidden threats to an organization (be it a company or country) to protect them against threats both attributable and non-attributable to enable their policy (which for a private company is to return value to shareholders), protect their operations, and prevent disclosure of secrets.

I propose that cyber threat intelligence is nothing more than the application of intelligence principles and tradecraft to information security. Its outcome is nothing different from traditional intelligence: to inform and empower decision-making at all levels with knowledge of threats. We don’t require a radical new definition of cyber threat intelligence, because the traditional definitions of intelligence are applicable by simply broadening them outside of their state-only constraint.

EDIT: Robert M. Lee blogged in response – “Intelligence Defined and its Impact on Cyber Threat Intelligence“. He came to the conclusion that the definition is, “the process and product resulting from the interpretation of raw data into information that meets a requirement as it relates to the adversaries that have the intent, opportunity and capability to do harm.”

The Darker Side of Threat Intelligence: Cyber Stockholm Syndrome

By Sergio Caltagirone

On July 12, 2016

In Analysis

Stockholm Syndrome is a psychological phenomenon described in 1973 in which hostages express empathy and sympathy and have positive feelings toward their captors, sometimes to the point of defending and identifying with the captors. - Wikipedia

Maturing as a threat intelligence analyst involves “living with your threat.” In my interview process I ask potential analysts about threats they’ve tracked in their career. Tracking a threat for months or years creates a unique learning environment and I look for that in analysts. Unsurprisingly, in that environment an analyst becomes intimate with the adversary’s routines, their interests, and even begins to distinguish characteristics of individuals from within a larger group. An analyst gets truly connected when they can successfully predict a threat’s activity.

However, while this sounds like an analytic panacea and also something threat intelligence production cells strive to build, it comes at a cost. The risk is that analysts go beyond being closely connected and become “married” to a threat. In living with that threat every day, spending all of your professional time studying them, spending hundreds of hours discussing them with others, it is impossible not to closely connect with the adversary on the other side of your screen. Analysts become personally attached to the “bad guys” – a “Cyber Stockholm Syndrome.” I personally know analysts who have fallen into depression when their threat goes away.

Not only is this unhealthy for the analyst, this relationship also affects their communication and infects their analytic capabilities reducing objectivity.

Symptoms of “Cyber Stockholm Syndrome”

An analyst gets particularly protective and defensive regarding perceived encroachment on their territory
An analyst unnecessarily hides intelligence and data to prevent others from knowing details helping to maintain their superiority
Overwhelming and obvious confirmation bias – an analyst “seeing their threat in everything”
An unwillingness to work on other threats even given clear direction and obvious priorities
An analyst continues to work on a threat even after the threat is “gone” against overwhelming evidence and analytic consensus

What may cause this?

One hypothesis: an analyst may associate their self-worth with an adversary. As the analyst grows in mastery of knowledge of an adversary, they produce spectacular intelligence and amazing insight providing great value to others; this results in praise from leaders and admiration from peers creating a feedback loop. The cycle strengthens the bond the analyst builds with a threat as the threat continues to provide value to the analyst.

What should happen?

When this happens managers may respond by immediately separating the analyst from the threat. I don’t believe that is the right answer. Separation causes resentment and potential psychological problems such as depression. Instead, managers of analysts should look to slowly incorporate other analysts into the equation and ultimately strive to return the analyst to a proper relationship so they don’t lose that valuable expertise.

Most importantly, analysts must recognize this problem in themselves. For their own professional and personal well-being.

Additional Discussion

Chris Sanders (@chrissanders88) made an excellent point that Stockholm Syndrome requires empathy with an aggressor which is lacking in my description. I agree that the syndrome’s description includes that requirement but its exclusion from the DSM means there is no consistent definition. Further, active academic discussion on the topic includes whether Stockholm Syndrome actually exists or is really one facet of a larger aggressor-bonding trait. While empathy is not the right aspect of the bond I describe here there is an attachment bond created either through the return on investment (ROI) the analyst receives through the adversary or otherwise. This is evidenced by both the confirmation bias present and the sense of depression described by analysts. I agree that the application of the Stockholm Syndrome may be imprecise.

Keeping up with the Stream: How I Maintain External Situational Awareness

By Sergio Caltagirone

On July 11, 2016

In Best Practices, Strategy

In any field related to intelligence and security it is critical to stay abreast with external news and developments. But, your time is a zero-sum game and all security and intelligence analysts must balance their time “reading the news” (consuming news from others) with “creating the news” (generating new intelligence and insight for others) – this is how I view my work time strategically. Building tools and techniques to more efficiently “read the news” allows you to spend more time “creating the news.” So it is no surprise that I get asked regularly what I do to stay connected with the world and the community. Here is my answer, for my particular situation and need. Mileage will vary.

For me, the key is to take advantage of curated news/information streams instead of curating it myself. However, just like relying on any one news source, relying on one or a few curators for your news will quickly introduce you to the bias of the curators themselves. Therefore, I don’t rely entirely on this method and also self-curate to a small extent to lower that risk.

I organize my professional reading into three categories: world, profession (computer science/security/analysis/data science), and discipline (threat intelligence). Usually, I begin by reading the world news, followed by threat intelligence, and lastly information I need about my profession. I feel that this appropriately prioritizes my time and gives me the best perspective to solve problems throughout the day.

Here is my particular strategy:

I begin with the top stories on Google News and then to the Economist. I then browse the front page of Reddit. Together this gives me a healthy sense of major events in the larger world. This is critical because my discipline is heavily influenced by larger world events. However, within this set I also focus my time reading articles which have direct impact on areas of world my daily work touches.
I read curated security and intelligence emails: Team Cymru Dragon News Bytes; SANS NewsBites (weekly); and two others which come from paid services via my employer.
Twitter. I use key hashtags and user lists to pare down the stream to a consumable chunk. This is very much an art form and I’ve yet to feel a mastery.
RSS Feeds. I use Feedly to curate my RSS feeds. However, over time I’ve found that my other strategies tend to surface most of the gems from the feeds.
If I have time, I’ll then use a financial news site to browse the news about my company as well as major players in cyber security to maintain awareness about the larger business pressures and events which may impact my work.
Return to Twitter. About 2-3 times/day I’ll return to Twitter to scroll through tweets by key hashtags and user lists to make sure I find anything critical right away.

The Long & Important Ones

About once-per-day I find a white paper or article on which I want to focus and absorb. For those, I print them out (yes, on paper) and read them later with a pen in my hand so that I practice Active Reading; making marks, underlining, and making comments which help me absorb the material and create an internal conversation. I find this a highly enjoyable activity which stimulates creativity and engagement helping to foster new ideas.

How do you maintain your external situational awareness? Please comment below or tweet @cnoanalysis

13 Principles of Threat Intelligence Communication

By Sergio Caltagirone

On March 9, 2016

In Best Practices

I have written at length about bad threat intelligence. However, I think it is time that I spend the effort communicating my key principles to making great threat intelligence. One aspect of great threat intelligence is great communication. As I have said before, you may be the greatest analyst in the world, but if you can’t effectively communicate your knowledge then it is of little use.

I’ve found these principles apply to all modes of my communication when discussing threat intelligence with others. They’ve guided me well and I hope they do the same for you.

Answer the Three Questions

All threat intelligence communication should work towards answering three critical questions, if you clearly articulate the answer to these questions your communication will be generally successful.

What is it? (give me the information)
Why should I care? (tell me about the threat and its relevance to me)
What am I going to do? (enable my decision and action)

Maintain Your Focus

Focus is key to your communication – understand your audience and your objective and maintain that throughout. Here are some elements which help me:

Remember the four qualities of good intelligence (CART): completeness, accuracy, relevance, and timeliness. Fulfill them as best you can.
Remember the purpose of threat intelligence to inform and enable effective decision-making, whether that be tactical/technical, operational, or strategic. You don’t need to provide EVERYTHING, only that which will support and enhance the intelligence.
Length matters: your communication should be as long as it needs to be but never longer than it should be. Here’s a secret: it’s okay to not communicate everything in one vehicle – sometimes separating the material makes the threat intelligence more effective.
Don’t derail your audience. After reading your 30 page report, make sure I know the value of the information and that you’ve addressed the key questions. For example: don’t all of a sudden drop an unrelated element in your conclusion just because you want to make a point.

Analytic Integrity is All You Have

Intelligence is about trust. When people can’t independently verify your findings and conclusions (which most won’t/can’t) then they must trust you. You must create, support, and encourage that trust by practicing analytic integrity in your communications. If you break that trust you lose your integrity and nobody will listen to you. Here are some of my rules to creating and encouraging trust with your audience:

Don’t lie – if you don’t know, just say that
Don’t embellish – don’t use hyperbole or language which might cause an over-reaction
Don’t plagiarize – never intentionally (and avoid accidents) copy the work of another
Practice humility – hubris infers overcompensation for weakness, be bold but not stupid

Be a Storyteller

Threat intelligence is a story – tell it as one. Threat intelligence should have a beginning, middle, and an end. Engage your audience.

The Summary IS the Communication

I know it sounds weird, but your summary is the most important part of your communication. This is what people will remember and what they’ll rely on most afterwards. For many, this is the only part to which they’ll pay attention. The summary (or key points, etc.) should be par excellence. I instruct analysts to spend at least 20% of their time on their summary and conclusion – it is that important.

As the old adage goes: “tell them what you’re going to tell them, tell them, tell them what you told them.” This is CRITICAL advice and not often heeded by technical analysts.

However, I want to caution you. Others suggest that following this old adage only bores an audience. I agree that it is a pitfall for most, only because many follow the guidance without understanding it. Avoid the summary and conclusion containing the same bullet points or phrasing – that is boring. However, your summary/introduction/key points/etc. and your conclusion should carry your key message and information, but in different ways.

Language Matters

The language you use greatly determines the effectiveness of your communication.

Use Active Voice – this isn’t some joke or regurgitation of high-school English. It matters. Active voice has been proven to decrease ambiguity and increase comprehension. It improves your intelligence.
- Science: “Certain syntactic constructions are known to cause the processor to work harder than others. Sentences with passive verbs are more difficult to comprehend than those with active verbs (Gough 1966; Slobin 1966; Olson and Filby 1972; Ferreira 2003) since they not only reverse the standard subject-verb-object order of the participants but are often used without a byphrase , which omits one participant altogether and can obscure the grammatical relations.”
Use Estimative Probability – judgements, hypotheses, and conclusions are never 100% certain; use words of estimative probability to clarify your certainty to your audience.
Clarity wins over all – don’t use complex language when simple will do.
Minimize subjective qualifications – avoid words/phrases like (sophisticated adversary) or (complex encryption) unless you can measure them either objectively or in comparison with others. These phrases only add ambiguity.
Words mean things – don’t dilute your language or create a phrase when one already exists.
Analysis is not a religion – don’t use the word believe; hold measured judgements expressed in language differentiating fact and hypothesis.

Value Your Audience

Value their intelligence and their time. They are not fish caught by click-bait or hyperbole but respected for their interest in your work. Your audience is spending time with you because they think you have something valuable to communicate and they have come to learn something new – GIVE IT TO THEM! Or, they will leave you.

Images are Powerful

Use images strategically to tell your story, reinforce critical concepts, and increase accessibility and understanding. Images should not become overwhelming, distracting, or superfluous.

Write for Your Future Self

Communicating intelligence and analysis is HARD. It’s hard because you’re trying to take a very complex cognitive process and share that with others. I’m not the only one who has read something they wrote a year ago only to scratch my head and wonder what I was smoking. I’ve found that to make this easy I simply imagine that I’m communicating to my future self – say 1, 2, or 3 years from now. This helps ensure that I include important details which are obvious now but will be lost later. Further, it ensures that I make my logic chains clear and easily followed by others.

Don’t be an Island

Be part of the community. Respect the community. Expand on the work of others and fill in knowledge gaps. Confirm others’ findings and add support to their conclusions or hypotheses. Add exculpatory evidence and provide alternative hypotheses. And here’s a secret: it’s okay to point to the analysis of others in your communication – you don’t always have to self-reference. This actually adds value for your audience and makes you more valuable to them because they trust you’re going to tell them the whole story – not just your story.

Respect Your Adversaries

Don’t belittle adversaries in your threat intelligence. Don’t give them undue credit, but also don’t take away from their effectiveness. This will only lead to hubris – and hubris is deadly. We all know of an analyst who called a threat “unsophisticated” or “simple” only to later report a massive compromise.

Be Bold, Be Honest, Be Right, But Always Be Willing to be Wrong

I’ve said it before, I like my analysis like I enjoy my coffee, bold. I want analysts to be analysts – not reporters. I want to hear ideas, conjecture, assessment, opinions. I want those clearly separated from the facts.

Separate Fact From Everything Else

This is a pretty simple rule. But harder to follow in practice while working through a complex analysis. Strive to use language, format, font, etc. to separate fact from hypothesis. Because threat intelligence enables decision-making, decision makers (whether a SOC analyst, a CIO, or whoever) should make their own judgement based on your analysis. If your facts and hypotheses are indistinguishable it is highly likely they’ll make poor decisions based on misinterpreted analysis.

Cyber Threat Language Dilution

By Sergio Caltagirone

On March 7, 2016

In Analysis

A “trojanized document” hides malware inside itself, but rarely do we call a webpage doing the same a “trojanized webpage”. The word Trojan, derived from Homer’s epic poem, intended to describe a seemingly innocuous object containing damaging material, now describes almost all cyber threat delivery vectors. The term “Trojan” in cybersecurity has become diluted to the point of nonsense.

Trojan is just one example in a diluted language space now including other terms like virus, rootkit, targeted, etc. As the community grows in both terms of depth and breadth, it will carry with it historical baggage and loose terminology. Poor phraseology will infect those writing on the topic not familiar with nuances further contributing to the problem. Lastly, as cyber threats grow and change the language must evolve as well causing further issues. For example, increased modularization of capabilities challenge attempts to clearly categorize with existing language.

This is a problem for effective threat intelligence communication. Good threat intelligence accurately communicates the context of the threat relativizing it to a risk environment. A reliance on diluted language increases ambiguity therefore decreasing accuracy and effectiveness.

My message to those responsible for communicating cyber threats: consider language dilution, both your own actions contributing to dilution but also leveraging diluted language and its effect on your customers. Language dilution is a fact-of-life for any discipline, but how it’s addressed makes the difference.

Names…Names Everywhere! The Problem, and Non-Problem, of Name Pollution

By Sergio Caltagirone

On March 3, 2016

In Best Practices

Naming pollution is real. It’s a real problem. First anti-malware/AV malware detection names, now APT group names – and their campaigns – and their malware. Analysts are in love with names – and marketing is in love with their names.

You see, naming is powerful. It’s why we agonize over a child’s name. It’s why (in the Judeo-Christian tradition) God’s name was truncated and not to be uttered. At about 2 years old we start learning the names of things and are able to start uttering them back. This gives us power, because when the 2-year-old is able communicate a thing’s name – we give it to them! It’s powerful to a 2-year-old and that same power follows us throughout life – see “name dropping” – or the honor of naming a new geographic/astrological feature.

It’s followed us into the information security space – for both good and bad. You see, we need names. Names are important. It’s part of how we organize cognitive information and make sense of our world – through abstraction. It’s important to how we communicate. But, like any power, it can be misused and misappropriated. Every organization now loves to name “adversaries,” “actors,” “activity groups,” or whatever you call them. They can blog about it, tweet about it, produce nice glossy materials and presentations. It gives them power – because that’s what names do.

The problem isn’t names, it’s the power we attribute to them and their use in our analysis. When ThreatToe calls something BRUCESPRINGSTEEN and CyberCoffin identifies a similar activity and names it PEARLJAM, everyone else starts updating their “Rosetta Stone” and makes the association BRUCESPRINGSTEEN = PEARLJAM. Everyone else now starts attributing their intelligence to these two named groups. But, nobody actually knows what the heck these things are aside from a few properties (e.g. IPs/domains/capabilities/etc). That is not enough to understand.

I can’t tell you how many time’s I’ve heard: “Did you see the recent report from CyberVendor – can you believe they attributed that activity to PEARLJAM?! That is clearly STEVIEWONDER – those guys don’t know what they’re talking about.” The problem with that statement is that assumes: (1) you actually know what you’re talking about (you’ve correct correlated activity) and (2) you understand their definition of PEARLJAM. Within their own analytic definition the correlation could be absolutely correct. It’s that we’ve made unfounded assumptions and assigned too much power to the names.

But, WHY CAN’T WE JUST ALL AGREE ON NAMES!!!!! (as this is usually said in an elevated tone and usually while slightly-intoxicated) Because we can’t. That’s why. It’s not about the names. The names are just crutches – simple monikers for what is very complex activity and analytic associations which we still don’t know how to define properly. To understand this, you need to understand how we’re actually defining, correlating, and classifying these into groups – read the Diamond Model section 9 for this information.

The simple answer: it’s hard enough to correlate activity consistently within a 10 person team let alone across a variety of organizations. The complex answer: correlation and classification is a complex analytic problem which requires us to share the same grouping function and feature vector.

What we shouldn’t do is to start using each other’s names – because, again, it’s not about the names. If you begin to use the names of others you start to take on their “analytic baggage” as well since you are now intimately associating your analysis with theirs. This means you may also take on their errors and mis-associations. Further, it may mean that you agree with their attribution. Its highly unlikely that you’ll want intertwine your analysis with that of others whose you don’t really understand.

Instead, we need to rely on definitions. We need to openly share our correlation and classification logic and the feature vectors which we’re applying. But to those who are now saying, “Finally! An answer! Let’s just share this!” sorry, it’s not a silver bullet. Because, the feature vector is highly dependent on visibility. For instance, some organizations have excellent network visibility, some have outstanding host visibility, others may have great capability/malware visibility, etc. It means that generally, I need the same visibility as another organization to effectively use the shared functions to produce accurate output.

So, reader, here I am, telling you about this problem forcing poor analytic practices on daily basis causing us all these issues but without a real solution in sight. Yes, I think that sharing our definitions will get a LONG way towards improving correlation across organizations and giving those names real value – but it is by no means a silver bullet. I’m a proponent of this approach (over pure name/Rosetta stone work) but I know we’ll still spend hours on the phone or in a side conversation at a conference hashing all of this out anyways. But maybe, just maybe, it will reduce some analytic errors – and if that is the case it is better than what we have today.

Questions for Evaluating an External Threat Intelligence Source

By Sergio Caltagirone

On March 2, 2016

In Best Practices

I’ve spoken before on the cost of poor threat intelligence and its risk to an organization. I’ve also spoken about the 4 qualities of good intelligence: relevance, timeliness, accuracy, and completeness. To better evaluate threat intelligence sources – DRIVE FOR TRANSPARENCY! If you treat threat intelligence like a black box you’re going to lose.

Here are questions to use when evaluating an external source. These are just a starting point or additions to your own list based on your unique needs.

[Relevance] Why do I need threat intelligence?

Before you go out evaluating threat intelligence sources, you need to know what you’re looking for. This is best done using a threat model for your organization and asking where threat intelligence supports visibility and decision making within that model. Remember, your own threat intelligence is almost ALWAYS better than that produced by an external source. External intelligence should complement your own visibility and reduce gaps.

Kudos: Thanks to Stephen Ramage for his comment highlighting the exclusion of such a critical question.

[Relevance] What types of intelligence are available?

Strategic country-level reporting? Cyber threats mixed with political threats? Technical indicators? Campaign behaviors? Written context? These all determine how useful, actionable, and relevant the intelligence will be for your organization.

[Relevance] Give me your context!

Make sure you understand the context provided with any data. There is a difference between threat data and threat intelligence. Intelligence helps drive effective decision-making. Context makes data relevant.

[Relevance] Which threat types?

Is it limited to botnet C2 nodes? Commodity threats in general? Does it cover targeted threats? Does the threat intelligence provide insight into your threat model?

Related Questions: How many unique threats are distinguishable in the intelligence?

[Relevance] How many direct threats to my organization or those in my industry has your intelligence identified?

Has the source ever shown direct success in highlighting threats in your industry?

[Relevance] How is the intelligence made available to consumers?

If the intelligence is not provided in a usable form, it will not be successful.

[Relevance] What types of use-cases produce the best experience/feedback? In which use cases has your intelligence failed?

This is a soft-ball question but one which should provoke a good question-answer session. The answers will illuminate their decisions developing the intelligence and highlight where the intelligence may fit best (or not fit at all).

Related question: What threat model is this intelligence attempting to address?

[Completeness/Relevance] What is the source of the intelligence?

Is this intelligence derived from human sources crawling the dark-web? Global network apertures? VirusTotal diving? This question should frame their visibility into threats and inform the types of intelligence expected. This also highlights any natural biases in the collection. Look for sources of external intelligence which complement your own internal threat intelligence capabilities.

[Completeness] What phases of the kill-chain does the intelligence illuminate?

Understand how wide, against any single threat, the intelligence goes. Does it only show C2, or will it also illuminate pre-exploitation activities as well. The wider the intelligence, the greater the likelihood of it being useful.

[Completeness] What is the volume and velocity of the intelligence?

“How much” intelligence is actually produced? Numbers don’t matter that much – but if the number is ridiculously small or ridiculously large, it is an indicator of possible issues.

[Accuracy] How is the intelligence classified and curated?

Drive for transparency in their process which helps improve your evaluation on accuracy. Be wary of “silver bullet” buzz-word answers such as “machine learning” or “cloud.”

[Accuracy] How is the intelligence validated?

Do you want to track down false positives all day? No! Do you want to rely on poor analysis? No! Make sure this question gets enough attention.

Related questions: How often is it re-validated? How are false positives handled? How can customers report false positives? What is your false positive rate? How many times in the last month have you had to recall or revise an intelligence report?

[Accuracy] Does the intelligence expire?

Expiration of intelligence is key. Is there a process which continuously validates the intelligence?

[Timeliness] How quickly is the intelligence made available to customers after detection?

Related questions: What part of your process delays intelligence availability? What is the slowest time to availability from initial detection?