Author: Sergio Caltagirone Page 3 of 5

Sergio is the head of threat intelligence analysis at Microsoft and operates a global threat intelligence mission to discover, understand, track, and disrupt malicious activity against Microsoft and its customers. He is passionate about empowering defenders with timely and accurate information and moving information security from a reactive to a proactive posture.

Before Microsoft he worked for the United States Government for 8+ years and built and led several threat intelligence missions.

Sergio grew up in Western Washington State and attended the University of Portland where he received his Bachelor of Science in Computer Science and also a degree in Theology with a strong liberal arts background. He went on to the University of Idaho in 2005 where he received his Master of Science in Computer Science. At Idaho Sergio expanded his education by becoming the first computer science student allowed to take Law classes where he focused on legal topics connected to computer security.

Sergio has been very active in research and innovation receiving his first patent working with cognitive psychologists on graphical passwords (US20100169958) and had over 12 publications and a thesis on the topic of Active Response. He has gone on to work in several organizations doing computer and network security, forensics, and intrusion analysis.

Sergio is also the Chief Scientist of The Center for Cyber Intelligence Analysis and Threat Research working towards the goal of moving cyber from an art to a science.

Sergio Caltagirone+

CART: The 4 Qualities of Good Threat Intelligence

By Sergio Caltagirone

On July 1, 2015

In Analysis

I write often of poor quality threat intelligence which pervades the security community. Poor quality threat intelligence not only has a heavy cost on its consumers, it also threatens the confidence threat intelligence consumers place in their providers. Confidence is the cornerstone of threat intelligence. Nobody will take intelligence from an untrustworthy source and act – at least they shouldn’t. It is important that the producer and consumer trust each other. That trust needs to be based on transparency and verification.

However, how does one appropriately assess threat intelligence? The first step must be to identify the qualities which define “good” threat intelligence. However, these are not binary qualities – there is a clear gradient based on use case. Timeliness is a good example of this gradient as some intelligence (likely more strategic) has a more fluid timeliness requirement while tactical threat intelligence has stricter requirements.

Further, one single threat intelligence source will not likely be able to satisfy all qualities simultaneously. For instance, it is unlikely any one provider will have complete visibility across Diamond elements or Kill Chain phases and consumers will have to rely on more than one to achieve satisfactory completeness.

The four qualities are (CART): Completeness, Accuracy, Relevance, and Timeliness.

Completeness

Threat intelligence must be sufficiently complete to provide effective detection and (hopefully) prevention. For instance, providing a domain indicator used in the exploitation of only one victim is not sufficient for other victims and therefore the intelligence is effectively incomplete and unhelpful.

Accuracy

Threat intelligence must save organizations more in success than it costs them in errors and mistakes.

Relevance

Threat intelligence must address a threat to the organization in a method that allows for effective action. Intelligence addressing threats not faced by the organization is of no value. Further, intelligence delivered in a type or method not usable by the organization is also unhelpful.

Timeliness

Threat intelligence must be received and operationalized fast enough to make an impact more valuable than the cost of the threat intelligence itself.

The Cost of Bad Threat Intelligence

By Sergio Caltagirone

On May 22, 2015

In Analysis, Best Practices

There is no doubt that threat intelligence is now “a thing.” At RSA 2015 I couldn’t help but notice how many vendor booths were hawking their relevance to threat intelligence. I hear about a threat intelligence start-up almost weekly. That is not surprising given venture capital is flowing and C-suite customers are now investing in “threat intelligence.” Everyone wants a piece of the pie.

While market growth for threat intelligence produces innovations it also produces negative by-products (welcome to capitalism). The most concerning by-product is the reduction in threat intelligence quality.

A growing number of published threat intelligence reports contain inaccuracies and poor analysis. A growing number of indicators across a variety of producers are either stale, irrelevant, or generate so many false positives to be useless.

What so many fail to realize is the cost of poor quality intelligence. Here are some of the costs:

If a single threat intelligence-sourced alert generates $1000 worth of time to investigate a false positive, it is easy to see how that relatively small amount can multiple within an organization and across enterprises worldwide.
If an intelligence producer reports incorrectly categorizes a threat as APT (say instead of cyber crime) an organization’s security response to the threat will be (and should be) different likely involving a deeper investigation. Again, this additional, and likely unnecessarily deep, investigation is costly in both time and resources.
Every poor quality report costs time to read and digest. Time that could be spent understanding a high-quality report.
Every poor association or correlation derails an analytic effort at an organization.

Because organizational security resources are finite and already stretched thin these mistakes, errors, and poor practices consume critical resources which could be spent on other problems and reduces the security of an organization.

Two market elements have caused this quality reduction:

A need to garner attention in the growing cacophony of the threat intelligence market feeding a “first to publish” mentality which usually results in a “rush to publish.”
A lack of customer education resulting in a poor evaluation of providers thereby incentivizing the wrong aspects of threat intelligence – such as volume of indicators over their quality or relevance

Obviously, only threat intelligence providers can solve the problem, but what pressures can help drive effective change? Here are some:

Threat intelligence customers armed with evaluation criteria (particularly quality metrics) which helps them leverage threat intelligence effectively without generating unnecessary costs – this will help create market drivers for higher quality
Industry must self-police bad intelligence by being honest with ourselves and each other.
Threat intelligence aggregation platforms should have quality assessment capabilities informing the intelligence consumer of potential problems (likewise they are also be in a position to highlight timely, relevant, and unique intelligence of great value)
Threat intelligence analysts trained in analytic tradecraft stressing quality and accepting an ethical duty

Security professionals practicing threat intelligence must understand the implications of mistakes and poor analysis. Bad intelligence can and does decrease the security effectiveness of an organization. Therefore it is an ethical duty of the threat intelligence practitioner to reduce errors. Threat intelligence is difficult – intelligence by definition attempts to illuminate the unknown and works by making judgments with imperfect data – errors are natural to the domain. But, with proper practices and procedures bad intelligence can, and must, be minimized.

15 Things Wrong with Today’s Threat Intelligence Reporting

By Sergio Caltagirone

On February 19, 2014

In Best Practices

What I think when I read most threat intelligence reporting

As I have written before, intrusion analysis is equal parts knowing the technical elements of an intrusion and being an analyst. However, most in this domain spend an inordinate amount of time studying technical details compared to honing any analytic skills.

How long has it been since you’ve taken a highly technical course? (probably within the last year or two) How about an analysis course? (probably in the last 5 years, 10 years, never?)

I read several threat intelligence reports daily. It is painfully obvious how the lack of analytic skill is harming the discipline. Many folks come from technical degree backgrounds and analyze packets and binaries well enough but can’t seem to tell the difference between inductive, deductive, or abductive reasoning. Furthermore, their managers and mentors never recognize a problem, they just send them to more technical courses.

What is the risk? Threat intelligence provides insight and context to improve decision making. The risk of bad intelligence is high. Bad decisions can easily be made from poor intelligence – potentially doing more harm than good. Good analytic practices improve analysis thereby decreasing the risk of poor intelligence. You could have the best packet analysis skills in the world, but if you cannot communicate your conclusions effectively to those who need to act on your information those skills are effectively useless in threat intelligence.

We need to do better. I started this post about a month ago and wrote down a “lesson” whenever I saw an example of poor analysis. Needless to say, I saw some of these several times. (Contrary to the recommendation of others, I will not cite/quote specific examples – I believe that would only name and shame others)

Trend – the word actually means something

How many times per week must I read about a new “trend” from threat intelligence? One or two events does not constitute a trend. Even three or more events, depending on the universe of events, may not constitute a trend. Trends are serious. True trends in adversary activity and methodologies inferred by threat intelligence should drive data collection, analytic tradecraft, and defensive decisions. Before you start throwing out the word trend just because you’ve seen something a few times, consider the millions of other events you’re not seeing and consider if they’re just an anomaly rather than a trend.

Analysts and conclusions are like horses: sometimes you need to lead them to water

In many cases I can follow the logical progression of hypotheses and facts to the conclusion. In some cases I cannot. Either because an analyst failed to include the appropriate evidence/fact on which now an assumption must rest or because of convoluted logical reasoning. Ensure evidence supports your conclusions and the logical reasoning is clear. Don’t assume that what is clear in your mind will be clear in mine.

You can’t be completely confident all of the time – use words of estimative probability

Do you know how often I see the effective use of estimative probability in recent threat intelligence reporting? Almost never. This is a problem. Not everything presented is irrefutable fact; in fact, a good analysis will have a proper mix of data/fact, hypotheses and conclusions. The confidence values of these conclusions vary. When you don’t effectively apply estimative probability and variable measures of confidence it removes value from the analysis and increases the risk of poor decision making by consumers. First, if you don’t know what estimative probability is, LEARN about it. Then learn how and when to apply it properly. Importantly, also know what words/phrases to avoid (i.e. weasel words).

Never be afraid to include contrary evidence

Do you know how many times I saw evidence contrary to the conclusion presented in a threat intelligence report this month? Never. Practice analytic honesty. If there is exculpatory evidence, contrary evidence, or an alternative hypothesis – show it. As long as you’re following some of the other lessons here (e.g., separating fact and hypothesis, using words of estimative probability) it will strengthen your analysis and provide more value to the consumer.

Just because you’ve seen something for the first time doesn’t mean it’s the first time it happened

We all love finding something awesome and telling the world. It’s cool because we all want to know what you’ve found! But, please don’t assume it is the first time it has happened or even the first time it has been seen. Having confidence is critical, but hubris is deadly to analysis.

Don’t operate on an island

You are not alone! Don’t act like it. Share and consume, enrich and enhance. Go ahead and build on the analysis of others (citing appropriately). Whatever your observation point or data sources, they’re not omnipresent. I rarely see analysis reference other (obviously) related pieces. How is that? The power of defenders lies in our community and our ability to work together against an adversary.

Be bold, but don’t be stupid

I like my analysis like I like my coffee: bold. But, there is a line between taking facts to their logical conclusion and taking facts to crazy-land. The difference is logic. Ensure your conclusions and hypotheses follow logically from the facts through induction, deduction, or abduction. If your conclusions cannot be logically traced or tested, then they’re likely living in crazy-land.

Don’t mix hypotheses, conclusions, and facts

Hypotheses, conclusions, and facts are not the same. Your intelligence reports should not treat them as such. Ensure that your readers can effectively separate the three through your use of language, formatting, etc. When the three are confused it can lead to erroneous assumptions by consumers and lead to decisions made on weak conclusions rather than facts.

Save hyperbole for the glossy promotion material

Hyperbole has its place. It doesn’t have a place in threat intelligence. Save that for the glossies. Be precise, honest, and accurate. Don’t embellish or exaggerate. Trust me when I say we have enough people running around like chickens with their heads cut off in this field.

Logical fallacies are just that, get to know them

Enough said. I’m sorry I have to say this, but please understand the differences and applicability of deductive, inductive, and abductive reasoning BEFORE writing your first threat intelligence report. Or, at the very least, have an editor who knows the difference.

Don’t create new words when existing words suffice

I’m not going to name-call here. You know who you are. There are times when words/phrases have multiple meanings. I understand that. But, aside from that….stop it.

Tell a story!

Your analysis is a story. You’re effectively documenting history – studying the past – in the hopes of making conclusions and judgments which will help the present and future. While you are documenting the activity of computers you are ultimately describing the actions caused by adversaries. Just like any story your report should have a beginning, middle, and an end.

Answer my questions

Write as if you are in a conversation. Think about what somebody else may ask of what you’re saying, and address those questions in the text. Any questions left unanswered have the ability to form into assumptions on the part of the consumer/customer. If you don’t have an answer, feel free to write: no further information.

Be concise, be accurate

Practice analytic honesty and respect the time of your reader. The report you’re considering may actually need to be three different reports – one describing all of the malware reverse engineering, one describing all of the network activity, and another describing the threat itself which references the other report. Putting everything in one report does not make it more consumable, it makes it less consumable and allows analysts to muddle up various lines of analysis.

Describe diagrams, charts, and tables both in the narrative text but also in a caption

This is just a pet-peeve of mine, but one which I find increases the readability of threat intelligence reports. Make sure you describe your diagrams, charts, and tables in both the narrative text (as part of the story) and also in a caption. I find this necessary because as I move backwards and forwards through a report reading and re-reading, forming and re-forming logical chains, I don’t want to hunt for the description in the text every time. I also don’t want to jump to the caption in the middle of text if not necessary which breaks my concentration.

Discover All Websites Hosted on an IP Address

By Sergio Caltagirone

On January 5, 2014

In Analysis, Tools

There have been many times I’ve worked to discover all websites being hosted on a single host address (e.g., IP address). This required some effort and none of my techniques generated anything I considered comprehensive or authoritative. Usually only a list good enough to get my analysis to the next step.

I found this very useful post today on how to accomplish that exact task, easily. There is also a bash script posted to do this from the command line.

http://robert.penz.name/722/howto-find-all-websites-running-on-a-given-ip-address/

When I saw the post I immediately recognized how obvious this is. Of course search engines know this information! They crawl the web constantly visiting every website and they must have a website-IP mapping. But, until know I didn’t know they exposed this mapping.

The post shows to how do it in Bing – simply use the following syntax:

ip:XXX.XXX.XXX.XXX

Click here to see an example using current IP address of this website, activeresponse.org: http://www.bing.com/search?q=ip%3A69.195.124.131&go=Submit+Query&qs=bs&form=QBRE

A search to reveal websites hosted on an IP address

One challenge I see of this technique is that Bing does not expose the timestamp of this information. Of course Bing would do some caching of information for performance purposes and as such I cannot guarantee that all of these sites are still hosted on that same IP address. Given the nature of dynamic hosting and cloud services websites can move around pretty quickly depending on their hosting service.

Therefore, and as I’ve cautioned previously on this blog, ensure you know your data source and their biases and limitations. In this case the data may be cached and out-of-date.

On the other hand, having a cache showing what was hosted where in the past is also helpful.

But, it’s a pretty cool and helpful capability to have.

Let me know if you have any other easy ways to accomplish this task! I tried the task with other search engines but was unsuccessful.

DISCLAIMER: I am employed by Microsoft

What It Takes to Fight the Hackers

By Sergio Caltagirone

On January 27, 2013

In News

I’ve practiced cyber security for 10 years. Not as long as some, but longer than most. I don’t consider myself an expert because I don’t believe the field is mature enough to identify an expert. But I’ve fought many battles with the adversary. I’ve felt elated success and stinging failure. I have my share of war stories. I struggle regularly with ethics and moral dilemmas. I try to stay true to the simple promise I made to myself many years ago: always use my powers for good.

I read a very well written article, “What it Takes to Fight the Terrorists,” on the psychological impact of working counter-terrorism for years. The toll of the long hours. The moral dilemmas they face daily and the stress imposed by the cost of failure.

I am not going to sit here and preach that the stress of an intrusion analyst and network defense operator is the same – it is not. At the moment there are real costs to our failure but none as great as that caused by terrorists. We don’t have to wake up and see the results of thousands of innocents dead and question why we could not stop it.

But I’m afraid that one day we will. As our systems become even more interconnected and a greater number of life-safety and community-critical systems become connected, it is a high possibility that a hacker, intentional or not, will cause large-scale loss of life. See my earlier article as an example. Instead of the smell of the site of a terrorist bombing, maybe we will instead be ingrained with an image of an exploded power plant caused by someone behind a computer half a-world away.

Maybe one day there will be a cyber equivalent to 9/11 and those of us who could have stopped it will plumb the depths of our being to answer why we did not stop it. On that day, as with 9/11, the world will change.

They call cyber security the new counter-terrorism. The new nuclear threat for the next 20 years. I’m afraid that one day this article will be written about us. But until that time, we must learn from our counter-terrorists colleagues – from their courage, fortitude, successes, and failures.

5 Cyber Security Predictions for 2013

By Sergio Caltagirone

On January 7, 2013

In Strategy

2012 has been an interesting year with a growth in our understanding of our adversaries and some high-profile international security incidents. 2013 will continue to impress, but differently. It will ultimately be a year of strategic growth.

Here are 5 cyber security predictions for 2013.

1. There will be little change to the threat landscape

There will be little change to the threat landscape in 2013 as our adversaries are already achieving their intent (extrapolating the size and scale of currently known adversary operations) and therefore have little pressure to change. However, I do not see this as holding into 2014 as greater innovation in the threat intelligence and mitigation space is made (prediction #4) and the role of government is better defined (#5).

2. Cyber attacks will have a greater impact to a greater number

As data and service providers co-locate in cloud environments, attacks on the infrastructure providing these services will rise (attackers will always go to where the data lives) resulting in greater collateral damage to non-intended victims simply based on with whom they are co-located.

2.1 Corollary: Risks will be more difficult to assess as control of the location of data and an accurate knowledge of the infrastructure is lost in the cloud. This will cause businesses to continue to mismanage public and customer relations when incidents occur.

3. The cost of cyber threats will grow and there will be an increased awareness and visibility of those costs resulting in greater effective action in the mid-and-long term.

Based on prediction #3 the cost of cyber threats to all organizations will grow. However, as has been the trend, visibility of security issues and incidents will rise forcing business change to address this challenge in new ways (hence prediction #5). Innovation will then lead to greater effective action in the mid-and-long term.

4. The role of government in securing computer systems from domestic and foreign cyber threats will continue to be muddled.

The role of government in any area is generally slow to evolve. Cyber security has not been any different. As governments around the world are consumed by domestic and international economic affairs, little attention will be focused on this problem further delaying necessary action.

5. Private industry, vice government or research, will make great innovations in the threat intelligence and mitigation space.

Based on: (1) the amount of venture capital flowing into cyber security industry to produce innovations in threat intelligence and mitigation, (2) the market growth for such innovations (based on predictions #2 & #3), and (3) with the growth in funding means the ability for private industry to recruit and retain the best talent in the field — it is no great stretch of the imagination to see that this is where the innovations necessary to combat the threat and increase risk and cost on the adversary will originate during 2013 changing the threat landscape in 2014 and beyond.

Snakes and Ladders: How Intrusion Analysis and Incident Response is Like a Board Game and the Critical Role of Pivoting

By Sergio Caltagirone

On November 12, 2012

In Analysis, Defense, Discovery

Pivoting is, in my humble opinion, the most important skill of intrusion analysis and incident response. I have been teaching/training/mentoring intrusion analysts for over 7 years. In my experience, this is the most difficult skill to train as it requires creativity, attention to detail, and a full knowledge of their data sources and how to exploit those.

Pivoting is the ability to identify a critical piece of information and being able to maximally exploit that information across all of your sources to substantially increase your knowledge of the adversary and further identify the next critical piece of information – which is then pivoted upon moving you deeper into the operation of the adversary – hopefully earlier into the kill-chain.

An example: trolling through log files you discover a very odd HTTP user-agent accessing your website. You then query on this user-agent across all the log entries and identify a significant number of users providing this string value. (Pivot on user-agent) You then extract all of those particular log entries and identify a regular time pattern of access indicating automated functionality. You also discover that all the requests have a very odd resource/page request – bob.php. (Pivot on bob.php) You then take that page name (bob.php) and examine all HTTP traffic in your network over the last 2 days and discover that several hosts in your network have been POSTing odd data to bob.php….at this point you may retrieve and conduct a forensic analysis on the hosts, etc. When you finally discover that the adversary has compromised several internal hosts and has had them HTTP POSTing data to a webpage on your external-facing website of which the adversary then uses to extract the information/data. At this point, you now have several pieces of mitigative value: the source IP of the adversary’s infrastructure on the outside, the page deposited on your website, any malicious tools discovered on the hosts, the HTTP traffic, etc. All of which are collectively more valuable to defense than any one of those pieces of information independently.

A Step Function

In this way, analysis and incident response is a step-function. Most of the time analysis is, in a sense, rote. It involves looking through log files, examining and validating alerts, looking at various binaries. Step by step peeling back the onion of the adversary’s operations. At times we even move backwards as an analyst makes an incorrect assumption or a poor hypothesis which costs time/money/resources to recover and correct the analytic path. However, when a piece of critical information is discovered it should be exploited and a deeper knowledge should be achieved moving the analysis to a “new level” of the function substantially increasing the knowledge as a whole – which, in theory, should lead to additional mitigative opportunities.

Chutes and Ladders

My favorite analogy is that of the game of “Chutes and Ladders” (or “Snakes and Ladders” for those outside the US). A player slowly moves across the board block-by-block but then happens on a ladder which moves them up substantially in the board. Other times, they land on a snake/chute which then brings them back down. This is the process of analysis.

Why does this matter? It matters because this understanding can help us better understand the process and model of analysis thereby providing an opportunity for researchers to target parts of analysis to increase the chances/likelihood of a step-function increase in knowledge and decrease the chance of a decrease.

One way is to increase the capability of analytic tools to maximize pivoting. Allowing for an easy and quick way to query other data sources with a new discovery and integrating that into the analytic picture. The tools should also allow an analyst to ‘back-up’ their analysis removing a possible poor path once an error is discovered.

This is just a couple of ideas. I’d love to hear yours.

Two Computer Security Experts Jailed for Failure to Prevent Hospital Hack

By Sergio Caltagirone

On October 31, 2012

In News

Washington DC – After a major computer attack on a hospital network by a relatively unknown hacker caused the death of three patients there were many questions. Why did the hacker do it? Was the hospital doing enough protect its patients? Why wasn’t the security good enough to prevent the attack?

Now that the trial has concluded we have some more answers, but still many questions remain. First, we know that the hacker did not intend to attack a hospital. He thought he was attacking a bank network as part of a protest movement. He was relatively unskilled, using complicated but effective tools downloaded from the Internet. He successfully survailed the bank network, but when it was time for the attack he mistakenly typed in a wrong number for his target unknowingly sending his tools to attack a hospital network.

The computers which were managing the newly installed electronic patient records, which included the medication and dosage, went down causing confusion throughout the hospital. The records which normally hold critical information about a patient’s medical history, allergies, and current state were now gone. Doctors and nurses who were on shift during the day did not know the correct dosage or even the correct drug to administer which were prescribed during the night shift. This led to three patients either being given an overdose or another drug entirely causing a serious, and fatal, reaction.

The hacker was sentenced last month to a life for criminally negligent manslaughter of the three patients. However, in a turn of events, two network security experts were charged with the protection of the hospital’s network are now in jail facing 10 years for their failure to prevent the attack.

Prosecutors argued that the security experts should have detected and prevented the attack well before the damage to the hospital record system. They were specifically trained to do so and in the best position of anyone to detect the hacker and judge the risk. Yet, their failure to do so put the lives of every patient in the hospital at risk and eventually caused the death of three.

The defendants argued that the network was far to large and complicated to be effectively defended and they could not have predicted every possible attack and it’s consequence.

In the end the jury agreed more with the prosecution than defendants. What long-term consequences this holds is still unknown.

This is, of course, a fictitious story based on a real case of the jailed Italian scientists who were convicted of failing to effectively communicate the risk of a major earthquake. 300 people died in that earthquake. As they say, hindsight is 20/20. Looking back one could easily say that the earthquake was imminent given the signs. But then those signs occur in many places around the world daily without the devastating effects of a large earthquake immediately following.

After reading the story of the earthquake scientists, I could not help but think of many scenarios where, as a security professionals, we are asked to assess the risk and ultimately prevent damage to life,safety, and national security critical networks and systems. What if we were wrong and people died? Let alone the guilt I could imagine feeling, would society at large hold us responsible? Should we be held responsible?

I think back to my time studying computer ethics and the various ethical codes I have signed in my life agreeing to act responsibly, take responsibility for risk, and make good decisions. Yet bad things happen. And I cannot say whether society would judge my work good enough in such a situation.

We are the experts. We are being paid to make the right decision in the protection of our networks. There is nobody in a better position than us to make those decisions. We know the network. We know the systems. We know the threat. Yet we still fail.

Will our failure become so great one day that we are held to account for the death of innocents based on our faulty risk assessments and ineffective defenses?

Let us hope not.

How I Work To Music and 15 Songs I Work With

By Sergio Caltagirone

On October 20, 2012

In Uncategorized

I love to work to music. I almost don’t even care to what genre or song I’m listening. Of course, music is a highly personal choice.

While sometimes I listen to Pandora or another Internet radio station for variety, I also have a selection of my favorite albums at hand which I pick out like tools depending on my mood and my current task.

Here are three of my most common tasks and 5 songs that I associate with that task.

Authoring/Writing:

Jack Johnson – I wrote almost my entire thesis to his early albums in a coffee shop.

http://www.youtube.com/watch?v=gVAnlke_xUY

Programming:

Weezer – Their entire Blue and Green albums have led me through over 150,000 lines of code successfully

http://www.youtube.com/watch?v=1MwjX4dG72s

http://www.youtube.com/watch?v=RxI3BFSOns8

http://www.youtube.com/watch?v=LOpYh30ZN5w

[Honorable Mention]

http://www.youtube.com/watch?v=QE1Vge7Naaw

Analysis:

15 Knowledge Areas and Skills for Cyber Analysts and Operators

By Sergio Caltagirone

On September 29, 2012

In Analysis, Education

Rodin’s The Thinker

Here are some knowledge areas which I consider necessary to conduct effective intrusion analysis and operations. In future articles I will go into further details on how to improve your skills in each of these areas (and link them from here). The knowledge areas are not listed in any particular order.

Every organization’s mission, focus, and needs are different and therefore I don’t pretend to define the ‘perfect’ analyst for any mission.

Critical Thinking and Logic

I will be forthright and say that I consider this skill the most important above all others. It is a gateway skill which allows an analyst to become proficient in many others. It is also the skill upon which I rely for analysts to temper their judgments and make the best decision as to how to approach a problem. Logic is complementary to critical thinking and the two cannot be separated. Without a proper foundation in logic critical thinking is ineffective.

US-CERT Incident Reponse Report

Critical Reading and Writing

Critical reading is being able to dissect the text of a document to extract the most important information and apply critical thinking skills to the information.Effective/Critical writing and documentation refers to writing correctly, logically, concisely, and effectively for your audience (which likely includes yourself). Most importantly, write in an organized manner to help others use their critical thinking skills.

History

As I have said previously: “Study History. It provides perspective.” Works like The Cuckoos Egg are a great start; but branch into other areas: military history, biographies of famous leaders, studies of famous events. Learn how others have been able to assess strategic situations, derive tactics, and evolve their strategy to a quickly changing situation. All of these skills are useful in intrusion analysis and incident response. Be able to step back from a situation and apply the lessons learned from others to your own.

Research Methods

In the cyber security domain we face more unknown than knowns. My favorite saying is “no analyst is an island” meaning that there is nobody who knows it all and we need to rely on others and the greater community to help to solve problems. Therefore, a significant skill is the ability to conduct effective research on hard problems to find existing solutions – preventing, as the saying goes, “recreating the wheel.” This skill, more than any other, will increase your effectiveness and efficiency.

This skill can and should be mixed with other skills described – critical reading to get through research material quicker, critical thinking to see through the B.S. and FUD, and effective writing to document your findings so you use it again in the future.

Analytic Approaches and Methods

When facing any problem, being able to identify and evaluate the various approaches to solving the problem is invaluable – some would say critical. Being knowledgeable in as many analytic approaches as possible is invaluable, and being able to create new approaches on-the-fly is even more invaluable.

Learn analytic methods from others. Look for their mixture of logic, research, tool use, and lines of critical thinking and apply them yourself.

Network Protocol Map

Network Protocol Analysis

Know your network protocols. More importantly, be able to research, analyze, and identify new or previously unknown protocols. Don’t be afraid of packets. Use your research methods and critical reading skills to dissect protocol definitions and RFCs.

Programming

A basic knowledge and ability to write computer programs is very useful in that it practices logic skills, helps one better dissect cyber security activities, and allows one to create and/or modify tools quickly as necessary.

Psychology

An understanding of the fundamental theorems of psychology is useful when attempting to determine the intent, context, and motivations of an adversary. For example, knowing and being able to apply the fundamentals of Maslow’s Hierarchy of Needs or Operant Conditioning will go towards influencing your adversary through operations to achieve a positive outcome and better protect your network.

See Also: A Hacker’s Hierarchy of Operational Needs based on Maslow’s Theory of Human Motivation

Hacker Tools and Methodology

Obviously, a working knowledge of hacker tools and methodologies is a must.

Binary Reverse Engineering

IDA Pro Binary Reverse Engineering

All hackers use capabilities and tools to achieve their desired effects. Most of these are binaries either live on command-and-control nodes or are delivered to the target for operations. Having a working knowledge and ability to reverse engineer a binary is necessary to conducting effective analysis Even if your organization has dedicated reverse engineers having this knowledge to effectively communicate and ask intelligent questions of these engineers is just as important.

Host-Based Log File and Forensic Analysis

Understanding the internal workings of a host and operating system help not only in investigations where host data is available but also as a learning tool to understand the adversary’s target environment. This will further inform the analyst by providing greater context to the choices of an adversary given the host environment.

This knowledge should be coupled with that of hacker tools and methodologies and network and host configuration and administration for full effect.

Network and Host Configuration and Administration

As I’ve said in another post, 5 Intrusion Analysis Ideas in 10 Minutes, I believe that cyber security professional should be just as proficient in understanding how networks and hosts are administrated and configured as in how those systems are attacked.

Signature Writing and Detection Tools

Example Snort Rule

Finding malicious activity on your network is important, being able to track that activity and detect when it returns is an imperative. Therefore, analysts and operators should be proficient in their organization’s particular signature and detection tools and learn how to author the best signatures.

It is just as important to understand how a detection tool works but also it’s biases and limitations – so you know when there are potential false positives and false negatives. This is one of my 20 Questions for an Intrusion Analyst.

Incident Response Methodology

Incident response methodology is obviously a requirement for anybody who is part of the incident response team in their organization. However, incident response should be well-known by every intrusion analyst. This is simply because they will likely be generating documentation and analysis for the incident response team. The better they understand the methodology, the better they can tailor their documentation and feedback to the needs of response and mitigation.

Tools

Wireshark

I am fond of saying, “there is no one tool to rule them all,” meaning no single tool will do everything you need. While I think that too much time is spent by cyber security professionals in becoming proficient in a specific tool-set, I cannot under estimate the criticality of these tools to our profession. However, I believe that over reliance on our tools breeds ignorance of the data the tool is processing and analysts become unwilling to challenge and blindly trusting the output.

Therefore, it is important to know how to operate and understand the tools that are best for your mission be it OllyDbg or Wireshark.

Lastly, with a strong or competent programming background, as described previously, you are empowered to write your own tools or improve existing tools for the benefit of the community.