Active Response

Always A Bad Day For Adversaries

Tag: questions

Questions for Evaluating an External Threat Intelligence Source

By

On March 2, 2016

In Best Practices

I’ve spoken before on the cost of poor threat intelligence and its risk to an organization.  I’ve also spoken about the 4 qualities of good intelligence: relevance, timeliness, accuracy, and completeness. To better evaluate threat intelligence sources – DRIVE FOR TRANSPARENCY!  If you treat threat intelligence like a black box you’re going to lose.

Here are questions to use when evaluating an external source. These are just a starting point or additions to your own list based on your unique needs.

[Relevance] Why do I need threat intelligence?

Before you go out evaluating threat intelligence sources, you need to know what you’re looking for.  This is best done using a threat model for your organization and asking where threat intelligence supports visibility and decision making within that model.  Remember, your own threat intelligence is almost ALWAYS better than that produced by an external source.  External intelligence should complement your own visibility and reduce gaps.

Kudos: Thanks to Stephen Ramage for his comment highlighting the exclusion of such a critical question.

[Relevance] What types of intelligence are available?

Strategic country-level reporting? Cyber threats mixed with political threats?  Technical indicators?  Campaign behaviors?  Written context?  These all determine how useful, actionable, and relevant the intelligence will be for your organization.

[Relevance] Give me your context!

Make sure you understand the context provided with any data.  There is a difference between threat data and threat intelligence.  Intelligence helps drive effective decision-making.  Context makes data relevant.

[Relevance] Which threat types?

Is it limited to botnet C2 nodes?  Commodity threats in general?  Does it cover targeted threats?  Does the threat intelligence provide insight into your threat model?

Related Questions: How many unique threats are distinguishable in the intelligence?

[Relevance] How many direct threats to my organization or those in my industry has your intelligence identified?

Has the source ever shown direct success in highlighting threats in your industry?

[Relevance] How is the intelligence made available to consumers?

If the intelligence is not provided in a usable form, it will not be successful.

[Relevance] What types of use-cases produce the best experience/feedback?  In which use cases has your intelligence failed?

This is a soft-ball question but one which should provoke a good question-answer session.  The answers will illuminate their decisions developing the intelligence and highlight where the intelligence may fit best (or not fit at all).

Related question: What threat model is this intelligence attempting to address?

[Completeness/Relevance] What is the source of the intelligence?

Is this intelligence derived from human sources crawling the dark-web?  Global network apertures?  VirusTotal diving?  This question should frame their visibility into threats and inform the types of intelligence expected.  This also highlights any natural biases in the collection.  Look for sources of external intelligence which complement your own internal threat intelligence capabilities.

[Completeness] What phases of the kill-chain does the intelligence illuminate?

Understand how wide, against any single threat, the intelligence goes.  Does it only show C2, or will it also illuminate pre-exploitation activities as well.  The wider the intelligence, the greater the likelihood of it being useful.

[Completeness] What is the volume and velocity of the intelligence?

“How much” intelligence is actually produced?  Numbers don’t matter that much – but if the number is ridiculously small or ridiculously large, it is an indicator of possible issues.

[Accuracy] How is the intelligence classified and curated?

Drive for transparency in their process which helps improve your evaluation on accuracy. Be wary of “silver bullet” buzz-word answers such as “machine learning” or “cloud.”

[Accuracy] How is the intelligence validated?

Do you want to track down false positives all day?  No!  Do you want to rely on poor analysis? No! Make sure this question gets enough attention.

Related questions: How often is it re-validated?  How are false positives handled?  How can customers report false positives?  What is your false positive rate?  How many times in the last month have you had to recall or revise an intelligence report?

[Accuracy] Does the intelligence expire?

Expiration of intelligence is key.  Is there a process which continuously validates the intelligence?

[Timeliness] How quickly is the intelligence made available to customers after detection?

Related questions: What part of your process delays intelligence availability?  What is the slowest time to availability from initial detection?

20 Questions for an Intrusion Analyst

By

On March 31, 2012

In Analysis

There are many approaches to finding the right people with the right talent to solve problems.  Intrusion analysis and incident response is no different.

I recently saw a great recruiting quiz to test potential employees in various knowledge areas which included programming, packet analysis, protocol analysis, snort rule writing, reverse engineering, data encoding, advanced mathematics, and other topics.  The test was designed so that it crossed so many topics one person would likely not successfully complete it.  However, it would highlight a person’s strengths and interests to give the assessor a more complete picture of the applicant.

This made me think, what topics and questions would I use to achieve the same effect?   After some deliberation, I have developed my own “20 Questions for an Intrusion Analyst” recruitment quiz (below) to highlight areas I think are important about a potential analyst joining a team.

As you may notice, I have covered several areas with these questions: analytic reasoning, creativity, adversary operations, packet analysis, intrusion detection, programming, reverse engineering, vulnerability analysis, exploit writing, and teaming.

I am purposefully not providing the answers 🙂

20 Questions for an Intrusion Analyst

  1. Describe you first experience with a computer or network threat
  2. You are given 500 pieces of straw and told that one piece is a needle which looks like straw.  How would you find the needle?  What other pieces of information would you like to have?
  3. Explain the difference between intrusion and extrusion detection
  4. Describe an adversary pivot, give an example, and explain its importance to intrusion analysis.
  5. Describe your analytic biases.
  6. Use the bit string 1101 to answer the following questions:
    1. The bit string when XORed with 0
    2. The decimal value of the string
    3. The string represented in hexadecimal
    4. Does this represent a printable ASCII character?  If so, which character?
  1. What is your favorite intrusion detection system?  What are its biases and limitations?
  2. Circle any of the following films you have seen: Hackers, War Games, Sneakers, Tron
  3. Describe a method to find an intruder using only network flow data (no content).
  4. Explain insertion and evasion of intrusion detection systems.  Give an example.
  5. Describe the activity detected by the following Snort rule.  What could be done to make the rule more effective?   alert icmp $EXTERNAL_NET any <> $HOME_NET any (msg: “activity alert!”; sid:10000011; content:”MZ”;)
  6. Write a code snippet to sort the following data by the first column
10,bob
8,sally
2,suzy
3,billy
5,joey
  1. How much time/week do you spend on your own researching computer security/threat topics?  What sources do you use to maintain situational awareness on threats in the wild?
  2. What will the following code print out?  Is there a vulnerability in the code?  If so, describe the vulnerability and a potential method of exploitation.
#include
#include
int main(int argc, char *argv[])
{
   char string[40];
   strcpy(string, argv[1]);
   printf("The message was: %s\n", string);
   printf("Program completed normally!\n\n");
   return 0;
}
  1. Describe and explain any “interesting” entries in the netstat log:
Proto Local Address     Foreign Address    State
 TCP  0.0.0.0:53        0.0.0.0:0          LISTENING
 TCP  0.0.0.0:135       0.0.0.0:0          LISTENING
 TCP  0.0.0.0:445       0.0.0.0:0          LISTENING
 TCP  0.0.0.0:5357      0.0.0.0:0          LISTENING
 TCP  192.168.1.4:53    91.198.117.247:443 CLOSE_WAIT
 TCP  192.168.1.4:59393 74.125.224.39:443  ESTABLISHED
 TCP  192.168.1.4:59515 208.50.77.89:80    ESTABLISHED
 TCP  192.168.1.4:59518 69.171.227.67:443  ESTABLISHED
 TCP  192.168.1.4:59522 96.16.53.227:443   ESTABLISHED
 TCP  192.168.1.4:59523 96.16.53.227:443   ESTABLISHED
 TCP  192.168.1.4:53    208.71.44.30:80    ESTABLISHED
 TCP  192.168.1.4:59538 74.125.224.98:80   ESTABLISHED
 TCP  192.168.1.4:59539 74.125.224.98:80   ESTABLISHED
  1. A host sends out an ICMP ECHO REPLY packet.  List all of your hypotheses to explain this activity.
  2. Describe the protocol stack of the following packet and the payload. Is the packet legitimate? Why or why not?
0000  00 00 c0 9f a0 97 00 a0 cc 3b bf fa 08 00 45 10   .........;....E.
0010  00 89 46 44 40 00 40 06 72 c7 c0 a8 00 02 c0 a8   ..FD@.@.r.......
0020  00 01 06 0e 00 17 99 c5 a1 54 17 f1 63 84 80 18   .........T..c...
0030  7d 78 cc 93 00 00 01 01 08 0a 00 9c 27 34 00 25   }x..........'4.%
0040  a6 2c ff fa 20 00 39 36 30 30 2c 39 36 30 30 ff   .,.. .9600,9600.
0050  f0 ff fa 23 00 62 61 6d 2e 7a 69 6e 67 2e 6f 72   ...#.bam.zing.or
0060  67 3a 30 2e 30 ff f0 ff fa 27 00 00 44 49 53 50   g:0.0....'..DISP
0070  4c 41 59 01 62 61 6d 2e 7a 69 6e 67 2e 6f 72 67   LAY.bam.zing.org
0080  3a 30 2e 30 ff f0 ff fa 18 00 78 74 65 72 6d 2d   :0.0......xterm-
0090  63 6f 6c 6f 72 ff f0                              color..
  1. What type of encoding is used in this example: aGVsbG8gd29ybGQNCg==
  2. Who do you turn to most on technical questions?

You didn’t expect the 20th question to be here did you?  You should expect the unexpected by now.

Powered by WordPress & Theme by Anders Norén