Open Source Intelligence

Indicators of Compromise (IOC) and How Security Professionals use them to defend against threats

If you’ve been in a security field that involves incident response/threat hunting, you’ve probably heard of the term “indicator of compromise” (IOC). In computer forensics, an IOC is an artifact that can be observed on a network or a host that indicates, with a relatively high level of confidence, a computer intrusion.

Not all artifacts from a cyber event will be considered an IOC. Artifacts that are left during an attempted, but perhaps not successful, intrusion are known as precursors. While an IOC can help to identify an intrusion that may have already occurred, a precursor can help to identify when an intrusion may be in the process of occurring.

What Do We Consider An Indicator of Compromise?

Indicators of compromise can range from a simple string to a series of actions performed in a certain order. Here is a comprehensive list of examples of different types of IOCs.

  • IP Address
  • Domain Name
  • URL
  • Website Name
  • File Hash
  • User/Account Name
  • Service and Process Names
  • Registry Key, Path, and Value
  • Directory Path
  • Virus Signature
  • “Strings” within a file
  • DNS txt record abnormalities
  • Files referencing /etc
  • System API Call

While those pieces of data can be easily found using a variety of security tools, there are also a number of behaviours that may indicate an instrusion.

  • Unusual/Unaccounted for outbound traffic
  • Unusual/Unaccounted for traffic between client networks (subnets)
  • Privileged account anomalous usage
  • User account active from anomalous IPs
  • Excessive failed logins
  • Activity from unexpected geographic regions
  • Increased traffic to specific resource
  • Baseline changes in RDBMS activity
  • Change in web browsing requests / request habits
  • Well known port vs. application usage
  • Encryption should be used over normally encrypted ports
  • Unexplainable Registry and File system changes
  • Malformed, overy short, and anomalous DNS requests
  • Patching that didn’t follow the official Change Management schedule
  • Changes to mobile platforms
  • Unexplained file creation
  • Web Browsing spikes and anomalous traffic patterns during irregular times
  • Service changes
  • Anomalous account management activity
  • Anomalous firetransfers
  • Changes to file permissions
  • Changes outside of normal maintenance hours
  • Access to URLs outside of the Alexa Top 1 Million
  • High CPU usage

There are a number of other behaviours that I’m sure you can think of that may also be an indicator of compromise. One of the best sources for techniques that that can be used to build a larger list of behaviours is the Mitre ATT&CK Enterprise Techniques matrix.

How Is This Information Valuable to a Security Professional?

Security monitoring is not a perfect science and relies on security professionals to create, tune, and update the various security tools that are used to detect information security threats, including a tool known as the Security Information and Event Management (SIEM) system. A SIEM is a key tool for Security Operations teams that ingests, manipulates, and displays logs from various services and provides security professionals a perfect tool for investigating security incidents.

Security professionals in the monitoring and detection role can use an indicator of compromise to build alerts and detection content based on real-world events and detections. This is an essential part within any SOC program, and is a proactive approach to cyber defense. Using IOCs in your environment is also a reactive approach to cyber defense, but can help to identify threats in your environment that may have been missed initially. For example:

A security professional will take a list of IOCs related to a recent data breach performed by Darkside, and investigated by FireEye. FireEye, like many security companies, publish intelligence data for use by others.

The security professional will take that list and not only include those IOCs within existing detection content, but they'll also review historical logs for the aforementioned IOCs which can give the security team a limited amount of certainty that the environment wasn't previously impacted by a newly detectable threat.

IOC Standards

IOCs are a type of open source intelligence (OSINT) and are available from many sources in many formats. You may get something as simple as a list of IP Addresses, however, there has been some work over the years to create a standard way of creating, storing, and sharing indicators of compromise.

STIX (Structured Threat Information eXpression)
TAXII (Trusted Automated eXchange of Intelligence Information)

Essentially, both STIX and TAXI offer the same results; they just do it in different ways. STIX uses relationships between objects to build intelligence, while TAXII utilizes collections within a more standard client-server distribution model. Many SIEM tools can ingest both STIX and TAXII feeds for automated intelligence gathering and for updating IOC block lists.

An Introduction to Automating Open Source Intelligence Using SpiderFoot

SpiderFoot - An Open Source Intelligence Tool

What Is OSINT?

Open Source Intelligence (OSINT) is a methodology for collecting, analyzing, and decision-making using publicly available sources of data. According the Wikipedia, OSINT sources can be devided into te following categories:

  • Media, print newspapers, magazines, radio, television
  • Internet, online publications, blogs, discussion groups, citizen media
  • Public government data, public government reports, budgets, hearings, telephone directories, press conferences, websites, speaches
  • Professional and academic publications, information acquired from journals, conferences, symposia, academic papers, dissertations, theses
  • Commercial data, commercial imagery, financial and industrial assessments, and databases
  • Grey literature, technical reports, preprints, patents, working papers, business documents, unpublished works, newsletters

The purpose of OSINT is to create a tailored level of knowledge (or intelligence) for supporting individuals and groups in making decisions.

A vast amount of information is available publicly. OSINT Framework provides a hierarchical view of hundreds of OSINT resources broken down by a variety of indicators.

What Is SpiderFoot?

SpiderFoot is an open source tool, built in Python, that can query a large number of data sources (over 100 according the website) to gather information on a number of different targets including ip addresses, domain names, and even bitcoin addresses.

SpiderFoot Scan Target Panel

The power of SpiderFoot comes from Modules. Modules are how SpiderFoot organizes data into containers. Some Modules like those that integrate with Shodan, AlienVault OTX, and HaveIBeenPwned, required an API key from those individual services. API Keys can be imported/exported as needed. Approximately 60 services that require API’s are available via SpiderFoot.

Scanning in SpiderFoot is as simple as giving the scan a title, a target, and then by selecting the Use Case, Required Data, or Modules that you’d like to use. Scans can be as detailed or as broad as you’d like.

SpiderFoot Scan Settings Panel

Results are available via several dashboards including the Summary visual below. You can also browse the data in a table, and exclude duplicates, as well as view the data in a graph showing you the connections between data points.

Spiderfoot Scan Summary Panel

In summary, SpiderFoot is a web-based tool for collecting, analyzing and storing OSINT data, and is completely open source. It has its limits, like only being able to complete one scan at a time. However, it’s so easy to setup and can be virtualized using Python Virtual Environments, that analysts can easily have their own instances.