Python

What Is OSINT?

Open Source Intelligence (OSINT) is a methodology for collecting, analyzing, and decision-making using publicly available sources of data. According the Wikipedia, OSINT sources can be devided into te following categories:

• Media, print newspapers, magazines, radio, television
• Internet, online publications, blogs, discussion groups, citizen media
• Public government data, public government reports, budgets, hearings, telephone directories, press conferences, websites, speaches
• Professional and academic publications, information acquired from journals, conferences, symposia, academic papers, dissertations, theses
• Commercial data, commercial imagery, financial and industrial assessments, and databases
• Grey literature, technical reports, preprints, patents, working papers, business documents, unpublished works, newsletters

The purpose of OSINT is to create a tailored level of knowledge (or intelligence) for supporting individuals and groups in making decisions.

A vast amount of information is available publicly. OSINT Framework provides a hierarchical view of hundreds of OSINT resources broken down by a variety of indicators.

What Is SpiderFoot?

SpiderFoot is an open source tool, built in Python, that can query a large number of data sources (over 100 according the website) to gather information on a number of different targets including ip addresses, domain names, and even bitcoin addresses.

The power of SpiderFoot comes from Modules. Modules are how SpiderFoot organizes data into containers. Some Modules like those that integrate with Shodan, AlienVault OTX, and HaveIBeenPwned, required an API key from those individual services. API Keys can be imported/exported as needed. Approximately 60 services that require API’s are available via SpiderFoot.

Scanning in SpiderFoot is as simple as giving the scan a title, a target, and then by selecting the Use Case, Required Data, or Modules that you’d like to use. Scans can be as detailed or as broad as you’d like.

Results are available via several dashboards including the Summary visual below. You can also browse the data in a table, and exclude duplicates, as well as view the data in a graph showing you the connections between data points.

Spiderfoot Scan Summary Panel

In summary, SpiderFoot is a web-based tool for collecting, analyzing and storing OSINT data, and is completely open source. It has its limits, like only being able to complete one scan at a time. However, it’s so easy to setup and can be virtualized using Python Virtual Environments, that analysts can easily have their own instances.