SpiderFoot tutorial: set up an OSINT and attack surface intelligence scanner locally

smicallef/spiderfoot is an automated OSINT tool for threat intelligence, attack surface mapping, and public information collection. It provides a web interface and can also be used from the command line.

Project repository:

https://github.com/smicallef/spiderfoot

Official site:

http://www.spiderfoot.net

Install the Stable Version

The README gives this stable installation method:

1
2
3
4
5


wget https://github.com/smicallef/spiderfoot/archive/v4.0.tar.gz
tar zxvf v4.0.tar.gz
cd spiderfoot-4.0
pip3 install -r requirements.txt
python3 ./sf.py -l 127.0.0.1:5001

After startup, open 127.0.0.1:5001 on your local machine.

Install the Development Version

If you want to track the latest code:

1
2
3
4


git clone https://github.com/smicallef/spiderfoot.git
cd spiderfoot
pip3 install -r requirements.txt
python3 ./sf.py -l 127.0.0.1:5001

The development version may include new features, but it may also introduce unstable changes. For formal use, the stable version is usually the better starting point.

Suitable Tasks

SpiderFoot is suitable for:

Enumerating public information for domains you own.
Mapping an external attack surface.
Collecting leads such as emails, domains, IPs, and leaked information.
Initial triage for threat intelligence.
Security learning and lab exercises.

Usage Suggestions

Do not scan a very large scope on your first run. Start with one test domain you own, then observe module output and false positives. OSINT tools connect to many external data sources, and results still need human judgment. Do not treat scan output as fact without verification.

Compliance Boundary

Only scan targets you own or have explicit authorization to assess. SpiderFoot collects public information, but heavy requests, cross-source correlation, and automated enumeration can still trigger risk controls or create misunderstandings. In an enterprise environment, it is best to document the scan scope and time window before running it.