GitHub: https://github.com/DedSecInside/TorBot

🧰 TorBot - Onion Crawler & Analysis Tool

TorBot is a flexible crawler for .onion domains and clearnet sites. It allows researchers to gather intelligence from Tor-hidden services and map out link relationships with optional visualization.

🔧 Key Features

• Onion Crawler (.onion)
  Returns the page title or hostname (if no title is available), along with a short description.
• Database Integration (Coming Soon)
  Save crawled links to a database.
• HTML Capture (Coming Soon)
  Output site HTML or save it as a file.
• JSON Export
  Save the discovered link tree in JSON format.
• Custom Domain Crawling
  Crawl any domain, not just .onion addresses.
• Live Link Checker
  Verify if a link is active.
• Built-in Updater
  Easily update TorBot to the latest version.
• Visual Link Tree
  Build a graph of link relationships and export it for analysis.
• Additional Features (Planned)
  More enhancements will be added in future releases.

📦 Dependencies

• Tor (Optional but recommended)
• Python >=3.9
• Poetry (Optional)

Python Dependencies

See pyproject.toml or requirements.txt for a complete list of libraries.

🚀 Installation

Using venv (Recommended)

bashCopyEditpython -m venv torbot_venv
source torbot_venv/bin/activate
pip install -r requirements.txt
pip install -e .
./main.py --help

Using Docker

Build the Docker image:

bashCopyEditdocker build -t torbot .

Run without Tor:

bashCopyEditdocker run torbot poetry run python torbot -u https://example.com --depth 2 --visualize tree --save json --disable-socks5

Run with Tor:

bashCopyEditdocker run --network="host" torbot poetry run python torbot -u https://example.com --depth 2 --visualize tree --save json

⚙️ Command Line Options

Note: -u is mandatory for crawling.

🗺️ Planned Features (Roadmap)

• Visual module revamp
• Implement BFS search logic
• Improve stability & error handling
• Save recent results to a database
• Randomized Tor identity & headers
• Keyword & phrase search
• Social media data extraction
• Enhanced anonymity features
• Screenshot capture of crawled pages

🤝 Contribution Guidelines

Found an issue?
Submit bugs or feature requests here.

Developer Notes:

• Fork the repository and create a new branch.
• Run pytest before submitting pull requests to the dev branch.
• Follow PEP8 coding style.
• Document new features thoroughly.
• Include a clear description when submitting pull requests.