> ## Content Index
> Fetch the complete content index at: https://darkwebinformer.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Link Dumper - Web Crawler for Extracting Links and JavaScript Files
- URL: https://darkwebinformer.com/link-dumper-web-crawler-for-extracting-links-and-javascript-files/
- Published: 2025-02-24T00:41:21.000Z
- Updated: 2025-09-04T22:23:53.000Z
- Author: Dark Web Informer
- Tags: Tools

![](https://storage.ghost.io/c/6b/16/6b16ac9c-cd67-432f-b0f3-bbec941084ff/content/images/2025/02/4309590834092-2.png)

## 📌 Overview

[](https://github.com/walidzitouni/Link%5Fdumper#-overview)

**Link Dumper** is a powerful Python-based web crawler designed for **pentesting and reconnaissance**. It scans websites for URLs and extracts **JavaScript (.js), text (.txt), JSON (.json), and XML (.xml)** files. This is useful for **subdomain enumeration, API key discovery, and security analysis**.

## 🚀 Features

[](https://github.com/walidzitouni/Link%5Fdumper#-features)

- ✅ **Extracts URLs** from `<a>` and `<script>` tags
- ✅ **Finds JavaScript files** that might contain sensitive data
- ✅ **Crawls additional links** recursively for deeper analysis
- ✅ **Extracts API keys & version numbers from JavaScript files**
- ✅ **Saves results in JSON format** for further processing
- ✅ **Handles relative and absolute links** automatically
- ✅ **Multi-threaded crawling** for fast performance

---

## 🛠️ Installation

[](https://github.com/walidzitouni/Link%5Fdumper#%EF%B8%8F-installation)

### **1️⃣ Clone the Repository**

[](https://github.com/walidzitouni/Link%5Fdumper#1%EF%B8%8F%E2%83%A3-clone-the-repository)git clone https://github.com/walidzitouni/Link\_dumper.git  
cd Link-Dumper

### **2️⃣ Install Dependencies**

[](https://github.com/walidzitouni/Link%5Fdumper#2%EF%B8%8F%E2%83%A3-install-dependencies)pip install -r requirements.txt

---

## 📌 Usage

[](https://github.com/walidzitouni/Link%5Fdumper#-usage)

### **Basic Usage**

[](https://github.com/walidzitouni/Link%5Fdumper#basic-usage)

Run the script and enter a target URL and Select \[1\]:

bash main.sh

or

python3 Link\_dumper.py

For version 2, you can also provide a file with a list of URLs and Select \[2\]:

bash main.sh 

### **Example Output**

[](https://github.com/walidzitouni/Link%5Fdumper#example-output)

The tool will generate:

- `file_links.json` → Contains extracted links
- `file_js.json` → Contains extracted JavaScript file URLs

---

## 🔥 Features in Detail

[](https://github.com/walidzitouni/Link%5Fdumper#-features-in-detail)

### **1️⃣ Crawling & Link Extraction**

[](https://github.com/walidzitouni/Link%5Fdumper#1%EF%B8%8F%E2%83%A3-crawling--link-extraction)

- Parses **all `<a>` and `<script>` tags** on a given website.
- Extracts **internal & external URLs**.
- Saves extracted URLs in `file_links.json`.

### **2️⃣ JavaScript File Detection**

[](https://github.com/walidzitouni/Link%5Fdumper#2%EF%B8%8F%E2%83%A3-javascript-file-detection)

- Finds **all JavaScript files (`.js`)**.
- Useful for discovering **API endpoints, hidden parameters, and sensitive data**.
- Saves `.js` file URLs in `file_js.json`.

### **3️⃣ Recursive Crawling**

[](https://github.com/walidzitouni/Link%5Fdumper#3%EF%B8%8F%E2%83%A3-recursive-crawling)

- Reads URLs from `file_links.json`.
- Recrawls found pages for **deeper enumeration**.
- Avoids duplicate links and saves unique results.

### **4️⃣ Multi-Threading**

[](https://github.com/walidzitouni/Link%5Fdumper#4%EF%B8%8F%E2%83%A3-multi-threading)

- Uses `concurrent.futures` for parallel processing.
- Makes scanning large websites much faster.

---

## ⚠️ Legal Disclaimer

[](https://github.com/walidzitouni/Link%5Fdumper#%EF%B8%8F-legal-disclaimer)

This tool is intended for **educational and ethical security testing purposes only**.  
**Do not use it on websites without explicit permission.**

The author takes **no responsibility** for any misuse of this tool.

---

## 📜 License

[](https://github.com/walidzitouni/Link%5Fdumper#-license)

**MIT License** \- Feel free to use and modify this tool.

---

## 👤 Author

[](https://github.com/walidzitouni/Link%5Fdumper#-author)

- **Your Name** (`@Napoli1372 aka walidzitouni`)
- LinkedIn: [Your Profile](https://www.linkedin.com/in/walid-zitouni-634809299/)
- Twitter: [Your Profile](https://x.com/walidzitouni04)

---

### **What’s Included:**

[](https://github.com/walidzitouni/Link%5Fdumper#whats-included)

✅ Proper Markdown formatting  
✅ Installation, usage, and advanced examples  
✅ Legal disclaimer  
✅ License information

Would you like **badges (Python version, license, etc.)** or a **contribution guide** added? 🚀