> ## Content Index
> Fetch the complete content index at: https://darkwebinformer.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Telegram Scrapper 2.0: A powerful Python script that allows you to scrape messages and media from Telegram channels using the Telethon library
- URL: https://darkwebinformer.com/telegram-scrapper-2-0-a-powerful-python-script-that-allows-you-to-scrape-messages-and-media-from-telegram-channels-using-the-telethon-library/
- Published: 2025-07-21T17:00:38.000Z
- Updated: 2025-09-04T22:21:06.000Z
- Author: Dark Web Informer
- Tags: Tools

---

**GitHub:** <https://github.com/robertaitch/telegram-scraper>

---

## What's New in v2.0 🎉

[](https://github.com/robertaitch/telegram-scraper#whats-new-in-v20-)

**Major Performance Improvements:**

- **5-10x faster scraping** with batch database operations
- **3x faster media downloads** with parallel processing (up to 3 concurrent downloads)
- **10-20x faster database operations** through connection pooling and batch insertions
- **Memory-efficient exports** that handle large datasets without running out of memory
- **Enhanced progress reporting** with actual message counts and percentages

**New Features:**

- **Message count display** in channel view
- **Configurable download concurrency** (adjustable in code)
- **Better error handling** with exponential backoff retry mechanism
- **Optimized database structure** with indexes for faster queries
- **Object-oriented design** for better code maintainability

**Technical Improvements:**

- Database connection pooling
- Batch message insertions (100 messages per batch)
- Streaming exports for large datasets
- Improved flood control handling
- Periodic state saving (every 50 messages)

## Features 🚀

[](https://github.com/robertaitch/telegram-scraper#features-)

- Scrape messages from multiple Telegram channels
- Download media files (photos, documents) with parallel processing
- Real-time continuous scraping
- Export data to JSON and CSV formats
- SQLite database storage with optimized performance
- Resume capability (saves progress)
- Media reprocessing for failed downloads
- Enhanced progress tracking with message counts
- Interactive menu interface

## Prerequisites 📋

[](https://github.com/robertaitch/telegram-scraper#prerequisites-)

Before running the script, you'll need:

- Python 3.7 or higher
- Telegram account
- API credentials from Telegram

### Required Python packages

[](https://github.com/robertaitch/telegram-scraper#required-python-packages)

```
pip install -r requirements.txt

```

Contents of `requirements.txt`:

```
telethon
aiohttp
asyncio

```

## Getting Telegram API Credentials 🔑

[](https://github.com/robertaitch/telegram-scraper#getting-telegram-api-credentials-)

1. Visit <https://my.telegram.org/auth>
2. Log in with your phone number
3. Click on "API development tools"
4. Fill in the form:
  - App title: Your app name
  - Short name: Your app short name
  - Platform: Can be left as "Desktop"
  - Description: Brief description of your app
5. Click "Create application"
6. You'll receive:
  - `api_id`: A number
  - `api_hash`: A string of letters and numbers

Keep these credentials safe, you'll need them to run the script!

## Setup and Running 🔧

[](https://github.com/robertaitch/telegram-scraper#setup-and-running-)

1. Clone the repository:

git clone https://github.com/robertaitch/telegram-scraper.git  
cd telegram-scraper

1. Install requirements:

pip install -r requirements.txt

1. Run the script:

python telegram-scraper.py

1. On first run, you'll be prompted to enter:
  - Your API ID
  - Your API Hash
  - Your phone number (with country code)
  - Your phone number (with country code) or bot, but use the phone number option when prompted second time.
  - Verification code (sent to your Telegram)

## Initial Scraping Behavior 🕒

[](https://github.com/robertaitch/telegram-scraper#initial-scraping-behavior-)

When scraping a channel for the first time, please note:

- The script will attempt to retrieve the entire channel history, starting from the oldest messages
- **Significantly faster than previous versions** due to batch processing and parallel downloads
- Initial scraping time depends on:
  - The total number of messages in the channel
  - Whether media downloading is enabled
  - The size and number of media files
  - Your internet connection speed
  - Telegram's rate limiting
- The script uses pagination and maintains state, so if interrupted, it can resume from where it left off
- **Enhanced progress display** shows actual message counts (e.g., "1,500/10,000 messages")
- Messages are stored in the database in batches for optimal performance
- **Media downloads run in parallel** (up to 3 simultaneous downloads) for faster processing

## Usage 📝

[](https://github.com/robertaitch/telegram-scraper#usage-)

The script provides an interactive menu with the following options:

- **\[A\]** Add new channel
  - Enter the channel ID or channelname
- **\[R\]** Remove channel
  - Remove a channel from scraping list
- **\[S\]** Scrape all channels
  - One-time scraping of all configured channels
- **\[M\]** Toggle media scraping
  - Enable/disable downloading of media files
- **\[C\]** Continuous scraping
  - Real-time monitoring of channels for new messages
- **\[E\]** Export data
  - Export to JSON and CSV formats (memory-efficient for large datasets)
- **\[V\]** View saved channels
  - List all saved channels **with message counts**
- **\[L\]** List account channels
  - List all channels with ID:s for account
- **\[Q\]** Quit

### Channel IDs 📢

[](https://github.com/robertaitch/telegram-scraper#channel-ids-)

You can use either:

- Channel username (e.g., `channelname`)
- Channel ID (e.g., `-1001234567890`)

## Data Storage 💾

[](https://github.com/robertaitch/telegram-scraper#data-storage-)

### Database Structure

[](https://github.com/robertaitch/telegram-scraper#database-structure)

Data is stored in SQLite databases, one per channel with **optimized indexes**:

- Location: `./channelname/channelname.db`
- Table: `messages`
  - `id`: Primary key
  - `message_id`: Telegram message ID (indexed)
  - `date`: Message timestamp (indexed)
  - `sender_id`: Sender's Telegram ID
  - `first_name`: Sender's first name
  - `last_name`: Sender's last name
  - `username`: Sender's username
  - `message`: Message text
  - `media_type`: Type of media (if any)
  - `media_path`: Local path to downloaded media
  - `reply_to`: ID of replied message (if any)

### Media Storage 📁

[](https://github.com/robertaitch/telegram-scraper#media-storage-)

Media files are stored in:

- Location: `./channelname/media/`
- Files are named using message ID or original filename
- **Parallel downloads** for faster media acquisition

### Exported Data 📊

[](https://github.com/robertaitch/telegram-scraper#exported-data-)

Data can be exported in two formats with **memory-efficient processing**:

1. **CSV**: `./channelname/channelname.csv`
  - Human-readable spreadsheet format
  - Easy to import into Excel/Google Sheets
  - **Streaming export** handles large datasets
2. **JSON**: `./channelname/channelname.json`
  - Structured data format
  - Ideal for programmatic processing
  - **Memory-optimized** for large files

## Performance Tuning ⚙️

[](https://github.com/robertaitch/telegram-scraper#performance-tuning-%EF%B8%8F)

You can adjust these performance settings in the code:

- `max_concurrent_downloads = 3`: Number of simultaneous media downloads
- `batch_size = 100`: Number of messages processed in each batch
- `state_save_interval = 50`: How often to save progress

## Features in Detail 🔍

[](https://github.com/robertaitch/telegram-scraper#features-in-detail-)

### Continuous Scraping

[](https://github.com/robertaitch/telegram-scraper#continuous-scraping)

The continuous scraping feature (`[C]` option) allows you to:

- Monitor channels in real-time
- Automatically download new messages
- Download media as it's posted with parallel processing
- Run indefinitely until interrupted (Ctrl+C)
- Maintains state between runs

### Media Handling

[](https://github.com/robertaitch/telegram-scraper#media-handling)

The script can download:

- Photos
- Documents
- Other media types supported by Telegram
- **Parallel downloads** for faster processing
- **Improved retry mechanism** with exponential backoff
- Skips existing files to avoid duplicates

## Error Handling 🛠️

[](https://github.com/robertaitch/telegram-scraper#error-handling-%EF%B8%8F)

The script includes:

- **Enhanced retry mechanism** with exponential backoff for failed media downloads
- State preservation in case of interruption
- **Improved flood control** compliance
- Comprehensive error logging for failed operations
- **Better rate limit handling** with automatic waiting

## Limitations ⚠️

[](https://github.com/robertaitch/telegram-scraper#limitations-%EF%B8%8F)

- Respects Telegram's rate limits
- Can only access public channels or channels you're a member of
- Media download size limits apply as per Telegram's restrictions

## Contributing 🤝

[](https://github.com/robertaitch/telegram-scraper#contributing-)

Contributions are welcome! Please feel free to submit a Pull Request.

## License 📄

[](https://github.com/robertaitch/telegram-scraper#license-)

This project is licensed under the MIT License - see the LICENSE file for details.

## Disclaimer ⚖️

[](https://github.com/robertaitch/telegram-scraper#disclaimer-%EF%B8%8F)

This tool is for educational purposes only. Make sure to:

- Respect Telegram's Terms of Service
- Obtain necessary permissions before scraping
- Use responsibly and ethically
- Comply with data protection regulations