YouTubeCrawler is a fast, efficient, and user-friendly tool for collecting data from YouTube channels. It provides a Graphical User Interface (GUI) built with tkinter and uses its own database to store and manage the gathered data. This project leverages only the requests library, ensuring faster performance than the free YouTube API.
- Ensure you have the tkinter installed. On mac os, you should run in the terminal:
brew install python-tk
- Clone this repository to your local machine with:
git clone https://github.com/Shulcz/youtube_crawler.git
- Navigate to the Repository Directory:
cd YouTubeCrawler - Set up a Virtual Environment (Optional but Recommended):
python -m venv venv
- Activate the virtual environment:
- On macOS and Linux:
source venv/bin/activate
- Install Required Packages:
pip install -r requirements.txt
- Run the Application:
python main.py
This will launch the application, and you can start using its features via the GUI. Use the GUI to start crawling YouTube channels, managing your collected data, and executing SQL queries.
Open the "YouTube Crawler" tab in the application. In the first input field, enter the channel name (from the YouTube channel URL, using "@", for example: @PewDiePie, @MrBeast). In the second input field, specify how many videos you want to parse from the channel. In the third input field, set the number of comments to collect for each video. Click the "Run Crawling" button and wait for the progress bar to fill up. After the crawling is complete, you can interact with the data using the built-in tools or for other purposes.
Open the "SQLViewer" tab in the application. Choose the table you want to view from the dropdown menu. Click the "Show the table" button to display the table's contents. You can also sort the table by clicking on a column header and adjust column widths right in the application.
Open the "DataBaseExecutor" tab in the application. Enter any SQL query in the input field. Click the "Execute a request" button to execute the query and receive the results.
Open the "DataBaseSearcher" tab in the application. Choose the parameter for your search: For "User," search for users with the specified name. For "Date," input a date in the format "YYYY-MM-DD" to retrieve comments starting from that date. For "Channel," find all channels with the provided name. For "Word," display comments containing the entered word. For "Min messages," retrieve all users with a message count exceeding the specified value (integer input required).
Contributions to YouTubeCrawler are welcome. If you have ideas for improvements or new features, please open an issue or submit a pull request.