Data-scraper for reddit made using java and sqlite.
Table of Contents
- Data-Scraper for Reddit is a GUI-based scraping tool that allows users to scrape data from Reddit using the official API from Reddit
- The GUI is implemented using Java Swing and SQLite for storing user data
- Features 10+ filters and a dedicated sorting panel
- Results are filtered from upto 250 pages
- Final results can be neatly saved in a CSV format
- Java
Linux/Mac
You can install java through Homebrew:
brew install java Windows
Download Java 18 from Oracle and follow the instructions from installer. Make sure you add the installation directory to your system path.
- Go to Reddit to create a free API app
- Select a name such as "dataScraper"
- Select "script"
- Leave the description and about URL as blank
- Use http://localhost:8080 for your redirect URL
- Create the app
- Next download the latest jar file from Releases
- Move the jar file to a new folder
- Run using the following command using the terminal,
java -jar RedditDataScraper-[VERSION].jar
- Click on "Register a new account"
- Enter your details and use client_id and client_secret from your API app
- For more details visit Oauth2
Examples:
- Selecting all posts whose title contains the keywords "putin" and "ukraine" from the subreddit "worldnews"
- Selecting all posts from the subreddit Coronavirus that contains numbers in the title. Results must also have a minimum score of 1000, and minimum comment count of 100. Results are finally sorted by their title.
- Selecting all controversial posts from all subreddits.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE.md for more information.