Scraping-Data-with-Python

Hacker News Data Scraper

You can use this script to quickly receive Stories/Links in your terminal without using a web browser, that can help you save time.

This script collects articles from Hacker News that have received more than 100 hacker news points. For data scraping, we'll be utilising Beautiful Soups.

Setting up the Virtual Environment for Non-Nerds

run setup.bat

Running the script for Non-Nerds

Setting up the Virtual Environment

python -m venv venv venv\Scripts\activate pip install -r requirements.txt

Running the script

python scrape.py

Using the script

1. Run the command python scrape.py
2. The script will now prompt you for the desired number of pages to scrape.
3. Be moral and ethical, don't scrape more than required. This may cause unnecessary burden on Hackernews Servers.
4. Enjoy!

GIF demo