A simple web crawler that crawls a specified website and generates a report of internal links found during the crawling process.
- Crawls a website starting from a base URL
- Normalizes and tracks unique URLs
- Handles both absolute and relative links
- Generates a report showing the number of links to each page
- Node.js
- npm
- Clone the repository
- Run
npm installto install dependencies
npm start [website_url]npm start https://example.comRun tests using Jest:
npm test- JSDOM
- Fetch API
- Jest (for testing)