How to avoid to scrape again and again the same data
Table of Contents
- 🎯 Objective
- 🏗 Prerequisites
- 👩💻 Just tell me what to do
- 📦 Suggested node modules
- 🛣️ Related Theme and courses
Store products in a database with node.js to create, read, update or delete data...
- Be sure to have a clean working copy.
This means that you should not have any uncommitted local changes.
❯ cd /path/to/workspace/clear-fashion
❯ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
- Pull the
master
branch to update your local with the new remote changes
❯ git remote add upstream git@github.com:92bondstreet/clear-fashion.git
## or ❯ git remote add upstream https://github.com/92bondstreet/clear-fashion
❯ git fetch upstream
❯ git pull upstream master
-
Create a free account on MongoDB Atlas, Database as a Service (DBaaS) Provider.
-
Create a MongoDB Cluster
-
Connect your node.js server script
const {MongoClient} = require('mongodb');
const MONGODB_URI = 'mongodb+srv://<user>:<password>@<cluster-url>?retryWrites=true&writeConcern=majority';
const MONGODB_DB_NAME = 'clearfashion';
...
const client = await MongoClient.connect(MONGODB_URI, {'useNewUrlParser': true});
const db = client.db(MONGODB_DB_NAME)
...
- Insert the Products into this database
const products = [];
...
const collection = db.collection('products');
const result = collection.insertMany(products);
console.log(result);
- Create at least 3 methods to find products according query.
These 3 methods should
- Find all products related to a given brands
- Find all products less than a price
- Find all products sorted by price
const brand = 'loom';
...
const collection = db.collection('products');
const products = await collection.find({brand}).toArray();;
console.log(products);
- Commit your modification
❯ cd /path/to/workspace/clear-fashion
❯ git add -A && git commit -m "feat(new-products): insert all products"
(why following a commit message convention?)
- Commit early, commit often
- Don't forget to push before the end of the workshop
❯ git push origin master
Note: if you catch an error about authentication, add your ssh to your github profile.
If you need some helps on git commands, read git - the simple guide