Skip to content

Latest commit

 

History

History
78 lines (72 loc) · 2.94 KB

tasks.org

File metadata and controls

78 lines (72 loc) · 2.94 KB

GoScrapy: planning next tasks and features for this project

Database

create a docker container with the database

up a mongo database with docker

create a new database with the purpose of testing

get all products by wholesaler

use different database when run tests

make a backup of the database with the data that we are going to use to run the tests

get one product by Id

use _id on mongodb

test insert and saved products

write a dockerfile for automate get up database container, and maybe cleanup testing database

create a collection for every wholesaler

Save data from CSV files in the database, to compare the prices of products at the time they were purchased with current prices.

API

Create endpoints for:

start scraping with a petition

get all products

separate handlers from main file

save wholesalers data

save products with price info and wholesaler

update products, save new price and date

get data from wholesaler web site, all at same time in parallel

gracefully shutdown

search product by name or other data

download csv files from Google Drive saved

get all prices changes through time

automate billing of a business, using ID of sales

Consume info

create front-ends (we will use React only with the propose of test somethings)

for administration

create users

update or add wholesaler

add sales

consulting data

see all products and prices

compare prices of same product at different periods of time

make petition to update data products

show percentage of change prices (selecting one data and compare with the actual price)

Repository

change architecture

folders organization must have a complete change

log error in a better a consistent way

change to logrus library

refactor: manage in a better way logs

refactor: folder data is not right

add comments (in a near future will be hard work, is really bad practice on every side project)

add test, code coverage now is really bad