Skip to content

Latest commit

 

History

History
221 lines (148 loc) · 4.21 KB

dev.md

File metadata and controls

221 lines (148 loc) · 4.21 KB

Amalgam

Note: These are the instructions for developer

Amalgam is collection of (mainly) Python scripts with a web interface (based on Flask) you can use in SEO.

Table of content

Project structure

  • [lab] - only used by developers to test new ideas
  • [pieces] - folder with other folders and sub-projects
  • [amalgam] - folder with sources
    • [crawler] - crawler
    • [models] - DB model
    • [static] - static files: css, js, images
    • [templates] - Jinja pages
    • [tests] - tests
  • [documentation] - folder with different documentation
    • [design] - things related to design
    • *.sql - SQLite SQL files
    • amalgam_mysql.sql - SQL script to create an Amalgam DB
  • app.py - the actual starting point of the application
  • manage_db.py - manage current database (SQLite | MySQL )
  • DEV.md - this file
  • requirements.txt - the Python dependencies for the project (used by pip3)

How to run tests

python -m amalgam.tests.test_all

How to run only crawler

While being inside the [amalgam] folder run:

python -m amalgam.crawler.crawler

How to play with database

While being inside the [amalgam] folder run:

# Create tables inside DB 
python manage_db.py -a empty

# Create tables inside DB  and add some data
python manage_db.py -a mock

How to run it

How to run it (Docker)

Step 1: Install Docker

Go on Docker's download page and install Docker

Step 2: Run the app

docker run -it -p 5000:5000 scriptoid/amalgam:0.1

Note 1 : Port 5000 of your PC should be available

Note 2 : If you want to run it on a different port use the following command:

docker run -it -p <Your Port>:5000 scriptoid/amalgam:0.1

replacing 'Your Port' with a free port at your desire.

Step 3: Use the application

Just open a browser and access http://localhost:5000

Step 4: Stop the application

Simply go to console application launched by Docker and press Ctrl-C.

How to run it - XUbuntu 20

Fix broken installs

sudo apt-get --fix-broken install

Install pip3

apt install python3-pip

Install virtualenv

pip3 install virtualenv

See that the virtualenv is in the path.

The executable are usually stored inside (/home/alex/.local/bin).

So you need to check if .profile contains

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

and if it does either logout / login or run

source .profile

Install virtualenvwrapper

	pip3 install virtualenvwrapper

Create folder to keep your virtual enviroments

	mkdir ~/.virtualenvs

Edit ~/.bashrc and add

	# Things for virtualenvwrapper
	export WORKON_HOME=$HOME/.virtualenvs
	VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
	. /"$HOME"/.local/bin/virtualenvwrapper.sh

Load .bashrc

	source ~/.bashrc

Setup virtual env

	virtualenv env

Make a virtual enviroment

	mkvirtualenv amalgam

Activate enviroment

	workon amalgam

For visualization install GraphViz

	sudo apt install graphviz

Install Python libraries

	pip3 install -r requirements.txt
	
	See https://github.com/realpython/discover-flask/blob/master/requirements.txt

Note: If you got an error like:

 " fatal error: libpq-fe.h: No such file or directory"

simply run:

sudo apt-get install libpq-dev

Run Flask

	python app.py

See in browser

Access it at: http://127.0.0.1:5000/

Visual Studion Code

* Load workspace
* Ctrl-Shift-P to select the Python interpreter (and pick amalgam). More: https://code.visualstudio.com/docs/python/environments

How to run it (Windows 10)