The purpose of this project is to build a Web App that will scrape several websites for the most recent Mars data. The extracted data is stored in a NoSQL database and then an HTML page is created to display the findings.
- Python 3.7
- splinter 0.14.0
- webdriver-manager 3.3.0
- Flask 1.1.2
- Flask-PyMongo 2.3.0
- BeautifulSoup (bs4) 0.0.1
- html5lib 1.1
- lxml 4.6.3
An example image of the HTML page can be viewed by clicking here.
Selecting the "Scrape New Data" button will obtain the latest news, images, and facts about Mars. News titles and summaries are extracted from NASA Mars Exploration Program News. The featured images are extracted from the Jet Propulsion Laboratory's Space Images. Mars hemisphere images are extracted from Astropedia. Finally, the Mars facts are gathered from Galaxy Facts. The scraping code used in this project is scraping.py.
After running app.py, the extracted data is successfully stored in MongoDB as seen in the screenshot below. A mars_app database must exist in mongo for the code to properly run.
To confirm that the mars_app database exists, run the following command in mongo:
> show dbs
Adding Bootstrap 3 components, such as the code below, allowed the four Mars hemisphere images to be displayed side-by-side on Desktop browsers, instead of a line. This allows users to see all four images at once.
<div class="col-md-3">
The Web App is also fully optimized for mobile devices, including the layout of the hemisphere images.
Author: Michael Mishkanian
For all questions and inquiries, please contact me on LinkedIn.
