Coursera_IBM_Data_Engineering

Master SQL, RDBMS, ETL, Data Warehousing, NoSQL, Big Data and Spark with hands-on job-ready skills

In this course, i have the opportunity to immerse myrself in the role of a data engineer and acquire the essential skills you need to work with a range of tools and databases to design, deploy, and manage structured and unstructured data.

After few weeks of training, I completed hands-on labs and projects to help me gain practical experience with Python, SQL, relational databases, NoSQL databases, Apache Spark, building data pipelines, managing databases, and working with data warehouses.

These are the projects that I got to do :

• Design a relational database to help a coffee franchise improve operations

• Use SQL to query census, crime, and school demographic data sets.

• Write a Bash shell script on Linux that backups changed files.

• Set up, test, and optimize a data platform that contains MySQL, PostgreSQL, and IBM Db2 databases.

• Analyze road traffic data to perform ETL and create a pipeline using Airflow and Kafka.

• Design and implement a data warehouse for a solid-waste management company.

• Move, query, and analyze data in MongoDB, Cassandra, and Cloudant NoSQL databases.

• Train a machine learning model by creating an Apache Spark application.

• Design, deploy, and manage an end-to-end data engineering platform.

Skills covered and its notebooks:

1️⃣ Python Project for Data Engineering

1. Extract Transform Load
2. Webscraping Engineer
3. API Engineer
4. Final Assignement Data_Engineer ETL

2️⃣ Introduction to Relational Databases (RDBMS)

Project : Database Design and Implementation

3️⃣ Databases and SQL for Data Science with Python

Advanced SQL Techniques : Use Join / Create and Query Views / Write and run stored procedures

4️⃣ Hands-on Introduction to Linux Commands and Shell Scripting

5️⃣ Relational Database Administration (DBA)

6️⃣ ETL and Data Pipelines with Shell, Airflow and Kafka

7️⃣ Getting Started with Data Warehousing and BI Analytics

8️⃣ Introduction to NoSQL Databases

9️⃣ Introduction to Big Data with Spark and Hadoop

🔟 Data Engineering and Machine Learning using Spark

Data Engineering Capstone Project