Energy Demand Forecasting System

Presentation link for the project

Project Documentation

Problem Statement

The energy industry is navigating a transformative phase characterized by rapid modernization and technological advancements. Infrastructure upgrades, integration of intermittent renewable energy sources, and evolving consumer demands are reshaping the sector. However, the progress comes with challenges like volatile supply, demand, and prices, rendering the future less predictable. Traditional business models are also being fundamentally challenged. In this competitive and dynamic landscape, accurate decision-making is pivotal.

Stakeholders in the energy industry heavily rely on probabilistic forecasts to navigate this uncertain future. Therefore, innovative and precise forecasting methods are essential to assist stakeholders in making strategic decisions amidst the shifting energy landscape.

Python Requirements

Installation

To set up the project, follow these steps:

1. Clone the repository:

   git clone https://github.com/zenml-io/zenml-projects.git
   cd zenml-projects/customer-satisfaction

2. Install dependencies:

   pip install -r requirements.txt

3. Launch the ZenML Server and Dashboard:

   pip install zenml["server"]
   zenml up

4. Install MLflow integration for deployment:

   zenml integration install mlflow -y

5. Configure ZenML stack with MLflow components:

   zenml experiment-tracker register mlflow_tracker --flavor=mlflow
   zenml model-deployer register mlflow --flavor=mlflow
   zenml stack register mlflow_stack -a default -o default -d mlflow -e mlflow_tracker --set

Training Pipeline

The training pipeline includes the following steps:

• ingest_data: Ingests and creates a DataFrame from the data.
• clean_data: Cleans the data and removes unwanted columns.
• train_model: Trains the model and saves it using MLflow autologging.
• evaluation: Evaluates the model and saves metrics using MLflow autologging.

Deployment Pipeline

The deployment pipeline extends the training pipeline and implements a continuous deployment workflow. Additional steps include:

• deployment_trigger: Checks whether the newly trained model meets deployment criteria.
• model_deployer: Deploys the model as a service using MLflow (if criteria is met).

The deployment pipeline uses ZenML's MLflow tracking integration for logging hyperparameter values, the trained model, and model evaluation metrics into the local MLflow backend. It launches a local MLflow deployment server to serve the latest model if its accuracy is above a configured threshold.

A Streamlit application is deployed to consume the latest model service asynchronously from the pipeline logic.

Diving into the Code

Run the training pipeline:

python run_pipeline.py

Run the continuous deployment pipeline:

python run_deployment.py

Run the Streamlit application:

streamlit run streamlit_app.py

FAQ

1. Error: No Step found for the name mlflow_deployer:

   Solution: Delete the artifact store and rerun the pipeline. Obtain the artifact store location:

   zenml artifact-store describe

   Delete the artifact store:

   rm -rf PATH

2. Error: No Environment component with name mlflow is currently registered:

   Solution: Install the MLflow integration:

   zenml integration install mlflow -y