Explore the Readme Folder for an in-depth overview.

End-to-End MLOps Project with ETL Pipelines - Building a Network Security System

Technology Stack Utilized for the MLOps Network Security System Project

Development Environment:

• IDE: Visual Studio Code
• Version Control: GitHub
• Packaging: Python setup.py

Backend Technologies:

• Programming Language: Python
• Database: MongoDB Atlas
• Cloud Platform: AWS (EC2, S3, ECR)

MLOps and Machine Learning Stack:

• ML Framework: TensorFlow or PyTorch (used for model training)
• Experiment Tracking: MLflow
• Remote Experiment Repository: DagsHub
• Hyperparameter Tuning: Scikit-learn or Optuna

Data Engineering:

• ETL Pipeline: Python-based data processing
• Data Validation: Custom validation components
• Data Transformation: Pandas and NumPy

DevOps and Deployment:

• Containerization: Docker
• CI/CD: GitHub Actions
• Deployment: AWS EC2 instance
• Container Registry: AWS ECR

Monitoring and Logging:

• Logging: Custom logging implementation
• Exception Handling: Custom error management

Key Project Components:

• Network security system
• Machine learning model training
• Batch prediction pipeline
• Model artifact management

3.Reference : Krish Naik https://www.udemy.com/course/complete-mlops-bootcamp-with-10-end-to-end-ml-projects/