Skip to content

Latest commit

 

History

History
85 lines (67 loc) · 7.11 KB

README.md

File metadata and controls

85 lines (67 loc) · 7.11 KB

GitHub Actions Workflow

In this section, we detail the GitHub Actions workflows employed in our project. GitHub Actions are a powerful tool for automating software workflows, enabling continuous integration and continuous deployment (CI/CD) practices. Our project leverages these workflows to automate various tasks such as code quality checks, automated testing, and deployment processes. Each YAML file described below represents a specific workflow designed for a particular aspect of our software development process. These workflows ensure that our codebase is rigorously tested, adheres to quality standards, and is consistently deployed to our production environment with minimal manual intervention.

pylint.yml

This file configures a GitHub Actions workflow to perform static code analysis using Pylint on Python code. It triggers on pushes to the stage branch and runs on an ubuntu-latest virtual machine. The workflow supports Python versions 3.8, 3.9, and 3.10.

The workflow includes the following steps:

  • Code Checkout: Uses actions/checkout@v3 to fetch the source code.
  • Python Setup: Sets up the Python environment using actions/setup-python@v3.
  • Dependencies Installation: Installs Pylint.
  • Code Analysis with Pylint: Executes Pylint on Python files in the src directory, excluding specific errors and limitations, and generates a textual report.
  • Report Upload: Uploads the Pylint report as a GitHub Actions artifact, available for review and download.

This file is crucial for ensuring that the Python code in the repository adheres to the quality and style standards encoded in Pylint's rules.

pytest.yml

This file sets up a GitHub Actions workflow for running automated tests using Pytest on Python code. It is activated by pushes to the stage branch, specifically targeting changes in the src, tests directories, but excluding the tests/model_testing directory.

Key features of this workflow:

  • Trigger Conditions: It runs when there are changes in the src and tests directories (excluding tests/model_testing) upon push to stage.
  • Environment Setup: The workflow runs on an ubuntu-latest virtual machine.
  • Workflow Steps:
    • Repository Checkout: Uses actions/checkout@v4 for fetching the latest code.
    • Python Environment Setup: Utilizes actions/setup-python@v4 with Python version 3.11.
    • DVC (Data Version Control) Setup: Implements iterative/setup-dvc@v1 for data and model version control.
    • Installation of Requirements: Upgrades pip and installs dependencies from requirements.txt.
    • Data and Model Preparation: Uses DVC to pull training data and model files, and runs preprocessing steps.
    • Testing Phases: Executes multiple Pytest commands to test dataset, preprocessing functions, model behavior, and APIs.
    • Commented-Out Test for Model Training: There's a section for model training tests, currently commented out.

This workflow ensures that the codebase remains stable and functional with each new push, covering a wide range of tests from dataset integrity to API functionality.

azure-static-web-apps-nice-island-02cd56d03.yml

This file outlines a CI/CD workflow for Azure Static Web Apps, designed to automate the deployment of a frontend application hosted on Azure. It triggers on push events to the main branch and on specific actions (opened, synchronize, reopened, closed) of pull requests, again targeting the main branch.

Key aspects of this workflow:

  • Triggering Conditions: It's set to run on push events to the main branch, specifically for changes in the frontend/** directory, and on pull request activities concerning the main branch.
  • Environment and Jobs:
    • Build and Deploy Job:
      • Condition: Executes if it's a push event or an open/synchronize/reopen action in a pull request (not on pull request closure).
      • Platform: Runs on ubuntu-latest virtual machine.
      • Steps:
        • Code Checkout: Uses actions/checkout@v3 with submodule and Large File Storage (LFS) settings.
        • Build and Deploy Action: Utilizes Azure/static-web-apps-deploy@v1 for deployment, configured with secrets for Azure and GitHub tokens, and specifies the locations for app source (/frontend/), optional API source, and output directory (dist).
    • Close Pull Request Job:
      • Condition: Only runs if the event is a pull request closure.
      • Platform: Also runs on ubuntu-latest virtual machine.
      • Steps:
        • Close Pull Request Action: Carries out the closure of the pull request using the same Azure deployment action.

This workflow is crucial for maintaining a streamlined and automated deployment pipeline for the frontend application, ensuring that each update is efficiently built and deployed to Azure Static Web Apps.

main_ITdisambiguation.yml

This YAML file outlines a GitHub Actions workflow for building and deploying a Docker container app to an Azure Web App named ITdisambiguation. The workflow is triggered on pushes to the main branch, specifically focusing on changes within the src/** directory, and can also be manually triggered via workflow_dispatch.

Key elements of this workflow:

  • Trigger Conditions: Activates on pushes to the main branch (for src/** changes) and allows manual triggers.
  • Environment: Both build and deploy jobs run on ubuntu-latest virtual machine.
  • Jobs:
    • Build Job:
      • Steps:
        • Code Checkout: Uses actions/checkout@v2.
        • Docker Buildx Setup: Prepares Docker Buildx environment using docker/setup-buildx-action@v2.
        • Docker Registry Login: Logs in to Docker registry with credentials stored in GitHub secrets.
        • Python Environment Setup: Configures Python 3.11 environment.
        • DVC Setup: Sets up Data Version Control (DVC) for data and model management.
        • Data and Model Preparation: Pulls the model weights using DVC.
        • Docker Image Build: Builds the Docker image with the tag based on the commit SHA and pushes it to Docker Hub.
    • Deploy Job:
      • Dependency: Depends on the successful completion of the build job.
      • Environment Info: Specifies production environment and retrieves the web app URL.
      • Deployment Steps:
        • Azure Web App Deployment: Deploys the Docker image to the Azure Web App (ITdisambiguation) using azure/webapps-deploy@v2, with the necessary configuration details and publish profile provided via GitHub secrets.

This workflow plays a critical role in automating the continuous integration and deployment process, ensuring a streamlined deployment of the latest version of the app to Azure Web App.

Training action

Initially we wanted to make a GitHub Action also to retrain the model whenever code or data changed, but in the end we decided to avoid implementing it. This choice was due to the fact that GitHub's virtual machines running the actions are not GPU-accelerated, this would make our training so slow that would exceed the maximum job execution time (6 hours).