05-ci.adoc

Continuous Integration with Jenkins

aka "CI"

CI: Why ?

Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove.

— Martin Fowler

CI: What ?

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily, leading to multiple integrations per day.

— Martin Fowler - Continuous Integration

CI: How ?

• Each integration is verified by an automated build (including test)

• Integrate code often, at least daily, to make integration a non-event

• Continuously build and integrate, with a feedback loop

Continuous Integration with Jenkins

CI: Accessing Jenkins

• Access your Jenkins instance:

  • {jenkins-url}

  • Log in as the user butler (password is the same)

  • This is the "Jenkins Classic GUI"

CI: Jenkins BlueOcean

• Switch to BlueOcean, the new UI

  • Direct link to BlueOcean GUI

  • Or click on the top button "Open Blue Ocean"

CI: Our First Pipeline Project

• Create your 1st Pipeline:

  • Stored in Git

  • Fetch URL from the Gitserver

    • Direct link to Git repository

  • Add a User/password credential ({admin-user} / {admin-user})

  • Pipeline is empty (for now): no Jenkinsfile

CI: Fast feedback with Webhooks

• We want Fast feedback !

  • Pushed code to repository ? Tell Jenkins to build it now

• Let’s use Webhook to the repository

  • HTTP request Gitserver → Jenkins

CI: Add a Gogs Webhooks

• From repo. in Gitserver → Settings → Webhooks

  • Direct link to Repository Webhook Settings

• Add a new webhook:

  • Type: Gogs (not Slack)

  • Payload URL: {demoapp-mb-job-build-url}

  • When should this webhook be triggered?: I need everything

CI: Starting with Pipelines

• Pipeline-as-code: We need a Jenkinsfile

• Where to start ?

  • Getting Started with Pipeline

  • Pipeline "Handbook"

  • Pipeline Syntax Reference

  • Pipeline Steps Reference

CI: Declarative or Scripted Pipelines ?

• Declarative

  • Easy syntax

  • Default syntax

  • Start with this one

• Scripted

  • Original syntax (~3 years)

  • "Great Power == Great Responsibility"

  • Use it when Declarative starts to be weird

CI: BlueOcean Pipeline Editor

• Provides the full round trip with SCM

• No Pipeline ? Follow the wizard (not Gandalf, fool !)

• Already have a Pipeline ? Edit, commit, run it

• Needs a compliant SCM

  • Only Github with BO 1.0.1

  • Interested ? Open-Source: Contribute !

CI: Use the Pipeline Editor

• Git is not supported (yet): let’s hack

  • Open the hidden BlueOcean Pipeline Editor: Direct URL

  • Use CTRL + S (On Mac: CMD +S) to switch to/from textual version

• The Pipeline Syntax Snippet Generator is useful:

  • Dynamic generation based on the installed plugins

  • A pipeline job is required: check the left menu icon on {demoapp-mb-job-url}

  • {demoapp-mb-job-url}/pipeline-syntax/

CI: Exercise - Your First Pipeline

• Use the BlueOcean Pipeline Editor and Gitserver

• Create a Pipeline that have a single stage "Hello"

• This stage have 1 step that prints the message "Hello World"

• Copy/Paste this Pipeline in a new file Jenkinsfile on the repository root

• A build will kick off immediately:

  • {demoapp-repo-name} Activity Dashboard

CI: Solution - Your first pipeline

pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        echo 'Hello World !'
      }
    }
  }
}

CI: Exercise - Simple Build Pipeline

• Exercise: Implement a simple build pipeline for demoapp

• We want 4 stages, for the 4 Maven goals:

  • compile, test, package, verify

• We need to build on the maven agent

CI: Solution - Simple Build Pipeline

pipeline {
  agent {
    node {
      label 'maven'
    }
  }
  stages {
    stage('Compile') {
      steps {
        sh 'mvn compile'
      }
    }
    stage('Unit Tests') {
      steps {
        sh 'mvn test'
      }
    }
    stage('Build') {
      steps {
        sh 'mvn package'
      }
    }
    stage('Integration Tests') {
      steps {
        sh 'mvn verify'
      }
    }
  }
}

CI: Exercise - Artifacts

• We want to simplify to 2 stages, based on Unit Tests definition:

  • Build: compile, unit test and package the application

  • Verify: Run Integration Tests

• We also want to archive the generated jar file

  • Only if the build is successful

• Clues: Keywords post + success (not in Editor), and archiveArtifacts

CI: Solution - Artifacts

pipeline {
  agent {
    node {
      label 'maven'
    }
  }
  stages {
    stage('Build') {
      steps {
        sh 'mvn package'
      }
    }
    stage('Verify') {
      steps {
        sh 'mvn verify'
      }
    }
  }
  post {
    success {
      archiveArtifacts 'target/demoapp.jar'
    }
  }
}

CI: Exercise - Integration Tests Reports

• We want the integration test reports to be published to Jenkins

  • Better feedback loop

• If Integration Tests are failing, do NOT fail the build

  • Make it UNSTABLE instead

• Clues:

  • Maven flag -fn ("Fails Never")

  • keyword junit (Pipeline keyword)

CI: Solution - Integration Tests Reports

pipeline {
  agent {
    node {
      label 'maven'
    }
  }
  stages {
    stage('Build') {
      steps {
        sh 'mvn clean compile test package'
      }
    }
    stage('Verify') {
      steps {
        sh 'mvn verify -fn'
        junit '**/target/failsafe-reports/*.xml'
      }
    }
  }
  post {
    success {
      archiveArtifacts 'target/demoapp.jar'
    }
  }
}

CI: Exercise - All Tests Reports

• We now want all test reports published

  • Problem: how to handle Unit test failure ?

• We also want to archive artifacts if build is unstable only due to the Verify stage

• Clues: post can be used per stage

CI: Solution - All Tests Reports

pipeline {
  agent {
    node {
      label 'maven'
    }
  }
  stages {
    stage('Build') {
      steps {
        sh 'mvn clean compile test package'
      }
      post {
        always {
          junit '**/target/surefire-reports/*.xml'
        }
      }
    }
    stage('Verify') {
      steps {
        sh 'mvn verify -fn'
        junit '**/target/failsafe-reports/*.xml'
      }
      post {
        unstable {
          archiveArtifacts 'target/demoapp.jar'
        }
      }
    }
  }
  post {
    success {
      archiveArtifacts 'target/demoapp.jar'
    }
  }
}

CI: Failing Tests

• Validate your changes by making your tests fails.

• Edit each one and uncomment the failing block:

  • Integration: src/master/src/test/java/hello/ApplicationIT.java

  • Unit Tests: src/master/src/test/java/hello/ApplicationTest.java

• Browse the top-level items "Changes", "Tests" and "Artifacts"

• Do NOT forget to correct your tests at the end

That’s all folks !