Jenkins on Kubernetes using Docker Desktop

Works on Apple Silicon

• Setup
• Deploy Kubernetes Manifests
• Configure Jenkins
• Configure Cloud
• Configure Github connectivity
• Configure Seed Job
• Create a Jenkins Pipeline
• Data Persistence

Setup

1. Create a Docker context

docker context create docker-jenkins \
  --default-stack-orchestrator=kubernetes \
  --kubernetes config-file=/Users/x/.kube/config \
  --docker host=unix:///var/run/docker.sock

> 'Successfully created context "docker-jenkins"'

Make sure that /Users/x/.kube/config exists

2. Change to the new context

docker context use docker-jenkins

3. Export the Kubernetes context

docker context export docker-jenkins --kubeconfig

> 'Written file "docker-jenkins.kubeconfig"'

4. Copy the content of docker-jenkins.kubeconfig and add it to the ~/.kube/config file.

Deploy Kubernetes Manifests

1. Create a Kubernetes namespace

kubectl create namespace jenkins

> 'namespace/jenkins created'

2. Use the new namespace

kubens jenkins

> 'Active namespace is "jenkins"'

3. Apply the Kubernetes manifests

kubectl apply -f ./kubernetes

> 'deployment.apps/jenkins created'
> 'persistentvolume/jenkins created'
> 'persistentvolumeclaim/jenkins-claim created'
> 'serviceaccount/jenkins created'
> 'clusterrole.rbac.authorization.k8s.io/jenkins created'
> 'rolebinding.rbac.authorization.k8s.io/jenkins created'
> 'service/jenkins created'

Configure Jenkins

Jenkins is accessible at:

http://localhost:<service-port>

Get the service port:

kubectl get service

NAME      TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                                       AGE
jenkins   NodePort   10.104.192.221   <none>        8080:32039/TCP,50000:31020/TCP,80:32317/TCP   18m

Navigate to:

http://localhost:32039

The initial Jenkins admin user password is outputted in the pod logs.
It may also be found at: /var/jenkins_home/secrets/initialAdminPassword

Configure Cloud

Install the Kubernetes Plugin and restart Jenkins. The running docker container (jenkins/jenkins) will automatically exit and restart.

1. Get cluster info

kubectl cluster-info

> 'Kubernetes control plane is running at https://kubernetes.docker.internal:6443'
> 'CoreDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy' 

2. Get the Jenkins IP from the pod:

kubectl describe pod <pod-name> 

Jenkins is running on http://10.1.0.99:8080 inside the cluster. The pod IP is used to configure clouds (kubernetes connectivity) on Jenkins.

1. Navigate to Configure Clouds on Jenkins and add the ~/.kube/config as a credential (secret file)

2. Configure Cloud

Configure Github Connectivity

1. Navigate to Github Developer Settings and generate a new Personel access token.

2. Create a new Jenkins credential with Github username and Personel access token.

Configure Seed Job

The seed job is a normal Jenkins job that run the job DSL script (jobs/jobs.groovy); in turn, the script contains instructions that create additional jobs. In short, the seed job is a job that creates more jobs.

The Job-DSL plugin is required.

1. Create a Jenkins job and configure the Source Code Management section.

2. Configure the Build step (process Job DSLs) to use the jobs/jobs.groovy DSL script.

⚠️ Save and build, but the job fails because you need to approve the DSL script.

3. Navigate to Manage Jenkins -> In-process Script Approval and Approve the DSL script.

Run the seed job again and it should build successfully. Any pipeline definitions in the jobs/jobs.groovy DSL script will also be created.

Create a Jenkins Pipeline

Creata a sample Jenkins pipeline using the Jenkinsfile.sample jenkinsfile. and build the job. A new "builder" pod will be created in the cluster which contains a docker container and a jenkins/inbound-agent container. The builder pod will be terminated once the pipeline has run.

The builder Pod running:

The two containers inside the builder pod:

Data Persistence

The Jenkins data is stored using a PersistenceVolume (PV), which is a piece of storage in the cluster that has been provisioned using a storage class. It is a resource in the cluster just like a node is a cluster resource. A PersistenceVolumeClaim (PVC) is used to consume the PV resources. Claims can request a specific size and access modes (e.g., they can be mounted ReadWriteOnce, ReadOnlyMany or ReadWriteManny).

The Jenkins data is accessible inside the Docker container - in the /var/jenkins_home directory.