In this article we are going to cover Integrate Remote Kubernetes Cluster with Jenkins | How to Connect Kubernetes Cluster with Jenkins,Connect Kubernetes Cluster using Jenkins Pipeline withKubeCredentials
Table of Contents
Step #1:Install Jenkins on Ubuntu 22.04 LTS
How to Install Jenkins on Ubuntu 22.04 LTS
Step #2:Create AWS EKS Cluster
How to Create Amazon EKS cluster using eksctl [2 Steps]
Step #3:Jenkins Plugins for Kubernetes
Below Kubernetes Plugins required to Integrate Remote Kubernetes Cluster with Jenkins
- Kubernetes
- Kubernetes Credentials
- Kubernetes CLI
click on “Manage Jenkins” > “Manage Plugins” > Click on the “Available tab” then search for “Kubernetes“. You will see various plugins listed. Click on the checkboxes for “Kubernetes“, “Kubernetes Credentials“, and “Kubernetes CLI” plugins then click on install
Step #4:Generate secret for Kubernetes service account
Create a kubernetes service account using below command:
kubectl create serviceaccount jenkins
Create a role binding based on the permission needed by the application using below code:
at <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: jenkins-integration
labels:
k8s-app: jenkins-image-builder
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: jenkins
namespace: default
EOF
Step #4:Extract Service account token using kubectl
kubectl get secrets $(kubectl get serviceaccounts jenkins -o jsonpath='{.secrets[].name}') -o jsonpath='{.data.token}' | base64 -d
Default secret no longer being generated for service account with Kubernetes >= v1.24.0
Create a secret for service account jenkins
Solution:
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
name: jenkins
annotations:
kubernetes.io/service-account.name: jenkins
EOF
Then extract the token using below command:
kubectl get secrets jenkins -o jsonpath='{.data.token}' | base64 -d
Step #5:Add Kubernetes service account secret in Jenkins Credentials
Once you have your token generated in the previous step,We will add the token as a Credential as follows
Login to Jenkins, click on “Manage Jenkins” > “ Credentials” > then click on “global” and click on add credentials
Under “Kind“, scroll on the drop-down list and then choose “Secret text“. Under secret, copy the Kubernetes token we generated earlier and paste it there. Then enter ID and description
Credential successfully added.
Step #6:Integrate Remote Kubernetes Cluster with Jenkins
To integrate Remote Kubernetes Cluster with Jenkins follow below steps
Fill in the Kubernetes plugin configuration. In order to do that, you will open the Jenkins UI and navigate to Manage Jenkins -> Manage Nodes and Clouds -> Configure Clouds -> Add a new cloud -> Kubernetes and enter the Kubernetes URL and Jenkins URL appropriately, unless Jenkins is running in Kubernetes in which case the defaults work.
Now click on kubernetes details and fill the details as shown in image
Kubernetes URL: Open your config file and enter server URL
Kubernetes server certificate key: You need to generate this key using this command :
Convert kubernetes server certificate key to base64 format using below command
ls -a
cd .kube
cat config
echo -n <contents_of_the_certificate-authority-data_entry_of_my_kubeconfig_file> | base64 --decode
Example:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJSnNJalJoL3VRVkl3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpBNU1qY3dOVEkyTUROYUZ3MHpNekE1TWpRd05USTJNRE5hTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUUM2a045NmpVSXFHc3lqQ0kvQlk0c1BWK216eTUzVVZnVmdHRWgzM1VrYnN3UjUrSlF6cFIxRU9LY0EKMzAxSnY4OUJFcVJqRjBJaTNKZVFQR2IzWVhMdFdWL1BpZ3pSR0pkVDA5d3F5OVpSaUE2andwTk1DZXUyNUlJSAp0V09lcEVVV1plSHBMRXlZaWNhU0JiNyttM1ZmcnJ6emhoV21JdEZYOG0wVGsvTWlac2czeStvb3lFRGloRDQ5Cll1aVlEUFBkRkZkelljclppUHVGNzV5eE51YVpsNmZoZC9sVzVIUFZqSndlMzkwYVZlVzN3b0pKcWRSS0txUUMKcVZNVGhETjgzcmRPZTBHKzhYcFNJM3hPUXFwajBjTjJXRUo5ZkxObDFFMHl3cyt3QUlZRzJ3TWYreDczMEhaYgpMcVBETjgycTdzNnkyNmZYK1NNSnFIeEtUdHg5QWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJUOEl2TVJ0NTNVWFUxa0I4RENHMTljRlpseUxqQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQWtnbU12c3FpMAppTW84NE1tV2l4OGticjk3dktVem9DMFJRc0kxZU0wM3RvMlVJUEZ1ejNwaDZZYXlrOUFjSXRubks2b3oxRmx6ClVXb0YzbVh5T0tpQ0VIR2hJTVRCYzQ2Wk9XcURVajVzNmZ1Y3NxdnlhQkxnY2VqY0RYL0ZxQkdseUdUSHVKd1EKeVZVd1V5V3dLZUgrazBQQVpjalN5Ni9vK3R4MlRUbVZnbXJjZWZpQi9Ec3lZWmU0dWlsaSs2b1YzRzJTbW1nMgo5UVI1by9aVi9ZMUFZbjkwSmJKcFdMMkh5Mnkzb3JuSlVGZkE3NEhhQ3M1UUovQW9OQnhsbWlTUm90bThncC92Cng0QjhrNmsxT1lVcWhMMWFoRUcrS2FReVV5T3k3dkV1aFViNHFMUStWR3Q2a1NzWFRUK05jSUZZQUJkMkZ6R0UKTE90QzRpNzNEMVZSCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
server: https://D2672844B030086E9221A7D55284F98B.gr7.ap-south-1.eks.amazonaws.com
name: sample.ap-south-1.eksctl.io
contexts:
- context:
cluster: sample.ap-south-1.eksctl.io
user: [email protected]
name: [email protected]
current-context: [email protected]
kind: Config
preferences: {}
users:
- name: [email protected]
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
args:
- eks
- get-token
- --output
- json
- --cluster-name
- sample
- --region
- ap-south-1
command: aws
env:
- name: AWS_STS_REGIONAL_ENDPOINTS
value: regional
provideClusterInfo: falseKubernetes Namespace: Default
Step #7:Use Pipeline Syntax to generate Kubernetes CLI configuration
We are going to use “Pipeline Syntax” tool that comes with Jenkins to generate a configuration we will use to connect to our Kubernetes cluster. First, let us create a new Pipeline, Login to Jenkins, click on “New Item“. Then enter the name of your project and select “Pipeline”
Then scroll down and click on “Pipeline Syntax”
After that you will see this page and here select this option
Then scroll down and click on generate pipeline script then you will see the output like this as shown in below:
withKubeCredentials(kubectlCredentials: [[caCertificate: '', clusterName: 'demo1', contextName: '', credentialsId: 'SECRET_TOKEN', namespace: 'default', serverUrl: '']]) {
// some block
}
Step #8:Connect Kubernetes Cluster using Jenkins Pipeline withKubeCredentials
Now lets create your pipeline using this code
pipeline {
agent any
stages {
stage('Integrate Remote k8s with Jenkins ') {
steps {
withKubeCredentials(kubectlCredentials: [[caCertificate: '', clusterName: 'demo1', contextName: '', credentialsId: 'SECRET_TOKEN', namespace: 'default', serverUrl: 'https://6D042D202083ACA895D7F3C8E30B5942.gr7.ap-south-1.eks.amazonaws.com']]) {
sh 'curl -LO "https://storage.googleapis.com/kubernetes-release/release/v1.20.5/bin/linux/amd64/kubectl"'
sh 'chmod u+x ./kubectl'
sh './kubectl get nodes'
}
}
}
}
}
Output:
Started by user admin
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins
in /var/lib/jenkins/workspace/k8s
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Integrate Remote k8s with Jenkins )
[Pipeline] withKubeCredentials
[kubernetes-cli] creating missing workspace to write temporary kubeconfig
[Pipeline] {
[Pipeline] sh
+ curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.20.5/bin/linux/amd64/kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 38.3M 0 95839 0 0 94046 0 0:07:08 0:00:01 0:07:07 94144
4 38.3M 4 1822k 0 0 925k 0 0:00:42 0:00:01 0:00:41 925k
41 38.3M 41 15.8M 0 0 5435k 0 0:00:07 0:00:02 0:00:05 5434k
82 38.3M 82 31.6M 0 0 8175k 0 0:00:04 0:00:03 0:00:01 8175k
100 38.3M 100 38.3M 0 0 8947k 0 0:00:04 0:00:04 --:--:-- 8948k
[Pipeline] sh
+ chmod u+x ./kubectl
[Pipeline] sh
+ ./kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-192-168-36-241.ap-south-1.compute.internal Ready <none> 60m v1.27.5-eks-43840fb
ip-192-168-6-160.ap-south-1.compute.internal Ready <none> 60m v1.27.5-eks-43840fb
[Pipeline] }
[kubernetes-cli] kubectl configuration cleaned up
[Pipeline] // withKubeCredentials
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Conclusion:
In this article we are going to cover Integrate Remote Kubernetes Cluster with Jenkins | How to Connect Kubernetes Cluster with Jenkins,Connect Kubernetes Cluster using Jenkins Pipeline withKubeCredentials.
