9 Steps to Setup Kubernetes on AWS using KOPS

InIn this article We are going to perform How to Setup Kubernetes on AWS using KOPS.

What is KOPS in Kubernetes?

Kops (Kubernetes Operations) used to you create, destroy, upgrade and maintain production-grade, highly available, Kubernetes clusters using command line.

Kubernetes KOPS Features

• Automatic creation of Kubernetes clusters in AWS and GCE 
• We can create multi master with HA
  
• Automatic creation of VPC, Security Groups, etc., while creating Cluster
• Support Public and Private DNS
• Runs in Auto scaling
  
• We can Add and Edit the Cluster configuration such as Master and worker nodes

Below are prerequisites to setup kubernetes on aws using kops.

1. AWS account 
2. Create a Domain to Access Kubernetes API
   
3. Create a Hosted Zone in Route53 and point AWS nameserver to Domain
   
4. Create a S3 bucket with Versioning Enabled to store Kubernetes Kops cluster state
5. IAM user with full S3, EC2, Route53 and VPC access 
6. Ubuntu 20.04/18.04/16.04 LTS with minimal installation
   
7. AWS-CLI
   
   

Assuming you have created Ubuntu Instance in your AWS account.

Step #1: Install AWS CLI on Ubuntu

To download the latest AWS CLI version for 64 bit Linux using curl use below command

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

if you want download specific version of AWS CLI version then use below command

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-awscliversion.zip" -o "awscliv2.zip"

To download AWS CLI Version 2.0.30 use below command

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-2.0.30.zip" -o "awscliv2.zip"

Install the unzip on Ubuntu if not installed

sudo apt install unzip

Extract AWS CLI Setup on Linux using unzip

unzip awscliv2.zip

Install AWS CLI on Linux

once extracted Install AWS CLI on Linux using below command

sudo ./aws/install

Output:

sudo ./aws/install
You can now run: /usr/local/bin/aws --version

Check the AWS CLI version using command line on Linux

/usr/local/bin/aws --version

Output:

aws-cli/2.1.28 Python/3.8.8 Linux/4.14.214-160.339.amzn2.x86_64 exe/x86_64.amzn.2 prompt/off

OR

aws --version

Output:

aws-cli/2.1.28 Python/3.8.8 Linux/4.14.214-160.339.amzn2.x86_64 exe/x86_64.amzn.2 prompt/off

Step #2: Install Kubectl Binary with CURL on Ubuntu

Download kubectl binary with curl on Ubuntu using below command

sudo curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

Make the kubectl binary executable

sudo chmod +x ./kubectl 

Move kubectl to /usr/local/bin/kubectl directory

sudo mv ./kubectl /usr/local/bin/kubectl 

Step #3: Install KOPS on Ubuntu Instance

Download the KOPS  setup on Ubuntu using curl

curl -LO https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-linux-amd64

set the execution permission

sudo chmod +x kops-linux-amd64

move the kops to /usr/local/bin directory

sudo mv kops-linux-amd64 /usr/local/bin/kops 

Step #4: Creating Domain and Hosted Zone in AWS

Kubernetes kops need DNS to discover within cluster and to connect Kubernetes API Server from Clients

You can create either public or private hosted zone, if you created public hosted zone you have to point AWS nameservers in Domain.

Here We have domain in GoDaddy, created public hosted zone in AWS and pointed AWS nameservers in GoDaddy Domain

We have created domain k8s.fosstechnix.com,

Step #5: Create and configure IAM User in AWS

Create IAM user in AWS using login console -> IAM -> ADD User -> Username -> Select Programmatic access -> then you will get Access Key ID and Secret Access Key.

Configure AWS CLI with your Access Key ID,  Secret Access  key and region

aws configure

Step #6: Create IAM User with full S3, EC2, Route53 and VPC access in AWS

Now create IAM user with full S3, EC2, Route 53 and VPC access in your AWS account as shown below

1. AmazonS3FullAccess 
2. AmazonEC2FullAccess 
3. AmazonRoute53FullAccess 
4. IAMFullAccess 
5. AmazonVPCFullAccess 

Step #7: Creating S3 Bucket using command line

Create the S3 bucket to store Kubernetes cluster states

aws s3 mb s3://k8s.fosstechnix.info

Enable versioning on S3 bucket

aws s3api put-bucket-versioning --bucket k8s.fosstechnix.info --versioning-configuration Status=Enabled

Export kops state

export KOPS_STATE_STORE=s3://k8s.fosstechnix.info

Step #8: Create SSH Keys

Create ssh keys on Ubuntu instance to exchange kubernetes cluster and connect

ssh-keygen 

Step #9: Setup Kubernetes on AWS using KOPS

Create Kubernetes  on AWS using Kops using below command

 kops create cluster --cloud=aws --zones=ap-south-1a --name=k8s.fosstechnix.info --dns-zone=k8s.fosstechnix.info --dns public

if you want to install calico networking along with setup of Kubernetes KOPS add –networking with calico as shown below

kops create cluster --cloud=aws --zones=ap-south-1a --networking calico --name=k8s.fosstechnix.info --dns-zone=k8s.fosstechnix.info --dns public

If you want to create K8s cluster using multimaster and different availability zone use below command

kops create cluster --cloud=aws --zones=ap-south-1a,ap-south-1b --networking calico --master-size t3.medium --master-count 3 --node-size t3.xlarge --node-count 3 --name=k8s.fosstechnix.info --dns-zone=k8s.fosstechnix.info --dns public 

Sample Output:

Must specify --yes to apply changes

Cluster configuration has been created.

Suggestions:
* list clusters with: kops get cluster
* edit this cluster with: kops edit cluster k8s.fosstechnix.info
* edit your node instance group: kops edit ig --name=k8s.fosstechnix.info nodes-us-east-1a
* edit your master instance group: kops edit ig --name=k8s.fosstechnix.info master-ap-south-1a

Finally configure your cluster with: kops update cluster --name k8s.fosstechnix.info --yes --admin

Configure the k8s kops cluster using below command,

kops update cluster k8s.fosstechnix.info --yes --admin

Sample Output:

Cluster is starting.  It should be ready in a few minutes. 

 Suggestions: 

 * validate cluster: kops validate cluster 

 * list nodes: kubectl get nodes --show-labels 

 * ssh to the master: ssh -i ~/.ssh/id_rsa admin@api.k8s.fosstechnix.info

 * the admin user is specific to Debian. If not using Debian please use the appropriate user based on your OS. 

 * read about installing addons at: https://github.com/kubernetes/kops/blob/master/docs/operations/addons.md

Validate the Kubernetes KOPS cluster

kops validate cluster 

To list the nodes

kubectl get nodes  

To get Cluster Information

kubectl cluster-info 

To delete Kubernetes KOPS cluster

kops delete cluster k8s.fosstechnix.info --yes 

Finally We have covered Setup Kubernetes on AWS using KOPS

How to add or change node size/type in Kops Cluster

kops get ig

Output:

Using cluster from kubectl context: k8s.fosstechnix.info

NAME ROLE MACHINETYPE MIN MAX ZONES
master-ap-south-1a Master t3.medium 1 1 ap-south-1a
nodes-ap-south-1a Node t3.medium 1 1 ap-south-1a

kops edit ig master-ap-south-1a

Output:

apiVersion: kops.k8s.io/v1alpha2
kind: InstanceGroup
metadata:
creationTimestamp: "2021-03-06T05:29:52Z"
labels:
kops.k8s.io/cluster: k8s.fosstechnix.info
name: master-ap-south-1a
spec:
image: 099720109477/ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20210119.1
machineType: t3.medium
maxSize: 1
minSize: 1
nodeLabels:
kops.k8s.io/instancegroup: master-ap-south-1a
role: Master
subnets:
- ap-south-1a

Troubleshooting:

kubectl get nodes

Error:

You must be logged in to the server (Unauthorized) 

if you are getting above error on Kubernetes kops cluster

Solution:

Login to Kubernetes kops cluster Master node from management node

ssh -i ~/.ssh/id_rsa [email protected]

Navigate to .kube directory

cd .kube

copy the client-certificate-data and client-key-data values, exit from master node

Replace above values to management node /.kube/config file

Now you can run the kubectl commands

To delete Kubernetes KOPS cluster use below command

Error: exactly one ‘admin’ SSH public key can be specified when running with AWS; please delete a key using `kops delete secret`

Solution:

To check list of secrets in Kubernetes Kops cluster

kops get secret

Output:

TYPE            NAME                            ID

Keypair         apiserver-aggregator-ca         6933429814222041362322666137

Keypair         ca                              6933429813663588303876984541

Keypair         etcd-clients-ca                 6933429813788008634835356814

Keypair         etcd-manager-ca-events          6933429814919214502125133873

Keypair         etcd-manager-ca-main            6933429813961434714756350017

Keypair         etcd-peers-ca-events            6933429814048350100913985823

Keypair         etcd-peers-ca-main              6933429814135513819119005976

Keypair         master                          6933429813874453479985214673

SSHPublicKey    admin                           01:98:16:b4:a0:44:99:6c:a8:f7:6c:3b:a2:43:0b:c2

SSHPublicKey    admin                           6e:14:33:11:9c:4a:0c:0a:70:68:53:c1:b1:7e:cb:bd

SSHPublicKey    admin                           78:6a:e1:00:74:e6:ee:95:fa:32:fc:ae:2a:d7:63:68

Secret          admin

Secret          kube

Secret          kube-proxy

Secret          kubelet

Secret          system:controller_manager

Secret          system:dns

Secret          system:logging

Secret          system:monitoring

Secret          system:scheduler

Delete the duplicate secret from kops cluster using below command

kops delete secret SSHPublicKey admin 78:6a:e1:00:74:e6:ee:95:fa:32:fc:ae:2a:d7:63:68

Creating Kubernetes KOPS Cluster in Existing VPC

To create Kubernetes cluster in AWS using KOPS in Existing VPC

Export the s3 bucket to store kops states

export KOPS_STATE_STORE=s3://k8s.fosstechnix.info

Export the cluster name

export CLUSTER_NAME=k8s.fosstechnix.info

Export the existing AWS VPC

export VPC_ID=VPC_ID

Export the VPC CIDR range

export NETWORK_CIDR=172.20.0.0/16

Now create Kubernetes Kops cluster

kops create cluster --zones=ap-south-1a,ap-south-1b --name=${CLUSTER_NAME} --vpc=${VPC_ID}

To create a secret on Kubernetes KOPS cluster

kops create secret --name k8s.fosstechnix.info sshpublickey admin -i ~/.ssh/id_rsa.pub

Login to kops master node

ssh -i ~/.ssh/id_rsa [email protected]

Conclusion:

We have covered How to Setup Kubernetes on AWS using KOPS, install kops, install kubectl, creating S3 bucket, genrate ssh keygen.

Related Articles:

• How to Setup Kubernetes Dashboard
• How To Setup Kubernetes Cluster Using Kubeadm on Ubuntu 18.04/16.04 LTS
• Deploy to Kubernetes using Helm and GitLab
• How to Install AWS CLI on Windows
• How to Install Minikube on Ubuntu

Reference:

Kubernetes Official Guide