This article reviews the process to set up a Kubernetes cluster using docker container runtime with 1 master node and 1 worker node on VMware based RHEL 8 instances.
All the commands listed will be ran against both the master and worker node.
Since this is a lab environment, we will be disabling firewalls. If it is a production environment, you can open specific ports for communication of your applications, and for Kubernetes components instead of disabling the firewall completely. (For a list of the required ports see: https://kubernetes.io/docs/reference/networking/ports-and-protocols/)
Deploy the cluster via Kubeadm then deploy the Flannel networking component.
#Deploy the Kubernetes cluster specifying the cluster network cidr and the container runtime
kubeadm init --pod-network-cidr=10.244.0.0/16 --cri-socket /run/cri-dockerd.sock
#After deploying the cluster you will receive a join command which you will save to run on the worker node.
kubeadm join masternode.bpic.local:6443 --token cll0gw.50jagb64e80uw0da \
--discovery-token-ca-cert-hash sha256:4d699e7f06ce0e7e80b78eadc47453e465358021aee52d956dceed1dfbc0ee34
###On Master Nodes only w/ non root user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=~/.kube/config
Deploy Flannel as the non-root user
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
ON WORKER NODE ONLY
Run the join command to add the node to the cluster.
#Join the node to the cluster
kubeadm join masternode.bpic.local:6443 --token cll0gw.50jagb64e80uw0da \
--discovery-token-ca-cert-hash sha256:4d699e7f06ce0e7e80b78eadc47453e465358021aee52d956dceed1dfbc0ee34
After joining the worker node to the cluster run the following on the master node to confirm cluster status.
kubectl get nodes -o wide
#The result should look similar to the following
NAME STATUS ROLES AGE VERSION INTERNAL-IP OS-IMAGE CONTAINER-RUNTIME
master Ready control-plane 2d5h v1.25.0 192.168.16.73 Red Hat Enterprise Linux 8.7 docker://23.0.4
worker1 Ready <none> 2d1h v1.25.0 192.168.16.153 Red Hat Enterprise Linux 8.7 docker://23.0.4
This blog describes how to incorporate an extensive quantity of current AWS infrastructure into Terraform. This process is relevant to an organization that manually constructed its AWS infrastructure and wants to implement it in Terraform code for improved automation and cost savings. This is a substantial undertaking with a number of benefits.
First, a brief description of TerraCognita. TerraCognita is a tool that helps import existing infrastructure from AWS, Google Private Cloud, AzureRM and VMware vSphere cloud providers as Terraform (v1.1.9) resource/state.1
I discovered through study that Terraform import only permits the import of a single resource at a time. You will need to import resources into one account at a time if you wish to build up your organization in the cloud at this time because there is no possibility to import resources into several accounts at the same time.
Adding TerraCognita will allow you to import a rather large file including a variety of variables. Using a flag, you have the ability to import only particular variables into a program. You can leverage the same strategy to restrict the variables that are generated by the variable.tf file. Installation:
Go Libraries: You can build and install with the latest sources. It uses Go Modules, so GO 1.17+ is required.
terracognita-s3 % terracognita aws --hcl s3 --tfstate terraform.tfstate --aws-default-region us-east-1 -i aws_s3_bucket
We are about to remove all content from "s3", are you sure? Yes/No (Y/N):
y
Starting Terracognita with version v0.8.1
Importing with filters:
Tags: [],
Include: [aws_s3_bucket],
Exclude: [],
Targets: [],
Importing aws_s3_bucket [6/6] Done!
Writing HCL Done!
Writing TFState Done!
saikrishnamadupu@Administrators-MacBook-Pro terracognita-s3 % ls -ltr
total 32
-rw-r--r-- 1 saikrishnamadupu staff 13164 Feb 12 00:07 terraform.tfstate
drwx------ 4 saikrishnamadupu staff 128 Feb 12 00:07 s3
It returns all the buckets info into the folder called s3 where it contains s3_storage.tf file that contains all the buckets list and its configuration.
Post-import manual work:
TerraCognita does not make a versions.tf file on its own, you will need to copy the information from the module file, where it is automatically generated, and paste it into a versions.tf file on your own if one is needed.
TerraCognita does not produce any outputs.tf files of its own. You are responsible for creating them if they are necessary.
This article reviews how to tail logs from multiple pods via Kubernetes and Stern.
Kubernetes (K8) is a scalable container orchestrator. It is fairly lightweight to support IoT appliances and it can also handle huge business systems with hundreds of apps and hosts .
Stern is a tool for the tailing of numerous Kubernetes pods and the numerous containers that make up each pod. To facilitate faster debugging, each result is color coded.
As the query is a regular expression, the pod name can be easily filtered, and the exact id is not required. For instance, for instance omitting the deployment id. When a pod is deleted, it is removed from the tail, and when a new pod is added, it is automatically tailed.
Stern can tail all of the containers in a pod instead of having to do each one manually. You can simply specify the container flag to limit the number of containers displayed. By default, all containers are monitored.
kubectl apply --filename nginx-svc.yaml -n keyva
O/p: service/nginx unchanged
deployment.apps/nginx created
we can validate and verify the svc and pods that being up and running:
kubectl get all -n keyva
NAME READY STATUS RESTARTS AGE
pod/nginx-cd55c47f5-gwtkn 1/1 Running 0 12s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx ClusterIP 10.96.58.31 <none> 80/TCP 88d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 12s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-cd55c47f5 1 1 1 12s
kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-cd55c47f5-gwtkn 1/1 Running 0 30s
KubeCtl has limits:
Using the label selection, it is evident that kubectl can read logs from numerous pods, however this technique has a drawback.
First, the logs from several pods are jumbled together, making it impossible to determine which log line originated from which pod.
Second, tail mode (using –follow (-f)) is not supported.
The reason for this is that –follow streams the API server's logs. You open a connection to the API server per pod, which opens a connection to the associated kubelet to stream logs continually. This does not scale well and results in many incoming and outgoing connections to the API server. As a result, it became a design decision to restrict the number of concurrent connections. Using Stern:
The command is fairly straightforward. Stern retrieves the logs from the given namespace for the specified application. In the case of Stern, you can view not only logs from a single Kubernetes object, such as a deployment or service, but also logs from all related objects. Example:
Stern -n keyva nginx
stern -n keyva nginx
+ nginx-cd55c47f5-86ql5 › nginx
+ nginx-cd55c47f5-bm55t › nginx
+ nginx-cd55c47f5-gwtkn › nginx
nginx-cd55c47f5-gwtkn nginx /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx-cd55c47f5-gwtkn nginx /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx-cd55c47f5-gwtkn nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx-cd55c47f5-gwtkn nginx 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
nginx-cd55c47f5-gwtkn nginx 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
nginx-cd55c47f5-gwtkn nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx-cd55c47f5-gwtkn nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx-cd55c47f5-gwtkn nginx /docker-entrypoint.sh: Configuration complete; ready for start up
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: using the "epoll" event method
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: nginx/1.23.3
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: OS: Linux 5.10.124-linuxkit
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: start worker processes
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: start worker process 35
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: start worker process 36
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: start worker process 37
nginx-cd55c47f5-gwtkn nginx 2023/01/17 10:42:19 [notice] 1#1: start worker process 38
nginx-cd55c47f5-bm55t nginx /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx-cd55c47f5-bm55t nginx /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx-cd55c47f5-bm55t nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx-cd55c47f5-bm55t nginx 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
nginx-cd55c47f5-bm55t nginx 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
nginx-cd55c47f5-bm55t nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx-cd55c47f5-bm55t nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx-cd55c47f5-bm55t nginx /docker-entrypoint.sh: Configuration complete; ready for start up
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: using the "epoll" event method
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: nginx/1.23.3
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
nginx-cd55c47f5-86ql5 nginx /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx-cd55c47f5-86ql5 nginx /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: OS: Linux 5.10.124-linuxkit
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: start worker processes
nginx-cd55c47f5-86ql5 nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: start worker process 36
nginx-cd55c47f5-86ql5 nginx 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: start worker process 37
nginx-cd55c47f5-86ql5 nginx 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
nginx-cd55c47f5-86ql5 nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: start worker process 38
nginx-cd55c47f5-bm55t nginx 2023/01/17 10:47:26 [notice] 1#1: start worker process 39
nginx-cd55c47f5-86ql5 nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
If you want to use stern in Kubernetes Pods, you need to create the following ClusterRole and bind it to ServiceAccount.
Stern facilitates the output of custom log messages. Using the —output flag, you may utilize the following prepared templates:
output
description
defult
Displays the namespace, pod and container, and decorates it with color depending on --color
raw
Only outputs the log message itself, useful when your logs are json and you want to pipe them to jq
json
Marshals the log struct to json. Useful for programatic purposes
It takes a custom template through the —template flag, which is subsequently compiled into a Go template and used for each log message. The following struct is passed to this Go template:
property
type
description
Message
string
The log message itself
NodeName
string
The node name where the pod is scheduled on
Namespace
string
The namespace of the pod
PodName
string
The name of the pod
ContainerName
string
The name of the container
In addition to the built-in functions, the template includes the following functions:
func
arguments
description
json
object
Marshal the object and output it as a json text
color
color.Color, string
Wrap the text in color (.ContainerColor and .PodColor provided)
parseJSON
string
Parse string as JSON
extjson
string
Parse the object as json and output colorized json
ppextjson
string
Parse the object as json and output pretty-print colorized json
Kubernetes can add complexity. Software programmers need logs quickly to fix problems. Set up your CLI with some aliases and get rolling to tail logs from your apps in real-time if you are using Kubernetes and have access to view logs on your Kubernetes cluster.
Reloader is a Kubernetes tool that automatically reloads configuration files in a running container when a change is detected. This can be useful for updating configurations without having to manually restart your application. This blog will walk through the process of setting up Reloader in Kubernetes. It will provide an example of MySQL deployment to enable reloader using annotations and MySQL deployment YAML to watch for changes in secrets.
Create deployment YML for MySQL container using above Secrets, PVC and PV:
kubectl apply -f my-sql.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql
spec:
ports:
- port: 3306
selector:
app: mysql
clusterIP: None
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
annotations:
reloader.stakater.com/auto: "true"
spec:
selector:
matchLabels:
app: mysql
replicas: 3
strategy:
type: Recreate
template:
metadata:
labels:
app: mysql
spec:
containers:
- image: mysql:5.6
name: mysql
env:
# Use secret in real usage
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: demo-secret
key: password
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pv-claim
Once deployed, you can verify the pods status:
Kubectl get pods
kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-5b8d8bd99b-277mf 1/1 Running 0 10s
mysql-5b8d8bd99b-6x5wz 1/1 Running 0 10s
mysql-5b8d8bd99b-srlnb 1/1 Running 0 10s
Describing the pod to view details:
kubectl describe deployment mysql
Name: mysql
Namespace: keyva
CreationTimestamp: Sun, 19 Feb 2023 20:28:02 -0600
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 1
reloader.stakater.com/auto: true
Selector: app=mysql
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: Recreate
MinReadySeconds: 0
Pod Template:
Labels: app=mysql
Containers:
mysql:
Image: mysql:5.6
Port: 3306/TCP
Host Port: 0/TCP
Environment:
MYSQL_ROOT_PASSWORD: <set to the key 'password' in secret 'demo-secret'> Optional: false
Mounts:
/var/lib/mysql from mysql-persistent-storage (rw)
Volumes:
mysql-persistent-storage:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: mysql-pv-claim
ReadOnly: false
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetAvailable
Available True MinimumReplicasAvailable
OldReplicaSets: <none>
NewReplicaSet: mysql-5b8d8bd99b (3/3 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 11m deployment-controller Scaled up replica set mysql-5b8d8bd99b to 3
Next, validate the Reloader by changing the secret values and redeploy the secret.yaml file. You should see the MySQL pods restart to pick reference the new secret value.
Reloader can monitor changes to ConfigMap and Secret, as well as perform rolling upgrades on Pods and their associated DeploymentConfigs, Deployments, Daemonsets Statefulsets, and Rollouts.
Validation of reloader:
kubectl logs reloader-reloader-7f6b8d49f7-9lxrx
time="2023-02-22T01:59:48Z" level=info msg="Changes detected in 'demo-secret' of type 'SECRET' in namespace 'keyva', Updated 'mysql' of type 'Deployment' in namespace 'keyva'"
Verify the age of Pods:
kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-7bd6b6d789-fqfgd 1/1 Running 0 3s
mysql-7bd6b6d789-tbxxp 1/1 Running 0 3s
mysql-7bd6b6d789-trd7v 1/1 Running 0 3s
This article reviews the process to set up a Kubernetes cluster using docker container runtime with 1 master node and 1 worker node on VMware based RHEL 8 instances.
All the commands listed will be ran against both the master and worker node.
Since this is a lab environment, we will be disabling firewalls. If it is a production environment, you can open specific ports for communication of your applications, and for Kubernetes components instead of disabling the firewall completely. (For a list of the required ports see: https://kubernetes.io/docs/reference/networking/ports-and-protocols/)
Deploy the cluster via Kubeadm then deploy the Flannel networking component.
#Deploy the Kubernetes cluster specifying the cluster network cidr and the container runtime
kubeadm init --pod-network-cidr=10.244.0.0/16 --cri-socket /run/cri-dockerd.sock
#After deploying the cluster you will receive a join command which you will save to run on the worker node.
kubeadm join masternode.bpic.local:6443 --token cll0gw.50jagb64e80uw0da \
--discovery-token-ca-cert-hash sha256:4d699e7f06ce0e7e80b78eadc47453e465358021aee52d956dceed1dfbc0ee34
###On Master Nodes only w/ non root user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=~/.kube/config
Deploy Flannel as the non-root user
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
ON WORKER NODE ONLY
Run the join command to add the node to the cluster.
#Join the node to the cluster
kubeadm join masternode.bpic.local:6443 --token cll0gw.50jagb64e80uw0da \
--discovery-token-ca-cert-hash sha256:4d699e7f06ce0e7e80b78eadc47453e465358021aee52d956dceed1dfbc0ee34
After joining the worker node to the cluster run the following on the master node to confirm cluster status.
kubectl get nodes -o wide
#The result should look similar to the following
NAME STATUS ROLES AGE VERSION INTERNAL-IP OS-IMAGE CONTAINER-RUNTIME
master Ready control-plane 2d5h v1.25.0 192.168.16.73 Red Hat Enterprise Linux 8.7 docker://23.0.4
worker1 Ready <none> 2d1h v1.25.0 192.168.16.153 Red Hat Enterprise Linux 8.7 docker://23.0.4
This article reviews the process to set up a Kubernetes cluster using docker container runtime with 1 master node and 1 worker node on VMware based RHEL 8 instances. All ...
By: Saikrishna Madupu – Sr Devops Engineer This blog describes how to incorporate an extensive quantity of current AWS infrastructure into Terraform. This process is relevant to an organization that manually constructed ...
By: Saikrishna Madupu – Sr Devops Engineer This article reviews how to tail logs from multiple pods via Kubernetes and Stern. Kubernetes (K8) is a scalable container orchestrator. It is fairly lightweight ...
By: Saikrishna Madupu – Sr Devops Engineer Reloader is a Kubernetes tool that automatically reloads configuration files in a running container when a change is detected. This can be useful for updating ...
