Getting started with Kubernetes using Rancher and VMware vSphere

By Brad Johnson, Lead DevOps Engineer

In this tutorial we are going to get Rancher set up for testing and development use. Rancher is fully open-source and allows us to easily deploy a Kubernetes cluster in VMware with only minimal configuration. The intent of this tutorial is to give you a base for a scalable development cluster where you can test deploying applications or configuring other Kubernetes software without setting up DNS or external load balancers.

We will use VMware vSphere 6.7 for our deployment. For the OS and software versions we are going to use the ones recommended by Rancher support. As of May 2020, Docker currently has an issue with cluster DNS and firewalld interfering with each other in CentOS/RHEL 8, so we will be using CentOS 7 and Docker 19.03.x for our management server, however you can use any supported OS. For the Master and Worker nodes we will be using RancherOS or CentOS. Using RancherOS eliminates the need to build a custom VM template in vSphere that uses cloud-init.

Requirements for this exercise:
– Admin access to vSphere or a service account with access.
– Ability to create RHEL/CentOS 7 VMs in vSphere.
– Guest VM network has internet access.

In this deployment Rancher has two primary components, the Rancher cluster manager and the Kubernetes cluster we will manage. For production use, the cluster management component would be a container deployed on its own kubernetes cluster. For ease of install and use in a testing and lab deployment, we can simply deploy the management application as a Docker container on a single server. This configuration is not recommended for production and cannot be converted into a production scenario later. If you want a single node cluster manager that can be converted into production ready setup, then you can deploy the management container on a one node kubernetes cluster which could then later be scaled up.

Rancher management server deployment

All commands run as root or with sudo unless noted:

Spin up a standard or minimal CentOS/RHEL 7 server, 2 CPU, 4GB RAM. I used a 100GB thin provisioned primary disk.

Install docker using the Rancher script. Alternatively, install by hand using documentation from docker.

curl https://releases.rancher.com/install-docker/19.03.sh | sh

Create a directory for persistent Rancher data storage

mkdir /opt/rancher

Run Rancher container with persistent data mount listening on port 80/443. This uses a docker self signed cert for SSL.

docker run -d --restart=unless-stopped -p 80:80 -p 443:443 -v /opt/rancher:/var/lib/rancher rancher/rancher:latest

Log in to the rancher web interface using your web browser. The first login will prompt you to set the password for the admin user. Set a password and you should see the main management user interface.

Optional – Creating a CentOS 7 node template for cluster nodes that includes cloud-init.
Cloud-init will allow you to specify additional configuration in Rancher that happen when Rancher creates new nodes, like firewall settings.

1. 1. Boot a new VM with a CentOS iso attached and install the OS manually
   2. Customize disk layout as needed
   3. Leave the system as DHCP
   4. Set a default root password
   5. Make any changed needed by your org
   6. After booting the system, clean things up so you can turn it into a VM. We have created a script for this, please edit as needed. This sets selinux to permissive as Rancher may have issues with the dns service in enforcing mode without additional configuration. The last command in this script will shut down the VM

https://raw.githubusercontent.com/keyvatech/blog_files/master/centos7_cloudinit_vmtemplate.sh

In vCenter find the VM, right-click on it, then select Clone > Clone To Template.
This template can now be used in Rancher with cloud-init for additional provisioning.

Now we can create your new Rancher cluster. Note that the Rancher coredns workload will not with with selinux set to enforcing. If you require enforcing mode you will need additional configuration. It is also important to use consistent DNS names when deploying, FQDNs are best, but do not mix short and full hostnames as it causes certificate issues. Rancher will generate self signed certs if you do not provide your own.

1) From the main web interface cluster page click add cluster, then select vSphere

2) Enter a cluster name like “rancher1”

3) Create a node template for your nodes. This can be used for both master and worker nodes.

1. 1. Click “Add Node Template”
   2. Fill out the Account Access section with your vSphere login info. If the credentials worked you will see the scheduling section populate. If it failed, you can add a new credential with a new name, then delete the ones that didn’t work later by clicking on the user profile picture and selecting “cloud credentials”.
   3. Fill in the scheduling information for your data center, resource pool, data store and folder.
   4. Edit the instance options and specify 2 CPUs and 4096MB RAM or more.
   5. Under Creation Method select either “Install from Boot2Docker ISO (legacy)” or the CentOS 7 node template if you made one.
   6. If you are using a CentOS template with cloud-init fill in the Cloud Config YAML section. We have created the following config which handles firewall config. You can extend this as needed or modify it and create a different template for each node type if desired.

      https://raw.githubusercontent.com/keyvatech/blog_files/master/rancher-centos7-cloud-init-config.txt

   7. Select a Network to deploy to.
   8. Review the remaining settings and adjust if you need them in your environment.
   9. Name the template at the bottom of the page. The template can likely be used for multiple types if desired so keep the name generic. I prefer to use names that indicate node OS and resources like “centos7-2CPU-4GB”
   10. Click create.

4) Enter the name prefix for your master and worked nodes. For example, “rancher1-master” and “rancher1-worker”, when nodes are created a number will be appended to the end.

5) For the master node select the etcd and control plane checkboxes

6) For the worker node select the worker checkbox.

7) Click Create at the bottom of the page. Rancher will now provision your nodes in vCenter.

You should now have a basic functional Kubernetes cluster.

If you are interested in deploying Windows worker nodes with Rancher please see our post here.

Helpful links: 

https://rancher.com/support-maintenance-terms/#2.4.x

https://rancher.com/docs/rancher/v2.x/en/installation/requirements/

https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/creating-credentials/

If you have any questions about the steps documented here, or have any feedback or requests, please let us know at [email protected].