Skip to main content

What is Kubernetes?

  • Author:
  • Updated date:

I specialize in developing digital content for B2B, IT, and law firms that enables them in generating more leads and driving revenue forward

what-is-kubernetes

Kubernetes (k8s) is an open-source container cluster manager for automating deployment, scaling, and management of containerized applications at scale. It has its origin in the Google's Borg project, which is used internally by Google. Kubernetes was announced to be donated to Cloud Native Computing Foundation (CNCF) on Nov 2014.

Summary of Kubernetes Origins:

Kubernetes is primarily used to manage Docker containers on Linux (Ubuntu, RHEL/CentOS). The first release was on 1.4.2 in Oct-2015, and since then, there have been 9 stable releases out. As of today, Kubernetes 1.9 is the latest stable release and it was released on July 12-2017.

The Kubernetes master manages compute resources and provides API endpoints for managing containers. The master knows where the kubelets are running, which pods exist, what state they're in, etc. The kubelet is a system service that runs on each node (Ubuntu/CentOS) in the cluster and reports to the master about its health status. It also watches pod events and reports back to the master about what has happened. Internally, it pulls images from docker hub (or any other registry), creates containers starting with an init process (for supervising all processes within one container) based on those images. Containerized applications can be provided either as Docker image or its tarball by deploying objects called pods, deployments, replication controllers. etc.

The Kubernetes API server is an API proxy that runs on the master node. It's responsible for routing requests to the pod/service/replication controller services that are being run by the cluster, then it routes the responses back to the client respectively. The kube-proxy module runs on each node (Ubuntu/CentOS) and watches all network traffic through iptables rules. When it sees a new connection to any of pods, it forwards them accordingly to corresponding containers using Linux netfilter system calls. For example, if you have two web servers running in separate containers, kubelet will create kube-proxy rules so that connections for port 80 are routed correctly between those two containers.

There are kube-apiserver, kube-controller-manager, kube-scheduler, docker, etc., and flannel components which help in building a fully operational cluster at scale.

Note: Master node should have at least 3 GB RAM available to it for an efficient functioning of the cluster. Also, the machine where master is running should have SSH access so that nodes may be added/removed from the cluster if required.

Kubernetes is written in Go language keeping extensibility & speed as its primary goals, it provides rich resource capabilities based on custom resources that can be deployed using existing kubectl command.

Kubernetes is being widely adopted by the enterprise users; it has a healthy community with an active GitHub project. It's also being used by developers for container management via tools like OpenShift. There are many hosting providers available to deploy Kubernetes on cloud, some of them are listed here. The future looks bright for this open-source project as it has been successfully adopted by giants like Redhat & Microsoft under their Azure cloud platform.

Kubernetes so far has been hosted by CNCF, but there are chances that it might be transferred to Linux Foundation in future. Also, there are many players in this container orchestration space like Docker swarm, Mesos, Marathon. Each of them has its own advantages over others, it's too early to say which one is the best. Some experts believe that "One Swam To Rule Them All”. We'll see what future holds for these projects in coming years.

Kubernetes was donated by Google to Cloud Native Computing Foundation in Nov-2014, and since then has become one of the most widely used containers orchestrator technology at scale with active participation from all major industry players. Also, there are many hosting providers available who offer turnkey solutions on Kubernetes for developer’s enterprise users alike. It's being actively developed with frequent releases every few months.

Kubernetes is being widely used by the enterprises in many industries, it's expected that it will gradually replace existing container orchestration tools like "Docker swarm, Mesos, etc., especially when combined with other cloud-native technologies like Istio, Prometheus, Fluentd, etc.

KubeCon is the official Kubernetes conference held annually in Europe & North America. You can attend talks on latest developments and updates to Kubernetes project from leading expert speakers at KubeCon conferences. The upcoming kubecon event news can be found here.

So far, we talked about basic concepts of Kubernetes, let's now deploy a few distributed applications.

You can either follow the kubernetes documentation to install on your own machine or you can deploy it using Google Container Engine [GKE] [doc]. Kubernetes only requires docker & flannel components to run, you need not bother about networking part with GKE as they provide a turnkey solution for enterprise users.

Google Cloud offers a free trial of 600$ with 60$ per month for one year with 1 master node & 3 worker nodes with 24/7 support from Stackdriver team. This is ideal for small teams who want to develop and test their distributed apps on GCE.

Deployment with minikube: Minikube is a tool that makes it easy to run Kubernetes locally, it is as simple as "minikube start".

Installation guide for Minikube can be found here & Docker installation tutorial is available here.

Scroll to Continue

Minikube has a provision to connect external load balancer from cloud providers like Digital Ocean, AWS, Azure, etc., which facilitates usage of kubernetes in production deployments for developers with low budget.

Note: Take care to adjust max_pods parameter based on the compute resource availbale to minikube , default settings might not suite your requirement if you have a large number of nodes in cluster.

Deployment with GKE: Google Container Engine is a fully managed solution by Google that offers painless installation and configuration of Kubernetes on Google Cloud.

It is a great option for developers who want to deploy and test their distributed apps or build & run their CI/CD pipelines without worrying about infrastructure management on cloud. GKE provisions dedicated bare metal machines with all provisioned software components automatically installed and configured for you, so we can focus more on our applications rather than focussing on installation part. One can connect external load balancer from cloud providers like Digital Ocean, AWS, Azure etc., which facilitates usage of kubernetes in production deployments for developers with low budget.

GKE pricing starts from $400 per month with 1 master node & 3 worker nodes with 24/7 support from Stackdriver team along with integrated Istio service mesh, which makes it easier to secure, manage & monitor multi-service applications with ease.

Deployment steps are well documented here:

After deploying kubernetes cluster on cloud provider of your choice, user can access the kubectl command line tool on any machine by installing CLI component and use it for executing commands against their Kubernetes clusters.

For e.g: "kubectl command --help" provides all possible options which we can perform using the kubectl CLI.

Let's deploy a simple nginx container in our newly created cluster, then expose it as an external load balancer to see how nginx is exposed through its services via kubernetes.

You can check the current nodes in cluster by running "kubectl get nodes."

To deploy a pod/container to kubernetes, we need a yaml configuration file called manifest which defines everything about our container & its readiness for deployment, including volumes, cpu request & limits, networking requirements etc.,

This configuration file needs to be defined as a string template with parameters for each of its content as key value pairs separated by commas.

So let's create one such yml manifest file nginx-app.yaml using your favorite text editor with following contents: apiVersion: v1 kind: Pod metadata: name: nginx labels: app: nginx spec: containers: - name: nginx image: nginx ports: - containerPort: 80 Now, we need to create this pod in kubernetes cluster by running following command : "kubectl create -f ./nginx-app.yaml" .

To check if your pod is created or not, run "kubectl get pods" and it shall display the current state of your pod like below (running=yes) .

Once you are ready with all configurations, run the following command to expose nginx service externally & make it accessible outside the cluster: "kubectl expose deployment nginx --type=LoadBalancer --port=80 --target-port=80" which will provision an external load balancer on your cloud provider which you had selected during minikube installation. Also, exposed port number (here 80) can be changed with users desired choice.

Now you can access the nginx service & check load balancer provisioned by minikube/GKE at: "http://minikube-public-ip:80".

Once our pod is up & running in cluster, we need to check its container logs for any errors. This is done by executing following command: "kubectl logs nginx" Which shall print live logs of nginx container in our case like below (nginx service started without any problems). You can follow same approach for creating/deploying more pods.

In addition to kubectl command, there are few more tools played a key role in managing containerized applications on kubernetes. Let's discuss those briefly below:

Kube-proxy - This is a network proxy service which comes with default installation of Kubernetes cluster and runs as a daemon set (DaemonSet), which creates & manages the required number of proxies with desired configuration for users cluster. It takes care of ingress & egress networking requirements for services deployed on the cluster by forwarding/load balancing requests from external load balancer. You need to enable it before its features can be used by user pod containers. Enabling this simply involves creating a service account & binding it with kube-proxy DaemonSet.

You can verify if the service is enabled correctly by executing following command: "kubectl get svc --namespace=kube-system" which should list all the services provided by cluster including their status like below.

Once our pod is up & running in cluster, we need to check its container logs for any errors. This is done by executing following command: "kubectl logs nginx" Which shall print live logs of nginx container in our case like below (nginx service started without any problems). You can follow same approach for creating/deploying more pods like word press, tomcat etc.

In addition to kubectl command, there are few more tools played a key role in managing containerized applications on kubernetes. Let's discuss those briefly below:

Kube-proxy - This is a network proxy service which comes with default installation of Kubernetes cluster and runs as a daemon set (DaemonSet), which creates & manages the required number of proxies with desired configuration for users cluster. It takes care of ingress & egress networking requirements for services deployed on the cluster by forwarding/load balancing requests from external load balancer. You need to enable it before its features can be used by user pod containers. Enabling this simply involves creating a service account & binding it with kube-proxy DaemonSet.

You can verify if the service is enabled correctly by executing following command: "kubectl get svc --namespace=kube-system" which should list all the services provided by cluster including their status like below.

Kube-dns - This is a DNS server that runs within clusters to generate DNS records for Kubernetes Pods & Services. It replication controller ensures that there are always at least one running pod per namespace to serve DNS queries directed to it. Since kubedns only create A records, you have use external servers to set up any other type of record for your application.

NAME TYPE CLUSTER- EXTERNAL-IP PORT(S) AGE SELECTOR kube-dns ClusterIP 100.64.0.53 53/UDP,53/TCP 0s app=kube-dns kube-dns ClusterIP None <none> 53/UDP,53/TCP 2m app=kube-dns

Above output shows that our DNS service is up & running. You can use any dns tool out there to create records for your services like below: dig @100.64.0.53 -p 10053 +short wordpress 52.4.6.173 dig @100.64.0.53 -p 10053 +short tomcat 10.152.183.47

Above output shows that our DNS service is up & running. You can use any dns tool out there to create records for your services like below:

dig @100.64.0.53 -p 10053 +short wordpress 52.4.6.173 dig @100.64.0.53 -p 10053 +short tomcat 10.152.183.47

Kubectl - This is a CLI client which comes with default installation of Kubernetes cluster and provides ability to send commands directly to API Server including listing pods, viewing logs, etc.

kubectl get pods -o wide

It lists all the pods running on cluster.

kubectl logs tomcat-812g6n5 -c wordpress

It can also be used to check logs for a specific pod (Ex: "TomCat-812g6n5") which matches given label name ("wordpress") from deployment configuration file. You can learn more about this here. You have multiple options available to set up authentication & connection between client & API Server. This is required so that it uses whatever token you pass as command line argument without prompting for username & password which you enter using interactive login form. It makes life easier since we do not have provide any credentials when calling above command against a remote cluster.

kubectl config set-cluster myCluster --server=https://192.168.1.120:6443 --certificate-authority=/root/ca.crt

You can also use this to create a kubeconfig file which holds all the configuration values for your user account on specific cluster from which you can generate client configuration using below command.

It allows you creation of a config file based on a certificate authority & Cluster connection information that has been entered in by user. You can learn more about it here. If you have correctly configured kubelet on your nodes, then the authentication between API Server and Kubelets is already set up properly.

You can check if authentication was successful by getting list of pods like below: kubectl get pods -o wide --watch which should return a line with number of nodes that we have configured with our cluster as shown in image below.

If you do not see this line, then either your nodes are not properly registered or the authentication is not working correctly. We will see how to debug this further on. NOTE: You might receive an error message if there is no configuration file present for user or if server cannot be reached. If there is any such problem, please check the logs & resolve them before moving forward. There are multiple ways to solve this issue and they depend on your setup which I will discuss further on.

Kube-proxy - This is service which runs on all nodes and manages incoming traffic. It watches for new services, pods, daemonsets, replication controllers, etc. and creates rules in iptables to direct traffic to appropriate container. For example, if you have two pods running on node1 & node2 respectively, then kube-proxy will setup one container to receive requests at port 8080 (i.e host:port of first pod) while other container receives requests at port 8081(i.e host:port of second pod). If you try to access "node1" or any other pod IP directly, it will not be accessible since this service routes traffic only via registered service names. You cannot access pods via hostnames like: http://kube-node1:8080. For such scenarios, every service has to be registered with API Server and we can use "kubectl get services" command to check if any of our pod's service was not registered or it might also be that the connection between apiserver & kubelet is lost which you will learn how to debug later on.

We will see more about how to do networking in next section. NOTE: If you create a custom namespace for your Kubernetes cluster, then all core components should belong to this namespace so that they can work seamlessly together instead of default namespace since different namespaces don't share network information by default.

Deployment - This is a higher-level concept which you can think of as an abstraction for running your application in cluster. It provides declarative specification of desired state, manages replica sets (RS) and defines how containers should run & behave within the pod. For example, we can create deployment with one container, running nginx utilizing 2 CPU cores and 4GB memory. We will see more about deployments in later sections. Since we have named our nodes in previous section, we can use node name to directly refer to it while deploying pods or replication controllers by executing kubectl command from within corresponding directory containing these objects. You can learn more about this here. You might already be familiar with docker-compose file format, if so then its good news since deployments are written in the same way as docker-compose file. We will create a simple deployment for running nginx web server by following yaml file below:

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nginx spec: replicas: 1 template: metadata: labels: app : nginx spec : containers : image : nginx ports : - containerPort : 80 protocol : TCP envFrom : - configMapRef : name : nginx-configuration --- apiVersion: v1 kind: Service metadata: name: nginx labels: app : nginx spec : type : NodePort ports : - port : 80 selector : app=nginx deployment file contains necessary information such as : what has to be deployed, how many replicas are required , desired state of the deployment, how containers should run & behave within pod. You can learn more about deployments here. If everything is done correctly, you can try to create an nginx deployment by executing command below: kubectl create -f example.yaml

We will go through specs one after another in next article. This article contains only basic information for getting started with Kubernetes.

Related Articles