Kubernetes

(Usage hints for this presentation)

IT Systems, Summer Term 2024
Dr. Jens Lechtenbörger (License Information)

Chair of Data Science: Machine Learning and Data Engineering (Prof. Gieseke)
Dept. of Information Systems
University of Münster, Germany

1. Introduction

1.1. Core Questions

How to manage a cloud of computers?

Cloud of computers as abstract execution environment

1.2. Learning Objectives

Explain concepts for container orchestration in general and with Kubernetes
- Including horizontal scaling for stateless servers
Explain digital sovereignty and cloud repatriation

1.3. Retrieval practice

What is digital sovereignty?
- Introduced as course goal
- Revisited in OS part
  - With free software as precondition
    - Similarly for free firmware (and hardware)
What is Kubernetes?
What are scalability and replication?
What is HTTP?
What is an IP address?

Agenda

1. Introduction
2. Container Orchestration
3. Kubernetes (K8s)
4. K8s Examples
5. Conclusions

2. Container Orchestration

2.1. Orchestrator Features

Resource limit control
Scheduling
Load balancing
Health check
Fault tolerance
Autoscaling

Container orchestrators provide features listed here.

Resource limits may define constraints for reservations of CPU or memory resources for containers. The orchestrator communicates such constraints to container managers, which in turn enforce them for containers.

Scheduling by the orchestrator determines which containers (or pods in Kubernetes) to assign to what machines of the cluster. Thus, this notion of scheduling is unrelated to CPU Scheduling in OSs.

Load balancing aims to distribute incoming requests among multiple container instances. The simplest policy may just use round-robin assignments, but more complex policies based on actual load are possible as well.

Health checks serve to determine whether a container is still available, i.e., able to answer requests.

With health checks, the orchestrator can provide fault tolerance: If a health check fails, the container is considered to be faulty and can be destroyed, to be replaced by a newly started container. In addition, containers can be replicated, and the orchestrator makes sure that a predefined number of healthy replicas is available.

Finally, autoscaling automatically adds and removes containers depending on the current load.

(Source: (Casalicchio 2019))

3. Kubernetes (K8s)

3.1. Assorted Facts

Free container orchestrator
“Kubernetes logo” under Kubernetes Branding Guidelines; from GitHub
- Originally developed at Google
- Maintained by Cloud Native Computing Foundation (CNCF)
  - (Project of the Linux Foundation)
  - Variety of distributions
  - The cloud infrastructure (end of 2023), “just 15% of organizations that consume cloud computing services have no plans to use Kubernetes”
Kubernetes was originally developed by Google as container orchestrator. Nowadays it is free software maintained by the Cloud Native Computing Foundation, which in turn is a project of the Linux Foundation. Kubernetes comes in a variety of distributions, and it is, and has been for some years, the dominant cloud infrastructure.

“Datacenter as a Service”
- Declarative description of cluster with compute, storage, networking
  - YAML files

3.2. Architecture Diagram

Kubernetes Cluster Architecture

This diagram illustrates the differentiation of the so-called control plane with orchestration tasks from ordinary worker nodes.

To the right, we see Kubernetes Nodes as worker nodes, where Pods are executed as containers. (In the figure, “CRI” is short for Container Runtime Interface, which provides an abstraction for a variety of container runtimes, including Docker.)

A node is just a machine, possibly virtualized, on which to run application containers. However, Kubernetes does not assign containers to nodes. Instead, the basic unit of scheduling and deployment in Kubernetes is called pod. Pods may just contain a single container each, but more complex use cases exist as well. In general, each pod implements a “part” of an application, e.g., different pods for frontend, caching, API, and database. Importantly, being a unit of scheduling means that containers of a pod are started and stopped together, co-located on the same node. Horizontal scaling of applications is achieved by adding pod replicas.

Before continuing with other names in the right part, let us focus on the Kubernetes control plane.

The control plane orchestrates communication across the entire Kubernetes cluster. It exposes an API and interfaces to manage the lifecycle of containers and stores cluster configuration information in a distributed key-value store, called etcd. The Kubernetes API offers the interface to configure and monitor the cluster. Scheduling in Kubernetes is extensible; it allocates pods to nodes, taking the resource allocation on each node and the resource constraints of pods into account. Importantly, a controller manager and various controllers for different types of resources exist, which monitor different parts of the cluster in reconciliation loops to make sure that the observed state matches the desired cluster state.

E.g., node failures are detected, followed by scheduling of affected pods towards other nodes. As another example, controllers for replication make sure that the desired number of healthy replicas exist.

Besides, an optional cloud controller manager provides bridging functionality to run Kubernetes on different cloud environments, each of which may come with its own specific API.

As a side remark, functionality of the control plane is running in the form of special-purpose pods, on one or more special-purpose nodes.

Returning to the right part, a kubelet per node manages the pods and containers on that node. It keeps track of the desired state as maintained in the control plane and performs pod management operations as necessary; e.g., it monitors the health of desired pods and restarts them if necessary. Finally, the kube-proxy is responsible for networking, including routing and load balancing.

3.3. Basic Concepts

Node, pod, container, controller: Previous slide
Resources
- Entities representing state
- Selected examples (full documentation):
  - Namespace: Working area, separates different environments
  - Pod: Collection of containers, unit of scheduling
  - Service: Abstraction for exposing network application with one or more pods; think of load balancer
  - Deployment: API object managing pods (including replication)
  - PersistentVolume: Piece of (cloud) storage
  - PersistentVolumeClaim: Request to create PersistentVolume

The bullet points here list important Kubernetes concepts. Namespaces provide working areas, separating different environments.

Pods are collections of containers and serve as units of scheduling, as explained previously.

A service exposes a network application consisting of one or more pods, to which the service forwards requests. Without a service, pods cannot be accessed from outside the cluster.

A deployment provides declarative updates for pods and their containers. It describes a desired state, e.g., the number of desired replicas. The deployment controller monitors the current state, changing it to the desired state if necessary.

A PersistentVolume provides an abstraction for pieces of storage, e.g., on a network filesystem or in a cloud storage.

With a PersistentVolumeClaim, a user can request storage of a specific size with a specific access mode.

Note that this list is just an excerpt. The hyperlink on the slides points to the full documentation.

4. K8s Examples

4.1. Minikube Installation

Based on https://learnk8s.io/deploying-nodejs-kubernetes
1. Install kubectl
2. Install minikube
  - (Blog article points there for Windows)
(Browser-based alternative at https://labs.play-with-k8s.com/)

4.2. Create K8s Cluster with nginx

minikube start # Just one node; use options for more
kubectl cluster-info
kubectl get nodes
kubectl get pods -A # Pods of all namespaces; so far, control plane
kubectl apply -f https://gitlab.com/oer/cs/programming/-/raw/main/k8s/nginx-deployment.yaml # Add nginx with 3 replicas
kubectl get pods -l run=my-nginx -o wide # Note names and IP addresses of pods
minikube ssh
curl <pod-ip-address> # Performs GET request to nginx in pod; shows HTML
exit
kubectl apply -f https://gitlab.com/oer/cs/programming/-/raw/main/k8s/nginx-service.yaml
minikube service nginx-service # Connect to nginx cluster
kubectl exec -it <pod-name-from-above> -- bash # Maybe change index.html of nginx
minikube delete --all
kubectl explain deployment
kubectl explain deployment.spec.selector

4.2.1. Sample Deployment

# SPDX-FileCopyrightText: 2024 Jens Lechtenbörger
# SPDX-License-Identifier: CC0-1.0

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 3
  template:
    metadata:
      labels:
	run: my-nginx
    spec:
      containers:
      - name: nginx-container
	image: nginx
	ports:
	- containerPort: 80

4.2.2. Sample Service

# SPDX-FileCopyrightText: 2024 Jens Lechtenbörger
# SPDX-License-Identifier: CC0-1.0

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    run: my-nginx
  ports:
    - port: 80
      targetPort: 80
  type: LoadBalancer

4.3. Web App with Frontend and Backend

Stateful vs stateless servers
- Stateless: No local state, can just spawn new replicas
  - E.g., web server
  - Horizontal scaling with replicas
  Two major state models exist for servers in distributed systems, namely stateful and stateless ones.
  
  Stateless servers do not maintain state information between requests; every request is served independently of previous requests.
  
  A web server for a static web page is a typical example. Importantly, stateless servers can be scaled horizontally by creating replicated pods, as you saw previously.
- Stateful: Maintain local state, need recovery in case of failures
  - E.g., database server, filesystem
  In contrast, stateful servers maintain state, which changes during operation. E.g., think of a web shop where inventory levels are updated in response to sales. With replicated pods running as isolated containers, each pod would maintain inventory levels independently of other pods. Clearly, this would not work.
  
  Instead, sales of all pods need to be reflected in one shared inventory level. In Kubernetes, persistent volumes can be used to specify such shared pieces of storage.

In class: Scale knote
- Note-taking app
  - Stateless frontend
  - Text in MongoDB, images in object storage MinIO; both with PersistentVolumeClaim

4.4. Self-Study

Install minikube
Create k8s cluster and experiment with it

5. Conclusions

5.1. Cloud Repatriation

Cloud repatriation: Bring workload back from cloud
- See examples in BBC article, June 2024

Reasons
- Rising cloud bills (depending on use, millions of Euro per year)
  - Instead, invest in own infrastructure and personnel
  - See (Murugesan 2024) for examples and discussion
- Digital sovereignty
  - Public cloud is not “much easier” any more
    - Private cloud with Kubernetes or Portainer
  - Full control over private cloud
  - Security concerns, e.g., confidentiality of R&D data or proprietary code

Although cloud computing is highly popular, it comes with downsides that may cause cloud users to move their workloads back home from the cloud, which is called cloud repatriation. (In fact, those downsides may prevent others from using public cloud offerings in the first place.)

An article from June 2024 hyperlinked here provides examples.

A major reason for this change is high cloud fees. Depending on companies’ cloud use, fees could just become too high. So, rather than spending this money on the cloud, companies may decide to build and manage their own technology infrastructure and teams. The research article cited here provides examples and a discussion.

Besides financial considerations, another important reason for shifting away from the cloud is gaining digital sovereignty. In the past, relying on external cloud services might have seemed easier, but today, switching to self-managed cloud solutions based on platforms like Kubernetes (or alternatives with graphical user interfaces such as Portainer) has become feasible. Clearly, a private cloud infrastructure offers full control, and it does neither require to share confidential data nor software with third parties.

5.2. Summary

Software architecture may contain numerous containers
- Container orchestrator for management
  - Kubernetes as dominant software solution
    - Cluster with control plane and work nodes
    - Declarative description in YAML files
    - Reconciliation loops by controllers
    - Pods as units of scheduling, services for network access
  - “Datacenter as a service”
Cloud repatriation
- Migration of cloud workloads “back home”
- Cost and digital sovereignty
- Kubernetes also as on-premise tool

When designing software architecture, it is common to include multiple containers within the system. The purpose of container orchestrators lies in the management of all these containers, and Kubernetes is the dominant solution in industry. With its help, developers create clusters containing both a control plane and worker nodes, which facilitate efficient coordination among different components.

The configuration and behavior of parts of Kubernetes clusters are often described through declarative definitions written in YAML files. These descriptions enable reconciliation loops performed by controllers responsible for ensuring desired states align with actual conditions. Additionally, pods serve as fundamental units of scheduling, and services provide essential networking capabilities.

Cloud repatriation refers to the process of migrating workloads previously hosted on cloud servers back to local, on-premises infrastructure. Two primary motivators behind this shift are cost reduction and digital sovereignty. Notably, Kubernetes can help as infrastructure for cloud repatriation as its mechanisms and functionalities stay the same, regardless of whether businesses operate primarily in remote datacenters or locally managed facilities.

Bibliography

Casalicchio, Emiliano. 2019. “Container Orchestration: A Survey.” In Systems Modeling: Methodologies and Tools, edited by Antonio Puliafito and Kishor S. Trivedi, 221–35. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-92378-9_14.

Murugesan, Ganesh Kumar. 2024. “Cloud Services –- Boon or Bane: A Comprehensive Review.” In Southeastcon 2024, 108–12. https://doi.org/10.1109/SoutheastCon52093.2024.10500027.

License Information

Source files are available on GitLab (check out embedded submodules) under free licenses. Icons of custom controls are by @fontawesome, released under CC BY 4.0.