This chapter has the following technical requirements:

• An Ubuntu 22.04+ server running Docker with a minimum of 4 GB of RAM, though 8 GB is suggested.
• Scripts from the chapter1 folder from the repository, which you can access by using this link: https://github.com/PacktPublishing/Kubernetes-An-Enterprise-Guide-Third-Edition

You may have experienced a conversation like this at your office or school:

Developer: “Here’s the new application. It went through weeks of testing and you are the first to get the new release.”

….. A little while later …..

User: “It’s not working. When I click the submit button, it shows an error about a missing dependency.”

Developer: “That’s weird; it’s working fine on my machine.”

Encountering such issues can be incredibly frustrating for developers when they’re deploying an application. Oftentimes, these problems occur due to a missing library in the final package that the developer had on their own machine. One might think that a simple solution would be to include all the libraries in the release, but what if this release includes a newer version of a library that replaces an older version, which another application may still rely on?

Developers have to carefully consider their new releases and the potential conflicts they may cause with existing software on users’ workstations. It becomes a delicate balancing act that often requires larger deployment teams to thoroughly test the application on various system configurations. This situation can result in additional work for the developer or, in extreme cases, render the application completely incompatible with an existing one.

Over the years, there have been several attempts to simplify application delivery. One solution is VMware’s ThinApp, which aims to virtualize an application (not to be confused with virtualizing the entire operating system (OS)). It allows you to bundle the application and its dependencies into a single executable package. By doing so, all the application’s dependencies are contained within the package, eliminating conflicts with other application dependencies. This not only ensures application isolation but also enhances security and reduces the complexities of OS migrations.

You might not have come across terms like application packaging or application-on-a-stick until now, but it seems like a great solution to the infamous “it worked on my machine” problem. However, there are reasons why it hasn’t gained widespread adoption as anticipated. Firstly, most solutions in this space are paid offerings that require a significant investment. Additionally, they require a “clean PC,” meaning that for each application you want to virtualize, you need to start with a fresh system. The package you create captures the differences between the base installation and any changes made afterward. These differences are then packaged into a distribution file, which can be executed on any workstation.

We’ve mentioned application virtualization to highlight that application issues such as “it works on my machine” have had different solutions over the years. Products such as ThinApp are just one attempt at solving the problem. Other attempts include running the application on a server using Citrix, Remote Desktop, Linux containers, chroot jails, and even virtual machines.

Kubernetes removed all support for Docker in version 1.24 as a supported container runtime. While it has been removed as a runtime engine option, you can create new containers using Docker and they will run on any runtime that supports the Open Container Initiative (OCI) specification. OCI is a set of standards for containers and their runtimes. These standards ensure that containers remain portable, regardless of the container platform or the runtime used to execute them.

When you create a container using Docker, you are creating a container that is fully OCI compliant, so it will still run on Kubernetes clusters that are running any Kubernetes-compatible container runtime.

To fully explain the impact and the supported alternatives, we need to understand what a container runtime is. A high-level definition would be that a container runtime is the software layer that runs and manages containers. Like many components that make up a Kubernetes cluster, the runtime is not included as part of Kubernetes – it is a pluggable module that needs to be supplied by a vendor, or by you, to create a functioning cluster.

There are many technical reasons that led to the decision to deprecate and remove Docker, but at a high level, the main concerns were as follows:

• Docker contains multiple pieces inside of the Docker runtime to support its own remote API and user experience (UX). Kubernetes only requires one component in the executable, dockerd, which is the runtime process that manages containers. All other pieces of the executable contribute nothing to using Docker in a Kubernetes cluster. These extra components make the binary bloated and can lead to additional bugs, security, or performance issues.
• Docker does not conform to the Container Runtime Interface (CRI) standard, which was introduced to create a set of standards to easily integrate container runtimes in Kubernetes. Since it doesn’t comply, the Kubernetes team has had extra work that only caters to supporting Docker.

When it comes to local container testing and development, you can still use Docker on your workstation or server. Considering the previous statement, if you build a container on Docker and the container successfully runs on your Docker runtime system, it will run on a Kubernetes cluster that does not use Docker as the runtime.

Removing Docker will have very little impact on most users of Kubernetes in new clusters. Containers will still run using any standard method, as they would with Docker as the container runtime. If you happen to manage a cluster, you may need to learn new commands when you troubleshoot a Kubernetes node – you will not have a Docker command on the node to look at running containers, clean up volumes, and so on.

Kubernetes supports a number of runtimes in place of Docker. Two of the most commonly used runtimes are as follows:

• containerd
• CRI-O

While these are the two commonly used runtimes, there are a number of other compatible runtimes available. You can always view the latest supported runtimes on the Kubernetes GitHub page at https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/container-runtime-interface.md.

Introducing Docker

Both the industry and end users were seeking a solution that was both convenient and affordable, and this is where Docker containers came in. While containers have been utilized in different ways over time, Docker has brought about a transformation by providing a runtime and tools for everyday users and developers.

Docker brought an abstraction layer to the masses. It was easy to use and didn’t require a clean PC for every application before creating a package, thus offering a solution for dependency issues, but most attractive of all, it was free. Docker became a standard for many projects on GitHub, where teams would often create a Docker container and distribute the Docker image or Dockerfile to team members, providing a standard testing or development environment. This adoption by end users is what eventually brought Docker to the enterprise and, ultimately, what made it the standard it has become today.

Within the scope of this book, we will be focusing on what you will need to know when trying to use a local Kubernetes environment. Docker has a long and interesting history of how it evolved into the standard container image format that we use today. We encourage you to read about the company and how they ushered in the container world we know today.

While our focus is not to teach Docker inside out, we feel that those of you who are new to Docker would benefit from a quick primer on general container concepts.

If you have some Docker experience and understand terminology such as ephemeral and stateless, you can jump to the Installing Docker section.

Docker versus Moby

When the Docker runtime was developed, it was a single code base. The single code base contained every function that Docker offered, whether you have used them or not. This led to inefficiencies, and it started to hinder the progression of Docker and containers in general.

The following table shows the differences between the Docker and Moby projects.

Table 1.1: Docker versus Moby features

To recap – Moby is a project that was started by Docker, but it is not the complete Docker runtime. The Docker runtime uses the components from Moby to create the Docker runtime, which includes the Moby open-source components and Docker’s own open-sourced components.

Now, let’s move on to understanding Docker a little more and how you can use it to create and manage containers.

This book assumes that you have a foundational understanding of Docker and container concepts. However, we know that not everyone will have prior experience with Docker or containers. Therefore, we have included this crash course to introduce you to container concepts and guide you through the usage of Docker.

If you are new to containers, we suggest reading the documentation that can be found on Docker’s website for additional information: https://docs.docker.com/.

Containers are ephemeral

The first thing to understand is that containers are ephemeral.

The term “ephemeral” means something that exists for a short period. Containers can be intentionally terminated, or automatically restarted without any user involvement or consequences. To better understand this concept, let’s look at an example – imagine someone interactively adds files to a web server running within a container. The uploaded files are temporary because they were not originally part of the base image.

This means that once a container is built and running, any changes that are made to the container will not be saved once it is removed, or destroyed, from the Docker host. Let’s look at a full example:

1. You start a container running a web server using NGINX on your host without any base HTML pages.
2. Using a Docker command, you execute a copy command to copy some web files into the container’s filesystem.
3. To test that the copy was successful, you go to the website and confirm that it is serving the correct web pages.
4. Happy with the results, you stop the container and remove it from the host. Later that day, you want to show a coworker the website and you start your NGINX container. You go to the website again, but when the site opens, you receive a 404 error (page not found error).

What happened to the files you uploaded before you stopped and removed the container from the host?

The reason your web pages cannot be found after the container was restarted is that all containers are ephemeral. Whatever is in the base container image is all that will be included each time the container is initially started. Any changes that you make inside a container are short-lived.

If you need to add permanent files to an existing image, you need to rebuild the image with the files included or, as we will explain in the Persistent data section later in this chapter, you could mount a Docker volume in your container.

At this point, the main concept to understand is that containers are ephemeral.

But wait! You may be wondering, “If containers are ephemeral, how did I add web pages to the server?” Ephemeral just means that changes will not be saved; it doesn’t stop you from making changes to a running container.

Any changes made to a running container will be written to a temporary layer, called the container layer, which is a directory on the localhost filesystem. Docker uses a storage driver, which is in charge of handling requests that use the container layer. The storage driver is responsible for managing and storing images and containers on your Docker host. It controls the mechanisms and processes involved in their storage and management.

This location will store all changes in the container’s filesystem so that when you add the HTML pages to the container, they will be stored on the local host. The container layer is tied to the container ID of the running image and it will remain on the host system until the container is removed from Docker, either by using the CLI or by running a Docker prune job (see Figure 1.1 on the next page).

Considering that containers are temporary and are read only, you might wonder how it’s possible to modify data within a container. Docker addresses this by utilizing image layering, which involves creating interconnected layers that collectively function as a single filesystem. Through this, changes can be made to the container’s data, even though the underlying image remains immutable.

Docker images

A Docker image is composed of multiple image layers, each accompanied by a JavaScript Object Notation (JSON) file that stores metadata specific to the layer. When a container image is launched, these layers are combined to form the application that users interact with.

You can read more about the contents of an image on Docker’s GitHub at https://github.com/moby/moby/blob/master/image/spec/v1.1.md.

Image layers

As we mentioned in the previous section, a running container uses a container layer that is “on top” of the base image layer, as shown in the following diagram:

Figure 1.1: Docker image layers

The image layers cannot be written to since they are in a read-only state, but the temporary container layer is in a writeable state. Any data that you add to the container is stored in this layer and will be retained as long as the container is running.

To deal with multiple layers efficiently, Docker implements copy-on-write, which means that if a file already exists, it will not be created. However, if a file is required that does not exist in the current image, it will be written. In the container world, if a file exists in a lower layer, the layers above it do not need to include it. For example, if layer 1 had a file called /opt/nginx/index.html in it, layer 2 does not need the same file in its layer.

This explains how the system handles files that either exist or do not exist, but what about a file that has been modified? There will be times when you’ll need to replace a file that is in a lower layer. You may need to do this when you are building an image or as a temporary fix to a running container issue. The copy-on-write system knows how to deal with these issues. Since images read from the top down, the container uses only the highest layer file. If your system had a /opt/nginx/index.html file in layer 1 and you modified and saved the file, the running container would store the new file in the container layer. Since the container layer is the topmost layer, the new copy of index.html would always be read before the older version in the image layer.

Persistent data

Being limited to ephemeral-only containers would severely limit the use cases for Docker. You will probably encounter use cases where persistent storage is needed or data must be retained even if a container is stopped.

Remember, when you store data in the container image layer, the base image does not change. When the container is removed from the host, the container layer is also removed. If the same image is used to start a new container, a new container image layer is created. While containers themselves are ephemeral, you can achieve data persistence by incorporating a Docker volume. By utilizing a Docker volume, data can be stored externally in the container, enabling it to persist beyond the container’s lifespan.

Accessing services running in containers

Unlike a physical machine or a virtual machine, containers do not connect to a network directly. When a container needs to send or receive traffic, it goes through the Docker host system using a bridged network address translation (NAT) connection. This means that when you run a container and you want to receive incoming traffic requests, you need to expose the ports for each of the containers that you wish to receive traffic on. On a Linux-based system, iptables has rules to forward traffic to the Docker daemon, which will service the assigned ports for each container. There is no need to worry about how the iptables rules are created, as Docker will handle that for you by using the port information provided when you start the container. If you are new to Linux, iptables may be new to you.

That concludes the introduction to container fundamentals and Docker concepts. In the next section, we will guide you through the process of installing Docker on your host.

The hands-on exercises in this book will require that you have a working Docker host. To install Docker, we have included a script located in this book’s GitHub repository, in the chapter1 directory, called install-docker.sh.

Today, you can install Docker on just about every hardware platform out there. Each version of Docker acts and looks the same on each platform, making development and using Docker easy for people who need to develop cross-platform applications. By making the functions and commands the same between different platforms, developers do not need to learn a different container runtime to run images.

The following is a table of Docker’s available platforms. As you can see, there are installations for multiple OSs, as well as multiple architectures:

Table 1.2: Available Docker platforms

Images that are created using one architecture cannot run on a different architecture. This means that you cannot create an image based on x86 hardware and expect that same image to run on your Raspberry Pi running an ARM processor. It’s also important to note that while you can run a Linux container on a Windows machine, you cannot run a Windows container on a Linux machine.

While images, by default, are not cross-architecture compatible, there are new tools to create what’s known as a multi-platform image. Multi-platform images are images that can be used across different architectures or processors in a single container, rather than having multiple images, such as one for NGINX on x86, another one for ARM, and another one for PowerPC. This will help you simplify your management and deployment of containerized applications. Since multi-platform images contain various versions, one for each architecture you include, you need to specify the architecture when deploying the image. Luckily, the container runtime will help out and automatically select the correct architecture from the image manifest.

The use of multi-platform images provides portability, flexibility, and scalability for your containers across cloud platforms, edge deployments, and hybrid infrastructure. With the use of ARM-based servers growing in the industry and the heavy use of Raspberry Pi by people learning Kubernetes, cross-platform images will help make consuming containers quicker and easier.

For example, in 2020, Apple released the M1 chip, ending the era of Apple running Intel processors in favor of the ARM processor. We’re not going to get into the details of the difference, only that they are different and this leads to important challenges for container developers and users. Docker does have Docker Desktop, a macOS tool for running containers that lets you use the same workflows that you used if you had a Docker installation on Linux, Windows, or x86 macOS. Docker will try to match the architecture of the underlying host when pulling or building images. On ARM-based systems, if you are attempting to pull an image that does not have an ARM version, Docker will throw an error due to the architecture incompatibilities. If you are attempting to build an image, it will build an ARM version on macOS, which cannot run on x86 machines.

Multi-platform images can be complex to create. If you want additional details on creating multi-platform images, visit the Multi-platform images page on Docker’s website: https://docs.docker.com/build/building/multi-platform/.

The installation procedures that are used to install Docker vary between platforms. Luckily, Docker has documented many of them on their website: https://docs.docker.com/install/.

In this chapter, we will install Docker on an Ubuntu 22.04 system. If you do not have an Ubuntu machine to install on, you can still read about the installation steps, as each step will be explained and does not require that you have a running system to understand the process. If you have a different Linux installation, you can use the installation procedures outlined on Docker’s site at https://docs.docker.com/. Steps are provided for CentOS, Debian, Fedora, and Ubuntu, and there are generic steps for other Linux distributions.

Preparing to install Docker

Now that we have introduced Docker, the next step is to select an installation method. Docker’s installation changes between not only different Linux distributions but also versions of the same Linux distribution. Our script is based on using an Ubuntu 22.04 server, so it may not work on other versions of Ubuntu. You can install Docker using one of two methods:

• Add the Docker repositories to your host system
• Install using Docker scripts

The first option is considered the best option since it allows for easy installation and updates to the Docker engine. The second option is designed for installing Docker on testing/development environments and is not recommended for deployment in production environments.

Since the preferred method is to add Docker’s repository to our host, we will use that option.

Installing Docker on Ubuntu

Now that we have added the required repositories, the next step is to install Docker.

We have provided a script in the chapter1 folder of the Git repository called install-docker.sh. When you execute the script, it will automatically install all of the necessary binaries required for Docker to run.

To provide a brief summary of the script, it begins by modifying a specific value in the /etc/needrestart/needrestart.conf file. In Ubuntu 22.04, there was a change in how daemons are restarted, where users might be required to manually select which system daemons to restart. To simplify the exercises described in the book, we alter the restart value in the needsrestart.conf file to “automatic” instead of prompting for each changed service.

Next, we install a few utilities like vim, ca-certificates, curl, and GnuPG. The first three utilities are fairly common, but the last one, GnuPG, may be newer to some readers and might need some explaining. GnuPG, an acronym for GNU Privacy Guard, enhances Ubuntu with a range of cryptographic capabilities such as encryption, decryption, digital signatures, and key management.

In our Docker deployment, we need to add Docker’s GPG public key. which is a cryptographic key pair that secures communication and maintains data integrity. GPG keys use asymmetrical encryption, which involves the use of two different, but related, keys, known as a public key and a private key. These keys are generated together as a pair, but they provide different functions. The private key, which remains confidential, is used to generate the digital signatures on the downloaded files. The public key is publicly available and is used to verify digital signatures created by the private key.

Next, we need to add Docker’s repository to our local repository list. When we add the repository to the list, we need to include the Docker certificate. The docker.gpg certificate was downloaded by the script from Docker’s site and stored on the local server under /etc/apt/keyings/docker.gpg. When we add the repository to the repository list, we add the key by using the signed-by option in the /etc/apt/sources.list.d/docker.list file. The full command is shown here:

deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu   jammy stable

By including the Docker repository in our local apt repository list, we gain the ability to install the Docker binaries effortlessly. This process entails using a straightforward apt-get install command, which will install the five essential binaries for Docker: docker-ce, docker-ce-cli, containerd.io, docker-buildx-plugin, and docker-compose-plugin. As previously stated, all these files are signed with Docker’s GPG key. Thanks to the inclusion of Docker’s key on our server, we can be confident that the files are safe and originate from a reliable source.

Once Docker is successfully installed, the next step involves enabling and configuring the Docker daemon to start automatically during system boot using the systemctl command. This process follows the standard procedure applied to most system daemons installed on Linux servers.

After installing Docker, let’s get some configuration out of the way. First, you will rarely execute commands as root in the real world, so we need to grant permissions to use Docker to your user.

Granting Docker permissions

In a default installation, Docker requires root access, so you will need to run all Docker commands as root. Rather than using sudo with every Docker command, you can add your user account to a new group on the server that provides Docker access without requiring sudo for every command.

If you are logged on as a standard user and try to run a Docker command, you will receive an error:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/images/json: dial unix /var/run/docker.sock: connect: permission denied

To allow your user, or any other user you may want to add, to execute Docker commands, you need to add the users to a new group called docker that was created during the installation of Docker. The following is an example command you can use to add the currently logged-on user to the group:

sudo usermod -aG docker $USER

To add the new members to your account, you can either log off and log back into the Docker host, or activate the group changes using the newgrp command:

newgrp docker

Now, let’s test that Docker is working by running the standard hello-world image (note that we do not require sudo to run the Docker command):

docker run hello-world

You should see the following output, which verifies that your user has access to Docker:

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete
Digest: sha256:37a0b92b08d4919615c3ee023f7ddb068d12b8387475d64c622ac30f45c29c51
Status: Downloaded newer image for hello-world:latest
Hello from Docker!

This message shows that your installation is working correctly – congratulations!

To generate this message, Docker took the following steps:

1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the hello-world image from Docker Hub (amd64).
3. The Docker daemon created a new container from the image that runs the executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.

To try something more ambitious – you can run an Ubuntu container with the following:

$ docker run -it ubuntu bash

For more examples and ideas, visit https://docs.docker.com/get-started/.

Now that we’ve granted Docker permission, we can start unlocking the most common Docker commands by learning how to use the Docker CLI.

You used the Docker CLI when you ran the hello-world container to test your installation. The Docker command is what you will use to interact with the Docker daemon. Using this single executable, you can do the following, and more:

• Start and stop containers
• Pull and push images
• Run a shell in an active container
• Look at container logs
• Create Docker volumes
• Create Docker networks
• Prune old images and volumes

This chapter is not meant to include an exhaustive explanation of every Docker command; instead, we will explain some of the common commands that you will need to use to interact with the Docker daemon and containers.

You can break down Docker commands into two categories: general Docker commands and Docker management commands. The standard Docker commands allow you to manage containers, while management commands allow you to manage Docker options such as managing volumes and networking.

docker help

It is quite common to forget the syntax or options of a command, and Docker acknowledges this. If you ever find yourself in a situation where you can’t recall a command, you can always depend on the docker help command. It will help you by providing what the command can do and how to use it.

docker run

To run a container, use the docker run command with the provided image name. But, before executing a docker run command, you should understand the options you can supply when starting a container.

In its simplest form, an example command you can use to run an NGINX web server would be docker run bitnami/nginx:latest. This will start a container running NGINX, and it will run in the foreground, showing logs of the application running in the container. Pressing Ctrl + C will stop the running container and terminate the NGINX server:

nginx 22:52:27.42
nginx 22:52:27.42 Welcome to the Bitnami nginx container
nginx 22:52:27.43 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-nginx
nginx 22:52:27.43 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-nginx/issues
nginx 22:52:27.44
nginx 22:52:27.44 INFO  ==> ** Starting NGINX setup **
nginx 22:52:27.49 INFO  ==> Validating settings in NGINX_* env vars
nginx 22:52:27.50 INFO  ==> Initializing NGINX
nginx 22:52:27.53 INFO  ==> ** NGINX setup finished! **
nginx 22:52:27.57 INFO  ==> ** Starting NGINX **

As you saw, when you used Ctrl + C to stop the container, NGINX also stopped. In most cases, you want a container to start and continue to run without being in the foreground, allowing the system to run other tasks while the container also continues to run. To run a container as a background process, you need to add the -d, or --detach option to your Docker command, which will run your container in detached mode. Now, when you run a detached container, you will only see the container ID, instead of the interactive or attached screen:

[root@localhost ~]# docker run -d bitnami/nginx:latest
13bdde13d0027e366a81d9a19a56c736c28feb6d8354b363ee738d2399023f80
[root@localhost ~]#

By default, containers will be given a random name once they are started. In our previous detached example, if we list the running containers, we will see that the container has been given the name silly_keldysh, as shown in the following output:

CONTAINER ID      IMAGE                      NAMES
13bdde13d002      bitnami/nginx:l

If you do not assign a name to your container, it can quickly get confusing as you start to run multiple containers on a single host. To make management easier, you should always start your container with a name that will make it easier to manage. Docker provides another option with the run command: the --name option. Building on our previous example, we will name our container nginx-test. Our new docker run command will be as follows:

docker run --name nginx-test -d bitnami/nginx:latest

Just like running any detached image, this will return the container ID, but not the name you provided. In order to verify that the container ran with the name nginx-test, we can list the containers using the docker ps command, which we will explain next.

docker ps

Often, you will need to retrieve a list of running containers or a list of containers that have been stopped. The Docker CLI has a flag called ps that will list all running and stopped containers, by adding the extra flag to the ps command. The output will list the containers, including their container ID, image tag, entry command, creation date, status, ports, and container name. The following is an example of containers that are currently running:

CONTAINER ID   IMAGE                  COMMAND                 CREATED
13bdde13d002   bitnami/nginx:latest   "/opt/bitnami/script…"  Up 4 hours
3302f2728133   registry:2             "/entrypoint.sh /etc…"  Up 3 hours

This is helpful if the container you are looking for is currently running, but what if the container has stopped, or even worse, what if the container failed to start and then stopped? You can view the status of all containers, including previously run containers, by adding the -a flag to the docker ps command. When you execute docker ps -a, you will see the same output from a standard ps command, but you will notice that the list may include additional containers.

How can you tell which containers are running versus which ones have stopped? If you look at the STATUS field of the list, the running containers will show a running time; for example, Up xx hours, or Up xx days. However, if the container has been stopped for any reason, the status will show when it stopped; for example, Exited (0) 10 minutes ago.

IMAGE                  COMMAND                  CREATED         STATUS
bitnami/nginx:latest   "/opt/bitnami/script…"   10 minutes ago  Up 10 minutes
bitnami/nginx:latest   "/opt/bitnami/script…"   12 minutes ago  Exited (0) 10 minutes ago

A stopped container does not mean there was an issue with running the image. There are containers that may execute a single task and, once completed, the container may stop gracefully. One way to determine whether an exit was graceful or whether it was due to a failed startup is to look at the exited status code. There are a number of exit codes that you can use to find out why a container has exited.

Table 1.3: Docker exit codes

docker start and stop

You may need to stop a container due to limited system resources, limiting you to running a few containers simultaneously. To stop a running container and free up resources, use the docker stop command with the name of the container, or the container ID, you want to stop.

If you need to start that container at a future time for additional testing or development, execute docker start <name>, which will start the container with all of the options that it was originally started with, including any networks or volumes that were assigned.

docker attach

In order to troubleshoot an issue or inspect a log file, it may be necessary to interact with a container. One way to connect to a container that is currently running is by using the docker attach <container ID/name> command. When you perform this action, you establish a connection with the active process of the running container. If you attach to a container that is executing a process, it is unlikely that you will see any prompt. In fact, it’s likely that you will see a blank screen for a period of time until the container starts producing output that is displayed on the screen.

You should always be cautious when attaching to a container. It’s easy to accidentally stop the running process and, in turn, stop the container. Let’s use an example of attaching to a web server running NGINX. First, we need to verify that the container is running using docker ps:

CONTAINER ID   IMAGE                 COMMAND                   STATUS
4a77c14a236a   nginx                 "/docker-entrypoint.…"    Up 33 seconds

Using the attach command, we execute docker attach 4a77c14a236a.

When you attach to a process, you will only be able to interact with the running process, and the only output you will see is data being sent to standard output. In the case of the NGINX container, the attach command has been attached to the NGINX process. To show this, we will leave the attachment and curl to the web server from another session. Once we curl to the container, we will see logs outputted to the attached console:

[root@astra-master manifests]# docker attach 4a77c14a236a
172.17.0.1 - - [15/Oct/2021:23:28:31 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.61.1" "-"
172.17.0.1 - - [15/Oct/2021:23:28:33 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.61.1" "-"
172.17.0.1 - - [15/Oct/2021:23:28:34 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.61.1" "-"
172.17.0.1 - - [15/Oct/2021:23:28:35 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.61.1" "-"
172.17.0.1 - - [15/Oct/2021:23:28:36 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.61.1" "-"

We mentioned that you need to be careful once you attach to the container. Those who are new to Docker may attach to the NGINX image and assume that nothing is happening on the server or a process appears to be hung so they may decide to break out of the container using the standard Ctrl + C keyboard command. This will stop the container and send them back to a Bash prompt, where they may run docker ps to look at the running containers:

root@localhost:~# docker ps
CONTAINER ID      IMAGE  COMMAND    CREATED    STATUS
root@localhost:~#

What happened to the NGINX container? We didn’t execute a docker stop command, and the container was running until we attached to the container. Why did the container stop after we attached to it?

As we mentioned, when an attachment is made to a container, you are attached to the running process. All keyboard commands will act in the same way as if you were at a physical server that was running NGINX in a regular shell. This means that when the user used Ctrl + C to return to a prompt, they stopped the running NGINX process.

If we press Ctrl + C to exit the container, we will receive an output that shows that the process has been terminated. The following output shows an example of what happens in our NGINX example:

2023/06/27 19:38:02 [notice] 1#1: signal 2 (SIGINT) received, exiting2023/06/27 19:38:02 [notice] 31#31: exiting2023/06/27 19:38:02 [notice] 30#30: exiting2023/06/27 19:38:02 [notice] 29#29: exiting2023/06/27 19:38:02 [notice] 31#31: exit2023/06/27 19:38:02 [notice] 30#30: exit2023/06/27 19:38:02 [notice] 29#29: exit2023/06/27 19:38:02 [notice] 32#32: exiting2023/06/27 19:38:02 [notice] 32#32: exit2023/06/27 19:38:03 [notice] 1#1: signal 17 (SIGCHLD) received from 312023/06/27 19:38:03 [notice] 1#1: worker process 29 exited with code 02023/06/27 19:38:03 [notice] 1#1: worker process 31 exited with code 02023/06/27 19:38:03 [notice] 1#1: worker process 32 exited with code 02023/06/27 19:38:03 [notice] 1#1: signal 29 (SIGIO) received2023/06/27 19:38:03 [notice] 1#1: signal 17 (SIGCHLD) received from 292023/06/27 19:38:03 [notice] 1#1: signal 17 (SIGCHLD) received from 302023/06/27 19:38:03 [notice] 1#1: worker process 30 exited with code 02023/06/27 19:38:03 [notice] 1#1: exit

If a container’s running process stops, the container will also stop, and that’s why the docker ps command does not show a running NGINX container.

To exit an attachment, rather than use Ctrl + C to return to a prompt, you should have used Ctrl + P, followed by Ctrl + Q, which will exit the container without stopping the running process.

There is an alternative to the attach command: the docker exec command. The exec command differs from the attach command since you supply the process to execute on the container.

docker exec

A better option when it comes to interacting with a running container is the exec command. Rather than attach to the container, you can use the docker exec command to execute a process in the container. You need to supply the container name and the process you want to execute in the image. Of course, the process must be included in the running image – if you do not have the Bash executable in the image, you will receive an error when trying to execute Bash in the container.

We will use an NGINX container as an example again. We will verify that NGINX is running using docker ps and then, using the container ID or the name, we execute into the container. The command syntax is docker exec <options> <container name> <command>:

root@localhost:~# docker exec -it nginx-test bash
I have no name!@a7c916e7411:/app$

The option we included is -it, which tells the exec to run in an interactive TTY session. Here, the process we want to execute is Bash.

Notice how the prompt name changed from the original user and hostname. The hostname is localhost, while the container name is a7c916e7411. You may also have noticed that the current working directory changed from ~ to /app and that the prompt is not running as a root user, as shown by the $ prompt.

You can use this session the same way you would a standard SSH connection; you are running Bash in the container and since we are not attached to the running process in the container, Ctrl + C will not stop any process from running.

To exit an interactive session, you only need to type in exit, followed by Enter, which will exit the container. If you then run docker ps, you will notice that the container is still in a running state.

Next, let’s see what we can learn about Docker log files.

docker logs

The docker logs command allows you to retrieve logs from a container using the container name or container ID. You can view the logs from any container that is listed in your ps command; it doesn’t matter if it’s currently running or stopped.

Log files are often the only way to troubleshoot why a container may not be starting up, or why a container is in an exited state. For example, if you attempt to run an image and the image starts and suddenly stops, you may find the answer by looking at the logs for that container.

To look at the logs for a container, you can use the docker logs <container ID or name> command.

To view the logs for a container with a container ID of 7967c50b260f, you would use the following command:

docker logs 7967c50b260f

This will output the logs from the container to your screen, which may be very long and verbose. Since many logs may contain a lot of information, you can limit the output by supplying the logs command with additional options. The following table lists the options available for viewing logs:

Table 1.4: Log options

Checking log files is a process you will find yourself doing often, and since they can be very lengthy, knowing options like tail, until, and since will help you to find the information in a log quicker.

docker rm

Once you assign a name to a container, the assigned name cannot be used on a different container unless you remove it using the docker rm command. If you had a container running called nginx-test that was stopped and you attempted to start another container with the name nginx-test, the Docker daemon would return an error, stating that the name is in use:

Conflict.  The container name "/nginx-test" is already in use

The original nginx-test container is not running, but the daemon knows that the container name was used previously and that it’s still in the list of previously run containers.

When you want to reuse a specific name, you must first remove the existing container before launching a new one with the same name. This scenario commonly occurs during container image testing. You may initially start a container but encounter issues with the application or image. In such instances, you would stop the container, resolve the problems with the image or application, and wish to redeploy it using the same name. However, since the previous container with that name still exists in the Docker history, it becomes necessary to remove it before reutilizing the name.

To remove the nginx-test container, simply execute docker rm nginx-test:

root@localhost ~:# docker rm nginx-test
nginx-test
root@localhost ~:#

Assuming the container name is correct and it’s not running, the only output you will see is the name of the image that you have removed.

We haven’t discussed Docker volumes, but when removing a container that has a volume, or volumes, attached, it’s a good practice to add the -v option to your remove command. Adding the -v option to the docker rm command will remove any volumes that were attached to the container.

docker pull/run

When running a pull, make sure to specify the architecture. docker pull and run are used to either pull an image or run an image. If you try to run a container that doesn’t exist on the Docker host already, it will initiate a pull request to get the container and then run it.

When you attempt to pull or run a container, Docker will download a container that is compatible with the host’s architecture. If you want to download a different image that is based on a different architecture, you can add the --platform tag to the build command. For example, if you are on a system that is running on arm64 architecture and you want to pull an x86 image, you would add linux/arm64 as your platform. When running a pull, make sure to specify the architecture:

root@localhost ~:# docker pull --platform=linux/amd64 ubuntu:22.04
22.04: Pulling from library/ubuntu6b851dcae6ca: Pull completeDigest: sha256:6120be6a2b7ce665d0cbddc3ce6eae60fe94637c6a66985312d1f02f63cc0bcdStatus: Downloaded newer image for ubuntu:22.04WARNING: image with reference ubuntu was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64/v8docker.io/library/ubuntu:22.04

Adding --platform=linux/amd64 is what told Docker to get the right platform. You can use the same parameter for docker run to make sure that the right container image platform is used.

docker build

Similar to pull and run, Docker will attempt to build the image based on the host’s architecture: arm64. Assuming you are building on an arm64-based image system, you can tell Docker to create an x86 image by using the buildx sub-command:

root@localhost ~:# docker buildx build --platform linux/amd64 --tag docker.io/mlbiam/openunison-kubernetes-operator --no-cache -f ./src/main/docker/Dockerfile .

This addition tells Docker to generate the x86 version, which will run on any x86-based hardware.

In this chapter, you learned how Docker can be used to solve common development issues, including the dreaded “it works on my machine” problem. We also presented an introduction to the most commonly used Docker CLI commands that you will use on a daily basis.

In the next chapter, we will start our Kubernetes journey with an introduction to KinD, a utility that provides an easy way to run multi-node Kubernetes test servers on a single workstation.

1. A single Docker image can be used on any Docker host, regardless of the architecture used.
   1. True
   2. False

Answer: b

2. What does Docker use to merge multiple image layers into a single filesystem?
   1. Merged filesystem
   2. NTFS filesystem
   3. EXT4 filesystem
   4. Union filesystem

Answer: d

3. Kubernetes is only compatible with the Docker runtime engine.
   1. True
   2. False

Answer: b

4. When you edit a container’s filesystem interactively, what layer are the changes written to?
   1. OS layer
   2. Bottom-most layer
   3. Container layer
   4. Ephemeral layer

Answer: c

5. Assuming the image contains the required binaries, what Docker command allows you to gain access to a running container’s bash prompt?
   1. docker shell -it <container> /bin/bash
   2. docker run -it <container> /bin/bash
   3. docker exec -it <container> /bin/bash
   4. docker spawn -it <container> /bin/bash

Answer: c

6. If you start a container with a simple run command, without any flags, and the container is stopped, the Docker daemon will delete all traces of the container.
   1. True
   2. False

Answer: b

7. What command will show you a list of all containers, including any stopped containers?
   1. docker ps -all
   2. docker ps -a
   3. docker ps -list
   4. docker list all

Answer: b

Join the book’s Discord workspace for a monthly Ask Me Anything session with the authors:

https://packt.link/K8EntGuide