The first and foremost driver for Docker-enabled containerization is to competently and completely overcome the widely expressed limitations of the virtualization paradigm. Actually, we have been working on proven virtualization techniques and tools for quite a long time now in order to realize the much-demanded software portability. That is, with the goal of eliminating the inhibiting dependency between software and hardware there have been several right initiatives that include the matured and stabilized virtualization paradigm. Virtualization is a kind of beneficial abstraction, accomplished through the incorporation of an additional layer of indirection between hardware resources and software components. Through this freshly introduced abstraction layer (hypervisor or Virtual Machine Monitor (VMM)), any kind of software applications can run on any underlying hardware without any hitch or hurdle. In short, the longstanding goal of software portability is trying to achieve through this middleware layer. However, the much-published portability target is not fully met even by the virtualization technique. The hypervisor software from different vendors gets in the way of ensuring the much-needed application portability. Further, the distribution, version, edition, and patching differences of operating systems and application workloads hinder the smooth portability of workloads across systems and locations.

Similarly, there are various other drawbacks attached to the virtualization paradigm. In data centers and server farms, the virtualization technique is typically used to create multiple Virtual Machines (VMs) out of physical machines and each VM has its own Operating System (OS). Through this solid and sound isolation enacted through automated tools and controlled resource-sharing, multiple and heterogeneous applications are being accommodated in a physical machine. That is, the hardware-assisted virtualization enables disparate applications to be run simultaneously on a single physical server. With the virtualization paradigm, various kinds of IT infrastructure (server machines, storage appliances, and networking solutions) become open, programmable, remotely monitorable, manageable, and maintainable. However, because of the verbosity and bloatedness (every VM carries its own OS), VM provisioning typically takes a few minutes and this longer duration is not acceptable for production environments.

The other widely expressed drawback closely associated with virtualization is that the performance of virtualized systems also goes down due to the excessive usage of precious and expensive IT resources (processing, memory, storage, network bandwidth, and so on). Besides the longer runtime, the execution time of VMs is on the higher side because of multiple layers ranging from the guest OS, hypervisor, and the underlying hardware.

Finally, the compute virtualization has flourished, whereas the other closely associated network and storage virtualization concepts are just taking off; precisely speaking, building distributed applications and fulfilling varying business expectations mandate the faster and flexible provisioning, high availability, reliability, scalability, and maneuverability of all the participating IT resources. Computing, storage, and networking components need to work together in accomplishing the varying IT and business needs. This sharply increments the management complexity of virtual environments.

Enter the world of containerization. All the aforementioned barriers get resolved in a single stroke. That is, the evolving concept of application containerization coolly and confidently contributes to the unprecedented success of the software portability goal. A container generally contains an application. Along with the primary application, all of its relevant libraries, binaries, and other dependencies are stuffed and squeezed together to be packaged and presented as a comprehensive yet compact container to be readily shipped, run, and managed in any local as well as remote environments. Containers are exceptionally lightweight, highly portable, rapidly deployable, extensible, and so on. Further on, many industry leaders have come together to form a kind of consortium to embark on a decisive journey towards the systematic production, packaging, and delivery of industry-strength and standardized containers. This conscious and collective move makes Docker deeply penetrative, pervasive, and persuasive. The open source community is simultaneously spearheading the containerization conundrum through an assortment of concerted activities for simplifying and streamlining the containerization concept. These containerization life cycle steps are being automated through a variety of tools.

The Docker ecosystem is also growing rapidly in order to bring in as much automation as possible in the containerization landscape. Container clustering and orchestration are gaining a lot of ground; thus, geographically distributed containers and their clusters can be readily linked up to produce bigger and better application-aware containers. The distributed nature of cloud centers is, therefore, to get benefited immensely with all the adroit advancements gaining a strong foothold in the container space. Cloud service providers and enterprise IT environments are all set to embrace this unique technology in order to escalate the resource utilization and to take the much-insisted infrastructure optimization to the next level. On the performance side, plenty of tests demonstrate Docker containers achieving native system performance. In short, IT agility through the DevOps aspect is being guaranteed through the smart leverage of Dockerization, and this in turn leads to business agility, adaptivity, and affordability.

It is pertinent, and it is paramount for extracting and expounding the game-changing advantages of the Docker-inspired containerization movement over the widely used and fully matured virtualization paradigm. As elucidated earlier, virtualization is the breakthrough idea and game-changing trendsetter for the unprecedented adoption of cloudification, which enables the paradigm of IT industrialization. However, through innumerable real-world case studies, cloud service providers have come to the conclusion that the virtualization technique has its own drawbacks and hence the containerization movement took off powerfully.

Containerization has brought in strategically sound optimizations through a few crucial and well-defined rationalizations and the insightful sharing of compute resources. Some of the innate and hitherto underutilized capabilities of the Linux kernel have been rediscovered. A few additional capabilities too are being embedded to strengthen the process and applicability of containerization. These capabilities have been praised for bringing in the much-wanted automation and acceleration, which will enable the fledgling containerization idea to reach greater heights in the days ahead. The noteworthy business and technical advantages of containerization include bare metal-scale performance, real-time scalability, higher availability, IT DevOps, software portability, and so on. All the unwanted bulges and flabs are being sagaciously eliminated to speed up the roll-out of hundreds of application containers in seconds. The following diagram on the left-hand side depicts the virtualization aspect, whereas the diagram on the right-hand side vividly illustrates the simplifications that are being achieved in containers:

As we all know, there are two main virtualization types. In Type 1 virtualization, the hypervisor provides the OS functionalities plus the VM provisioning, monitoring, and management capabilities and hence there is no need for any host OS. VMware ESXi is the leading Type 1 virtualization hypervisor. The production environments and mission-critical applications are run on the Type 1 virtualization.

The second one is the Type 2 virtualization, wherein the hypervisor runs on the host OS as shown in the preceding figure. This additional layer impacts the system performance and hence generally Type 2 virtualization is being used for development, testing, and staging environments. The Type 2 virtualization greatly slows down the performance because of the involvement of multiple modules during execution. Here, the arrival of Docker-enabled containerization brings forth a huge boost to the system performance.

In summary, VMs are a time-tested and battle-hardened software stack and there are a number of enabling tools to manage the OS and applications on it. The virtualization tool ecosystem is consistently expanding. Applications in a VM are hidden from the host OS through the hypervisor. However, Docker containers do not use a hypervisor to provide the isolation. With containers, the Docker host uses the process and filesystem isolation capabilities of the Linux kernel to guarantee the much-demanded isolation.

Docker containers need a reduced disk footprint as they don't include the entire OS. Setup and startup times are therefore significantly lower than in a typical VM. The principal container advantage is the speed with which application code can be developed, composed, packaged, and shared widely. Containers emerge as the most prominent and dominant platform for the speedier creation, deployment, and delivery of microservices-based distributed applications. With containers, there is a lot of noteworthy saving of IT resources as containers consume less memory space.

Great thinkers have come out with a nice and neat comparison between VMs and containers. They accentuate thinking in terms of a house (VM) and an apartment complex. The house, which has its own plumbing, electrical, heating, and protection from unwanted visitors, is self-contained. An apartment complex has the same resources as a house, such as electrical, plumbing, and heating, but they are shared among all the units. The individual apartments come in various sizes and you only rent what you need, not the entire complex. The apartment flats are containers, with the shared resources being the container host.

Developers can use simple and incremental commands to create a fixed image that is easy to deploy and can automate building those images using a Dockerfile. Developers can share those images easily using simple, Git-style push and pull commands to public or private Docker registries. Since the inception of the Docker technology, there is an unprecedented growth of third-party tools for simplifying and streamlining Docker-enabled containerization.

Containers are primarily presented as the next-generation application delivery platform. Containers are bringing forth a kind of mechanism for efficiently virtualizing the OS for the sole purpose of running applications on a single kernel host. Applications can also include the fast-emerging microservices. The open source Docker platform is now available primarily in two variants:

• Docker Enterprise Edition (Docker EE): This is designed for enterprise development and IT teams who build, ship, and run business-critical applications in production at scale. Docker EE is integrated, certified, and supported to provide enterprises with the most secure container platform in the industry to modernize all applications.

• Docker Community Edition (Docker CE): This is ideal for developers and small teams looking to get started with Docker and experimenting with container-based applications. Docker CE is available on many platforms, from desktop to cloud to the server. Docker CE is available for macOS and Windows and provides a native experience to help you focus on learning Docker. You can build and share containers and automate the development pipeline, all from a single environment.

Docker and Microsoft have entered into a long-lasting partnership to bring the much-needed agility, portability, and security benefits of the Docker platform to every edition of Windows Server 2016. Organizations that upgrade their servers to this new OS will then be able to use containers right from the development to the production environments. Windows uses namespace isolation, resource control, and process-isolation mechanisms to restrict the files, network ports, and running processes that each container can access. This isolation ensures applications running in containers can't interact with or see other applications running on the host OS or in other containers. Microsoft includes two different types of container. The first type is based on the Windows Server core image and is called a Windows Server container. The second one is called a Hyper-V container and is based on the Windows Nano Server image.

Windows Server containers share the underlying OS kernel. This architecture enables faster startup and efficient packaging while delivering the capability to run a number of containers per host. Containers share the local data and APIs with lower isolation levels between each. These containers are best for homogenous applications that do not require strong isolation and security constraints. Large microservice applications composed of multiple containers can use Windows Server containers for performance and efficiency.

Hyper-V containers offer the best of both worlds: VMs and containers. Since each container gets a dedicated copy of Windows kernel and memory, Hyper-V containers have better isolation and security levels than Windows Server containers. The containers are more secure because the interaction with the host operating system and other containers is minimal. This limited sharing of resources also increases the startup time and the size of packaged containers.

Hyper-V containers are preferred in multi-tenant environments such as public clouds. Here is a summary of Windows container jargon with descriptions:

• Container Host: Physical or VM configured with the Windows container feature.
• Container Image: A container image contains the base OS, application, and all the application dependencies that are needed to quickly deploy a container.
• Container OS Image: The container OS image is the OS.
• Container Registry: Container images are stored in a container registry and can be downloaded on demand. A registry can be off- or on-premise.
• Docker Engine: It is the core of the open source Docker platform. It is a lightweight container runtime that builds and runs Docker containers.
• Dockerfile: Dockerfile is used by developers to build and automate the creation of container images. With a Dockerfile, the Docker daemon can automatically build a container image.

Microsoft has its own public and official repository available via this URL: https://hub.docker.com/u/microsoft/. Amazon Web Services (AWS) has begun supporting Windows containers, providing a more direct way for older applications to jump into the cloud.

Windows containers provide the same advantages as Linux containers for applications that run on Windows. Windows containers support the Docker image format and Docker API. However, they can also be managed using PowerShell. Two container runtimes are available with Windows containers, Windows Server containers, and Hyper-V containers. Hyper-V containers provide an additional layer of isolation by hosting each container in a super-optimized VM.

This addresses the security concerns of running containers on top of an OS. Further, it also enhances the container density in a compute instance. That is, by running multiple containers in Hyper-V VMs, you can effectively take your density count to another level and run hundreds of containers on a single host. Windows containers are just Docker containers. Currently, you can deploy Windows containers in Windows Server 2016 (Full, Core, or Nano Server Editions), Windows 10 (Enterprise and Professional Editions), as well as Azure. You can deploy and manage these containers from any Docker client, including the Windows command line when the Docker Engine is installed. You can also manage them from PowerShell, which is open source software.

In this book, we have focused on the Docker CE.

The Docker Engine is built on top of the Linux kernel and it extensively leverages Linux kernel features such as namespaces and cgroups. Due to the burgeoning popularity of Docker, it is now being packaged by all the major Linux distributions so that they can retain their loyal users as well as attract new users. You can install the Docker Engine using the corresponding packaging tool of the Linux distribution, for example, using the apt-get command for Debian and Ubuntu, and the yum command for Red Hat, Fedora, and CentOS. Alternatively, you can use the fully automated install script, which will do all the hard work for you behind the scenes.

If you are a Mac or Microsoft Windows user, you can run Docker on Linux emulations (VMs). There are multiple solutions available to run Docker using Linux VM, which is explained in a later subsection. Docker and Microsoft are working towards supporting native Windows containers to run a native Windows application, which is outside the scope of this book.

Docker is currently supported only on 64-bit architecture Linux and the Linux kernel must be 3.10 or later. At the time of writing this book, the latest version of the Docker was
17.03.0-ce. The following steps prescribe the installation procedure of the Docker Engine on Ubuntu Linux 16.04 in detail:

1. First, the best practice for installing any software in Ubuntu begins with the resynchronization of the package repository. This step will essentially update the package repository in line with the latest published packages, thus we will ensure that we always get the latest published version using the command shown here:

      $ sudo apt-get update

2. Add the Docker package repository path for Ubuntu 16.04 to your APT sources, as shown here:

      $ sudo sh -c "echo deb https://apt.dockerproject.org/repo \
      ubuntu-xenial main > /etc/apt/sources.list.d/docker.list"

3. Add the GNU Privacy Guard (GPG) key by running the following command:

      $ sudo apt-key adv --keyserver \
       hkp://p80.pool.sks-keyservers.net:80 --recv-keys \    
       58118E89F3A912897C070ADBF76221572C52609D

4. Resynchronize with the package repository using the following command:

      $ sudo apt-get update

5. Install Docker and start the Docker service:

      $ sudo apt-get install -y docker-engine

6. Having installed the Docker Engine, let's verify our installation by running docker --version as shown here:

      $ docker --version
      Docker version 17.03.0-ce, build 60ccb22

Hurrah!! We have successfully installed Docker version 17.03.0 community edition.

In the previous section, we installed the Docker Engine by manually configuring the GPG key, APT repository, and so on. However, the Docker community has taken a step forward by hiding all these details in an automated install script. This script enables the installation of Docker on most popular Linux distributions, either through the curl command or through the wget command, as shown here:

• For the curl command:

      $ sudo curl -sSL https://get.docker.io/ | sh

• For the wget command:

      $ sudo wget -qO- https://get.docker.io/ | sh

On a Mac system, you can run Docker on Linux VM. Tools such as Vagrant and Docker Toolbox are quite handy to emulate Linux on Mac and in turn run Docker on it. Docker recently released Docker on Mac as a Beta, using the xhyve hypervisor to provide the Linux emulation. The xhyve hypervisor virtualizes the Docker Engine environment and Linux kernel-specific features for the Docker daemon.

The following steps describe the installation of Docker for Mac:

1. Download Docker for Mac from the link https://download.docker.com/mac/beta/Docker.dmg.
2. Double-click to download Docker.dmg and move it, as shown here:

3. Now, double-click on Docker.app in Applications and it will install all Docker components. During installation, it will ask for the machine's administrative password to install the software.
4. Upon successful installation, the whale icon will appear in the top status bar:

5. Finally, verify the Docker versions:

      $ docker --version
      Docker version 17.03.0-ce, build 60ccb22
      $ docker-compose --version
      docker-compose version 1.11.2, build dfed245
      $ docker-machine --version
      docker-machine version 0.10.0, build 76ed2a6

As with the Mac, on Windows, you can also run Docker on Linux VMs using tools such as Vagrant and Docker Toolbox. Recently, Docker released a Beta version of Docker for Windows, which uses Hyper-V to virtualize the Docker Engine and Linux kernel-specific features that are essential to run the Docker Engine.

At the time of writing this book, Docker on Windows is supported only on 64-bit Windows 10 Enterprise and Education (1511 November update, Build 10586 or later). In the future, Docker will support more versions of Windows 10. Be aware that the Hyper-V package must be enabled.

It is always recommended that you use Docker native if you have a supported Windows 10 operating system. The following steps are required to install Docker on Windows:

1. Download the Docker for Windows installer from https://download.docker.com/win/beta/InstallDocker.msi.
2. Double-click on InstallDocker.msi; the installation wizard will start. It will ask for the Windows administrative password to complete the installation:

3. Docker starts automatically and the whale will appear in the status bar:

4. Finally, verify the Docker versions:

      $ docker --version
      Docker version 17.03.0-ce, build 60ccb22
      $ docker-compose --version
      docker-compose version 1.11.2, build dfed245
      $ docker-machine --version
      docker-machine version 0.10.0, build 76ed2a6

For other versions of Windows, you can install Docker Toolbox from https://docs.docker.com/toolbox/overview/. The Docker Toolbox runs Boot2Docker, a lightweight Linux VM on the Oracle VirtualBox hypervisor and installs the Docker Engine on top of it.

It is important to understand the Docker components and their versions, storage, and the execution drivers, the file locations, and so on. Incidentally, the quest for understanding the Docker setup will also reveal whether the installation was successful or not. You can accomplish this using two Docker subcommands: docker version and docker info.

Let's start our Docker journey with the docker version subcommand, as shown here:

$ sudo docker version
Client:
 Version:      17.03.0-ce
 API version:  1.26
 Go version:   go1.7.5
 Git commit:   60ccb22
 Built:        Thu Feb 23 10:57:47 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.0-ce
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   60ccb22
 Built:        Thu Feb 23 10:57:47 2017
 OS/Arch:      linux/amd64
 Experimental: false  

Although the docker version subcommand lists many lines of text, as a Docker user you should know what these following output lines mean:

• The client version
• The client API version
• The server version
• The server API version

Here, both the client and server are of community edition 17.03.0 and the client API and the server API of version 1.26.

If we dissect the internals of the docker version subcommand, then it will first list the client-related information that is stored locally. Subsequently, it will make a REST API call to the server over HTTP to obtain server-related details.

Learn more about the Docker environment using the docker info subcommand:

As you can see, in the output of a freshly installed Docker Engine, the number of Containers and Images is invariably nil. The Storage Driver has been set up as aufs, and the directory has been given the /var/lib/docker/aufs location. The runtime has been set to runc. This command also lists details, such as Logging Driver, Cgroups Driver, Kernel Version, Operating System, CPUs, and Total Memory.

On Linux installations, Docker is usually programmed to carry out the server-client communication using the Unix socket (/var/run/docker.sock). Docker also has an IANA-registered port, which is 2375. However, for security reasons, this port is not enabled by default.

Having installed the Docker Engine successfully, the next logical step is to download the images from the Docker Registry. The Docker Registry is an application repository that hosts various applications, ranging from basic Linux images to advanced applications. The docker pull subcommand is used to download any number of images from the registry. In this section, we will download a sample hello-world image using the following command:

$ sudo docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
78445dd45222: Pull complete 
Digest: sha256:c5515758d4c5e1e838e9cd307f6c6a0d620b5e07e6f927b07d05f6d12a1ac8d7
Status: Downloaded newer image for hello-world:latest 

Once the images have been downloaded, they can be verified using the docker images subcommand, as shown here:

$ sudo docker images
REPOSITORY    TAG     IMAGE ID         CREATED      VIRTUAL SIZE
hello-world   latest  48b5124b2768     6 weeks ago  1.84 kB  

Now you can start your first Docker container as shown here:

Cool, isn't it? You have set up your first Docker container in no time. In the preceding example, the docker run subcommand has been used to create a container from the hello-world image.

Most of the times you will not encounter any issues when installing Docker. However, unexpected failures might occur. Therefore, it is necessary to discuss the prominent troubleshooting techniques and tips. Let's begin by discussing troubleshooting know-how in this section. The first tip is that Docker's running status should be checked using the following command:

$ sudo service docker status  

If the Docker service is up-and-running, the Active column (the third from the top) will list the status of the Docker service as active (running), as shown next:

However, if the Active column shows inactive or maintenance as the status, your Docker service is not running. In such cases, restart the Docker service, as shown here:

$ sudo service docker restart  

If you are still experiencing issues with the Docker setup, then you must extract the Docker log, using the journalctl -u docker command, for further investigation.

In a way, Docker containers are the lightweight, loosely-coupled, and nimble cousins of VMs. As elucidated before, containers enable packaging an application along with all of its dependencies compactly and shipping it elsewhere, running it smoothly in development, test, and production environments. Docker harnesses some powerful kernel-level features intelligently and provides a growing ecosystem of tools for realizing and running containers in an automated fashion. The end result is a potential game-changer for distributed application developers and system administrators. With hybrid clouds as the toast of worldwide enterprises for their IT needs, the Docker platform is a blessing in disguise for enterprise IT teams. Containers are typical sandboxes, isolating processes from each other. Docker does a nice and neat job of advancing the containerization paradigm for a slew of purposes such as lightweight packaging, frictionless shipping, faster deployment, and more rapid delivery of software applications.

The next chapter throws more light on the operational aspects of Docker containers, especially the sagacious handling of containers in order to produce real-world Dockerized applications.