The evolution of containers

Container technology is a means of packaging an application so that it may run with separated dependencies, and its compartmentalization of a computer system has radically transformed software development today. In this section, we'll look at some of the key aspects, including where this technology originated and the background behind the container technology:

Figure 1.3 – A brief history of container technology

Early containers (chroot systems with Unix version 7), developed in the 1970s, offered an isolated environment in which services and applications could operate without interfering with other processes, thereby creating a sandbox for testing programs, services, and other processes. The original concept was to separate the workload of the container from that of production systems, allowing developers to test their apps and procedures on production hardware without disrupting other services. Containers have improved their abilities to isolate users, data, networking, and more throughout time.

With the release of Free BSD Jails in the 2000s, container technology finally gained traction. "Jails" are computer partitions that can have several jails/partitions on the same system. This jail architecture was developed in 2001 with Linux VServer, which included resource partitioning and was later linked to the Linux kernel with OpenVZ in 2005. Jails were merged with boundary separation to become Solaris Containers in 2004.

Container technology advanced substantially after the introduction of control groups in 2006. Control groups, or cgroups, were created to track and isolate resource utilization, such as CPU and memory. They were quickly adopted and improved upon in Linux Containers (LXC) in 2008, which was the most full and stable version of any container technology at the time since it functioned without changes having to be made to the Linux kernel. Many new technologies have sprung up because of LXC's reliability and stability, the first of which was Warden in 2011 and, more importantly, Docker in 2013.

Containers have gained a lot of usage since 2013 due to a slew of Linux distributions releasing new deployment and management tools. Containers running on Linux systems have been transformed into virtualization solutions at the operating system level, aiming to provide several isolated Linux environments on a single Linux host. Linux containers don't need their own guest operating systems; instead, they share the kernel of the host operating system. Containers spin up significantly faster than virtual machines since they don't require a specialized operating system.

Containers can employ Linux kernel technologies such as namespaces, Apparmor, SELinux profiles, chroot, and cgroups to create an isolated operational environment, while Linux security modules offer an extra degree of protection, ensuring that containers can't access the host machine or kernel. Containerization in terms of Linux provided even more versatility by allowing containers to run various Linux distributions from their host operating system if both were running on the same CPU architecture.

Linux containers provided us with a way to build container images based on a variety of Linux distributions, as well as an API for managing the containers' lifespan. Linux distributions also included client tools for dealing with the API, as well as snapshot features and support for moving container instances from one container host to another.

However, while containers running on a Linux platform broadened their applicability, they still faced several fundamental hurdles, including unified management, real portability, compatibility, and scaling control.

The emergence of Apache Mesos, Google Borg, and Facebook Tupperware, all of which provided varying degrees of container orchestration and cluster management capabilities, marked a significant advancement in the use of containers on Linux platforms. These platforms allowed hundreds of containers to be created instantly, and also provided support for automated failover and other mission-critical features that are required for container management at scale. However, it wasn't until Docker, a variation of containers, that the container revolution began in earnest.

Because of Docker's popularity, several management platforms have emerged, including Marathon, Kubernetes, Docker Swarm, and, more broadly, the DC/OS environment that Mesosphere built on top of Mesos to manage not only containers but also a wide range of legacy applications and data services written in, for example, Java. Even though each platform has its unique approach to orchestration and administration, they all share one goal: to make containers more mainstream in the workplace.

The momentum of container technology accelerated in 2017 with the launch of Kubernetes, a highly effective container orchestration solution. Kubernetes became the industry norm after being adopted by CNCF and receiving backing from Docker. Thus, using a combination of Kubernetes and other container tools became the industry standard.

With the release of cgroups v2 (Linux version 4.5), several new features have been added, including rootless containers, enhanced management, and, most crucially, the simplicity of cgroup controllers.

Container usage has exploded in the last few years (https://juju.is/cloud-native-kubernetes-usage-report-2021) in both emerging "cloud-native" apps and situations where IT organizations wish to "containerize" an existing legacy program to make it easier to lift and shift onto the cloud. Containers have now become the de facto standard for application delivery as acceptance of cloud-native development approaches mature.

We'll dive more into Kubernetes components in the next section.