At a high level, WSL provides the ability to run Linux binaries on Windows. The desire to run Linux binaries has been around for many years, at least if the existence of projects such as Cygwin (https://cygwin.com ) is anything to go by. According to its homepage, Cygwin is 'a large collection of GNU and Open Source tools which provide functionality similar to a Linux distribution on Windows'. To run Linux application on Cygwin, it needs to be rebuilt from source. WSL provides the ability to run Linux binaries on Windows without modification. This means that you can grab the latest release of your favorite application and work with it immediately.

The reasons for wanting to run Linux applications on Windows are many and varied and include the following:

• You are currently using Windows but have experience and familiarity with Linux applications and utilities.
• You are developing on Windows but targeting Linux for the deployment of your application (either directly or in containers).
• You are using developer stacks where the ecosystem has a stronger presence on Linux, for example, Python, where some libraries are specific to Linux.

Whatever your reason for wanting to run Linux applications on Windows, WSL brings you this capability and does so in a new and productive way. Whilst it has been possible to run a Linux virtual machine (VM) in Hyper-V for a long time, running a VM introduces some barriers to your workflow.

For example, starting a VM takes enough time for you to lose your flow of thought and requires a dedicated amount of memory from the host machine. Additionally, the file system in a VM is dedicated to that VM and isolated from the host. This means that accessing files between the Windows host and Linux VM requires setting up Hyper-V features for Guest Integration Services or setting up traditional network file sharing. The isolation of the VM also means that processes inside and outside the VM have no easy way to communicate with each other. Essentially, at any point in time, you are either working in the VM or outside of it.

When you first launch a terminal using WSL, you have a terminal application in Windows running a Linux shell. In contrast to the VM experience, this seemingly simple difference already integrates better into workflows as it is easier to switch between windows on the same machine than between applications on Windows and those in a VM session.

However, the work in WSL to integrate the Windows and Linux environments goes further. Whereas the file systems are isolated by design in a VM, with the WSL file system access is configured for you by default. From Windows, you can access a new \\wsl$\ networked file share that is automatically available for you when the WSL is running and provides access to your Linux file systems. From Linux, your local Windows drives are automatically mounted for you by default. For example, the Windows C: drive is mounted as /mnt/c.

Even more impressively, you can invoke processes in Linux from Windows and vice versa. As an example, as part of a Bash script in the WSL, you can invoke a Windows application and process the output from that application in Linux by piping it to another command, just as you would with a native Linux application.

This integration goes beyond what can be achieved with traditional VMs and creates some amazing opportunities for integrating the capabilities of Windows and Linux into a single, productive environment that gives you the best of both worlds!

The integration that has been achieved between the Windows host and the Linux VM environments with WSL is impressive. However, if you have used WSL 1 or are familiar with how it works, you may have read the previous paragraphs and wondered why WSL 2 moved away from the previous architecture, which didn't use a VM. In the next section, we'll take a brief look at the different architectures between WSL 1 and WSL 2 and what the use of a VM unlocks despite the extra challenges the WSL team faced to create the level of integration that we have just seen.

While this book discusses version 2 of the Windows Subsystem for Linux (WSL 2), it is helpful to briefly look at how version one (WSL 1) works. This will help you to understand the limitations of WSL 1 and provide context for the change in architecture in WSL 2 and the new capabilities that this unlocks. This is what will be covered in this section, after which the remainder of the book will focus on WSL 2.

Overview of WSL 1

In the first version of WSL, the WSL team created a translation layer between Linux and Windows. This layer implements Linux syscalls on top of the Windows kernel and is what enables Linux binaries to run without modification; when a Linux binary runs and makes syscalls, it is the WSL translation layer that it is invoking and that makes the conversion into calls to the Windows kernel. This is shown in the following figure:

Figure 1.1 – Outline showing the WSL 1 translation layer

In addition to the translation layer, there was also investments made to enable other capabilities such as file access between Windows and WSL and the ability to invoke binaries between the two systems (including capturing the output). These capabilities help to build the overall richness of the feature.

The creation of the translation layer in WSL 1 was a bold move and opened up new possibilities on Windows, however, not all of the Linux syscalls are implemented and Linux binaries can only run if all the syscalls they require are implemented. Fortunately, the syscalls that are implemented allow a wide range of applications to run, such as Python and Node.js.

The translation layer was responsible for bridging the gap between the Linux and Windows kernels and this posed some challenges. In some cases, bridging these differences added performance overhead. Applications that performed a lot of file access ran noticeably slower on WSL 1; for example, due to the overhead of translating between the Linux and Windows worlds.

In other cases, the differences between Linux and Windows run deeper and it is harder to see how to reconcile them. As an example, on Windows attempting to rename a directory when a file contained within it has been opened results in an error, whereas on Linux the rename can be successfully performed. In cases such as this, it is harder to see how the translation layer could have resolved the difference. This led to some syscalls not being implemented, resulting in some Linux applications that just couldn't be run on WSL 1. The next section looks at the changes made in WSL 2 and how they address this challenge.

Overview of WSL 2

As impressive a feat as the WSL 1 translation layer was, it was always going to have performance challenges and syscalls that were hard or impossible to implement correctly. With WSL 2, the WSL team went back to the drawing board and came up with a new solution: a virtual machine! This approach avoids the translation layer from WSL 1 by running the Linux kernel:

Figure 1.2 – Outline showing the WSL 2 architecture

When you think of a virtual machine, you probably think of something that is slow to start (at least compared to starting a shell prompt), takes a big chunk of memory when it starts up, and runs in isolation from the host machine. On the face of it, using virtualization for WSL 2 might seem unexpected after the work put in to bring the two environments together in WSL 1. In fact, the capability to run a Linux VM has long existed on Windows. So, what makes WSL 2 different from running a virtual machine?

The big differences come with the use of what the documentation refers to as a Lightweight utility virtual machine (see https://docs.microsoft.com/en-us/windows/wsl/wsl2-about). This virtual machine has a rapid startup that only consumes a small amount of memory. As you run processes that require memory, the virtual machine dynamically grows its memory usage. Better still, as that memory is freed within the virtual machine, it is returned to the host!

Running a virtual machine for WSL 2 means that it is now running the Linux kernel (the source code for it is available at https://github.com/microsoft/WSL2-Linux-Kernel). This in turn means that the challenges faced by the WSL 1 translation layer are removed: performance and syscall compatibility are both massively improved in WSL 2.

Coupled with the work to preserve the overall experience of WSL 1 (interoperability between Windows and Linux), WSL 2 presents a positive step forward for most scenarios.

For most use cases, WSL 2 will be the preferred version due to compatibility and performance, but there are a couple of things worth noting. One of these is that (at the time of writing) the generally available version of WSL 2 doesn't support GPU or USB access (full details at https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq#can-i-access-the-gpu-in-wsl-2-are-there-plans-to-increase-hardware-support). GPU support was announced at the Build conference in May 2020, and at the time of writing is available through the Windows Insiders Program (https://insider.windows.com/en-us/).

Another consideration is that because WSL 2 uses a virtual machine, applications running in WSL 2 will connect to the network via a separate network adapter from the host (which has a separate IP address). As we will see in Chapter 5, Linux to Windows Interoperability, the WSL team has made investments in network interoperability to help reduce the impact of this.

Fortunately, WSL 1 and WSL 2 can be run side by side so if you have a particular scenario where WSL 1 is needed, you can use it for that and still use WSL 2 for the rest.

In this chapter, you saw what WSL is and how it differs from the experience of a traditional VM by allowing integration between file systems and processes across the Windows and Linux environments. You also saw an overview of the differences between WSL 1 and WSL 2 and why, for most cases, the improved performance and compatibility make WSL 2 the preferred option.

In the next chapter, you will learn how to install and configure WSL and Linux distributions.