vSphere High Performance Cookbook: A cookbook is the ideal way to learn a tool as complex as vSphere. Through experiencing the real-world recipes in this tutorial you'll gain deep insight into vSphere's unique attributes and reach a high level of proficiency.

To step through this recipe, you need a running ESXi Server, a Virtual Machine, vCenter Server, and a working installation of the vSphere Client. No other prerequisites are required.

Let's get started:

The VMM determines a set of possible monitor modes to use, and then picks one to use as the default monitor mode, unless something other than Automatic has been specified. The decision is based on:

There are three valid combinations for the monitor mode, as follows:

BT, HV, and HWMMU are abbreviations used by ESXi to identify each combination.

When a virtual machine is powering on, the VMM inspects the physical CPU's features and the guest operating system type to determine the set of possible execution modes. The VMM first finds the set of modes allowed. Then it restricts the allowed modes by configuration file settings. Finally, among the remaining candidates, it chooses the preferred mode, which is the default monitor mode. This default mode is then used if you have left Automatic selected.

For the majority of workloads, the default monitor mode chosen by the VMM works best. The default monitor mode for each guest operating system on each CPU has been carefully selected after a performance evaluation of available choices. However, some applications have special characteristics that can result in better performance when using a non-default monitor mode. These should be treated as exceptions, not the rule.

The chosen settings are honored by the VMM only if the settings are supported on the intended hardware. For example, if you select Use software instruction set and MMU virtualization for a 64-bit guest operating system running on a 64-bit Intel processor, the VMM will choose Intel VT-x for CPU virtualization instead of BT. This is because BT is not supported by the 64-bit guest operating system on this processor.

The virtual CPU consists of the virtual instruction set and the virtual memory management unit (MMU). An instruction set is a list of instructions that a CPU executes. The MMU is the hardware that maintains the mapping between the virtual addresses and the physical addresses in the memory.

The combination of techniques used to virtualize the instruction set and memory determines the monitor execution mode (also called the monitor mode). The VMM identifies the VMware ESXi hardware platform and its available CPU features, and then chooses a monitor mode for a particular guest operating system on that hardware platform. The VMM might choose a monitor mode that uses hardware virtualization techniques, software virtualization techniques, or a combination of hardware and software techniques.

We always had a challenge in hardware virtualization. x86 operating systems are designed to run directly on the bare metal hardware, so they assume that they have full control on the computer hardware. The x86 architecture offers four levels of privilege to operating systems and applications to manage access to the computer hardware: ring 0, ring 1, ring 2, and ring 3. User-level applications typically run in ring 3, the operating system needs to have direct access to the memory and hardware, and must execute its privileged instructions in ring 0.

Binary translation allows the VMM to run in ring 0 for isolation and performance, while moving the guest operating system to ring 1. Ring 1 is a higher privilege level than ring 3 and a lower privilege level than ring 0.

VMware can virtualize any x86 operating systems by using a combination of binary translation and direct execution techniques. With binary translation, the VMM dynamically translates all guest operating system instructions and caches the results for future use. The translator in the VMM does not perform a mapping from one architecture to another; that would be emulation not translation. Instead, it translates from the full unrestricted x86 instruction set issued by the guest operating system to a subset that is safe to execute inside the VMM. In particular, the binary translator replaces privileged instructions with sequences of instructions that perform the privileged operations in the virtual machine rather than on the physical machine. This translation enforces encapsulation of the virtual machine while preserving the x86 semantics as seen from the perspective of the virtual machine.

Meanwhile, user-level code is directly executed on the processor for high-performance virtualization. Each VMM provides each virtual machine with all of the services of the physical system, including a virtual BIOS, virtual devices, and virtualized memory management.

In addition to software virtualization, there is support for hardware virtualization. This allows some of the work of running virtual CPU instructions to be offloaded onto the physical hardware. Intel has the Intel Virtualization Technology (Intel VT-x) feature. AMD has the AMD Virtualization (AMD-V) feature. Intel VT-x and AMD-V are similar in aim but different in detail. Both designs aim to simplify virtualization techniques.