Critical performance consideration – VMM scheduler
The virtual machine monitor (VMM) is a thin layer that provides a virtual x86 hardware environment to the guest operating system on a virtual machine. This hardware includes a virtual CPU, virtual I/O devices, and timers. The VMM leverages key technologies in the VMkernel, such as scheduling, memory management, and the network and storage stacks.
Each VMM is devoted to one virtual machine. To run multiple virtual machines, the VMkernel starts multiple VMM instances, also known as worlds. Each VMM instance partitions and shares the CPU, memory, and I/O devices to successfully virtualize the system. The VMM can be implemented by using hardware virtualization, software virtualization (binary translation), or paravirtualization (which is deprecated) techniques.
Paravirtualization refers to the communication between the guest operating system and the hypervisor to improve performance and efficiency. The value proposition of paravirtualization is in the lower virtualization overhead, but the performance advantage of paravirtualization over hardware or software virtualization can vary greatly depending on the workload. Because paravirtualization cannot support unmodified operating systems (for example, Windows 2000/XP), its compatibility and portability is poor.
Paravirtualization can also introduce significant support and maintainability issues in production environments because it requires deep modifications to the operating system kernel and for this reason it was most widely deployed on Linux-based operating systems.
Getting ready
To step through this recipe, you need a running ESXi Server, a Virtual Machine, vCenter Server, and a working installation of the vSphere Client. No other prerequisites are required.
How to do it...
Let's get started:
Open up VMware vSphere Client.
Log in to the vCenter Server.
In the virtual machine inventory, right-click on the virtual machine, and then click on Edit Settings. The Virtual Machine Properties dialog box appears.
Click on the Options tab.
Change the CPU/MMU Virtualization option under Advanced to one of the following options:
Automatic
Use software for instruction set and MMU virtualization
Use Intel VT-X/AMD-V for instruction set virtualization and software for MMU virtualization
Use Intel VT-X/AMD-V for instruction set virtualization and Intel EPT/AMD RVI for MMU virtualization
Click on OK to save your changes.
For the change to take effect, perform one of these actions:
Reset the virtual machine
Suspend and then resume the virtual machine
vMotion the virtual machine
How it works...
The VMM determines a set of possible monitor modes to use, and then picks one to use as the default monitor mode, unless something other than Automatic has been specified. The decision is based on:
The physical CPU's features and guest operating system type
Configuration file settings
There are three valid combinations for the monitor mode, as follows:
BT: Binary translation and shadow page tables
HV: AMD-V or Intel VT-x and shadow page tables
HWMMU: AMD-V with RVI, or Intel VT-x with EPT (RVI is inseparable from AMD-V, and EPT is inseparable from Intel VT-x)
BT, HV, and HWMMU are abbreviations used by ESXi to identify each combination.
When a virtual machine is powering on, the VMM inspects the physical CPU's features and the guest operating system type to determine the set of possible execution modes. The VMM first finds the set of modes allowed. Then it restricts the allowed modes by configuration file settings. Finally, among the remaining candidates, it chooses the preferred mode, which is the default monitor mode. This default mode is then used if you have left Automatic selected.
For the majority of workloads, the default monitor mode chosen by the VMM works best. The default monitor mode for each guest operating system on each CPU has been carefully selected after a performance evaluation of available choices. However, some applications have special characteristics that can result in better performance when using a non-default monitor mode. These should be treated as exceptions, not the rule.
The chosen settings are honored by the VMM only if the settings are supported on the intended hardware. For example, if you select Use software instruction set and MMU virtualization for a 64-bit guest operating system running on a 64-bit Intel processor, the VMM will choose Intel VT-x for CPU virtualization instead of BT. This is because BT is not supported by the 64-bit guest operating system on this processor.
There's more...
The virtual CPU consists of the virtual instruction set and the virtual memory management unit (MMU). An instruction set is a list of instructions that a CPU executes. The MMU is the hardware that maintains the mapping between the virtual addresses and the physical addresses in the memory.
The combination of techniques used to virtualize the instruction set and memory determines the monitor execution mode (also called the monitor mode). The VMM identifies the VMware ESXi hardware platform and its available CPU features, and then chooses a monitor mode for a particular guest operating system on that hardware platform. The VMM might choose a monitor mode that uses hardware virtualization techniques, software virtualization techniques, or a combination of hardware and software techniques.
We always had a challenge in hardware virtualization. x86 operating systems are designed to run directly on the bare metal hardware, so they assume that they have full control on the computer hardware. The x86 architecture offers four levels of privilege to operating systems and applications to manage access to the computer hardware: ring 0, ring 1, ring 2, and ring 3. User-level applications typically run in ring 3, the operating system needs to have direct access to the memory and hardware, and must execute its privileged instructions in ring 0.
Binary translation allows the VMM to run in ring 0 for isolation and performance, while moving the guest operating system to ring 1. Ring 1 is a higher privilege level than ring 3 and a lower privilege level than ring 0.
VMware can virtualize any x86 operating systems by using a combination of binary translation and direct execution techniques. With binary translation, the VMM dynamically translates all guest operating system instructions and caches the results for future use. The translator in the VMM does not perform a mapping from one architecture to another; that would be emulation not translation. Instead, it translates from the full unrestricted x86 instruction set issued by the guest operating system to a subset that is safe to execute inside the VMM. In particular, the binary translator replaces privileged instructions with sequences of instructions that perform the privileged operations in the virtual machine rather than on the physical machine. This translation enforces encapsulation of the virtual machine while preserving the x86 semantics as seen from the perspective of the virtual machine.
Meanwhile, user-level code is directly executed on the processor for high-performance virtualization. Each VMM provides each virtual machine with all of the services of the physical system, including a virtual BIOS, virtual devices, and virtualized memory management.
In addition to software virtualization, there is support for hardware virtualization. This allows some of the work of running virtual CPU instructions to be offloaded onto the physical hardware. Intel has the Intel Virtualization Technology (Intel VT-x) feature. AMD has the AMD Virtualization (AMD-V) feature. Intel VT-x and AMD-V are similar in aim but different in detail. Both designs aim to simplify virtualization techniques.
