An introduction to VMware vSphere

VMware vSphere is a suite of core infrastructure solutions that help manage and monitor a virtual data center. The term vSphere was coined by VMware as a new name for their flagship virtual infrastructure in the year 2009 with the release of VMware Virtual Infrastructure 4. All the previous releases were called Virtual Infrastructure 3.x or 3.5 or 2.5, and backwards. The most recent version being vSphere 6 is the sixth generation of VMware's vSphere product line. They are the most feature rich and probably the only virtualization suite on the market covering every aspect of the virtual infrastructure with their own products or solutions.

So what really makes up vSphere? vSphere is basically a set of software solutions which include the hypervisor (ESXi), the vCenter server, and its plugins, supporting databases and host management agents. The hypervisors create a platform to run virtual machines and the vCenter forms the management layer. vCenter enables the creation of virtual data centers. Every other solution will interface and interact with the vCenter to manage or utilize the virtual data center. Having said that, VMware does offer APIs which allow third-party software developers to build tools that help manage platforms or leverage the management layer formed by the vCenter servers in an environment.

However, there are several components, tools, and features that fall under the umbrella of the vSphere suite. Not all components are within the scope of this book, but we will make an effort to include their relevance wherever possible. Most of the components are covered in depth in different chapters, but it is critical to have a brief understanding of these components before we learn about them in detail.

We will go through a very basic introduction of the following components and features.

vSphere ESXi

If you have read through the chapter up to this point then you will already have an understanding of what ESXi is. With the latest version, ESXi 6.0, there are a few scalability and a number of security enhancements.

Each ESXi 6.0 host can now support up to 480 logical CPUs, 12 terabytes of memory, and 1024 virtual machines. Let's compare this with some of the earlier versions of the ESXi hypervisor:

Limits	ESXi 6.0	ESXi 5.5	ESXi 5.1	ESXi 5.0	ESXi 4.1	ESXi 4.0
Number of logical CPUs per ESXi host	480	320	160	160	160	64
Amount of memory per ESXi host	12 TB	4TB	2TB	2TB	ITB	ITB
Number of virtual machines per ESXi host	1024	512	512	512	320	320

There are a number of security enhancements with the new version, and these include:

• Managing the local accounts on an ESXi host either via vCenter or using new ESXCLI commands. With the earlier versions the local account management was performed via a direct vSphere Client connection to the ESXi host or using the Linux-like user management commands from the ESXi console.
• New host advanced system settings to manage account lockout and password complexity policies.
• Better auditability. User information in the logs for all actions initiated from the vCenter will now include the actual vCenter username along with vxpuser.
• There are two different lockdown modes with the release of ESXi 6.0-Normal mode and Strict mode.
• Enhanced graphics performance for VMware Horizon virtual desktops by leveraging NVIDIA GRID^TM technology.

VMware vCenter Server

In the previous sections of this chapter, we learnt about ESXi and virtual machines. In a large infrastructure, these entities need to be centrally managed. The central management is achieved using VMware vCenter Server. It comes in the form of a Windows installable program and also as a Linux-based virtual appliance. Without the vCenter server, you cannot cluster the ESXi hosts, which is essential for the enablement of the VMware features such as vSphere HA, vSphere DRS, and vSphere DPM. Also, every other management solution that is out there will need to interface with the vCenter Server by means of a plugin.

vSphere desktop and web clients

Currently, there are two types of client available from VMware that can be used to connect and manage your vSphere infrastructure. One of them is a desktop client which can only be installed on a Windows machine. It can be used to connect directly to an ESXi host or a vCenter Server. This form of the client will reach its end of life very soon as VMware will transition every GUI action to be performed through their web client. The desktop client is C# based and it is currently available only for backward compatibility and to support a few plugins which haven't been completely transitioned to the vSphere Web Client. Unlike the desktop client, the vSphere Web Client is a server component installed and configured on a machine and the users willing to connect will rely on their web browsers to connect the web client server to access the vCenter GUI. The most critical difference is that the vSphere Web Client cannot be used to connect to an ESXi host directly. You need to rely on the vSphere C# based desktop client for that, and it is one of many reasons why the desktop client is still around.

vRealize Orchestrator

VMware vRealize Orchestrator, also known as the vCenter Orchestrator, is a GUI-based process automation tool that is installed along with your vCenter Server. It is primarily used to create workflows to automate repeatable IT processes. It has a plug-in framework which can be used by other solutions to perform actions. The vCenter Server, vRealize Automation, VROPS, VCM, and tools that can leverage the Orchestrator to perform actions.

vSphere Update Manager

It becomes necessary to upgrade or patch your vSphere environment to maintain a reliable platform for your virtual machines. Although the ESXi hosts can be patched or upgraded manually it becomes a very tedious process and would require many man-hours to perform the activity in a large environment. This is where VMware vSphere Update Manager (VUM) comes in handy. It provides a mechanism to patch and upgrade the ESXi hosts with reduced manual intervention. It can also be used to upgrade or patch third-party products such as the Cisco Nexus 1,000V.

VMware Power CLI

VMware Power CLI is a set of modules or snap-ins which include cmdlets based on Microsoft Power Shell. It is used as a scripting tool for managing or automating most of the vSphere actions. The latest version, 6.0, has more than 400 cmdlets for both vSphere and vCloud environments.

VMware VROPS

VMware vRealize Operations Manager (VROPS) is an infrastructure monitoring solution. It does provide greater insights into the performance, capacity, and health characteristics of your vSphere environment. It can present information in the form of dashboards, it can generate smart alerts, and can perform predictive analysis. It comes packaged with a vCenter plugin, but you can install several other third-party plugins to let VROPS gather information from other components as well. For instance, there are adapters for EMC Symmetrix, VNX storage systems, and many more.

vSphere Data Protection

vSphere Data Protection (VDP) is an EMC Avamar-based backup and recovery solution from VMware Inc. It is available in the form, a Linux virtual appliance and can support up to 8 terabytes of de-duplicated backup data per appliance and up to 20 such virtual appliances can be associated with a single vCenter Server.

vShield Endpoint

VMware vShield Endpoint is a security framework from VMware which enables hosting the load of performing antivirus or antimalware analysis on virtual machines onto a dedicated appliance. The framework utilizes a thin-agent included with VMware Tools and a heuristics engine running on a separate appliance provided by the security vendor. Every ESXi host will run such an appliance for the virtual machines running on it.

VMware vMotion and Storage vMotion

VMware vMotion will let you migrate the live state of a powered-on virtual machine from one ESXi host to another without affecting any of the applications or its services running on it. Whilst Storage vMotion can relocate all the files backing the virtual machine from one data store to another and also migrate its live state from one host to another, or it can migrate only the files backing the virtual machine and leave the live state on the same host.

vSphere High Availability

VMware vSphere High Availability (HA) is a functionality that is used to configure a cluster of ESXi hosts to respond to an unplanned downtime event and ensure the availability of the virtual machines that were running on them, with very minimal downtime possible. It has the ability to monitor the guest operating systems and the applications running inside of a virtual machine and then decide to restart the affected virtual machine in an effort to reduce the downtime of a service due to an affected guest operating system hosting the service or a nonresponsive application corresponding to the service. It is important to understand that even though HA is configured on a cluster of ESXi hosts, it only provides high availability for the virtual machines and not for the hosts. It cannot start up or restart an affected ESXi host.

vSphere Fault Tolerance

VMware vSphere Fault Tolerance (FT) is used to enable continuous availability of a virtual machine with zero downtime, maintaining an identical copy of the virtual machine in lock-step mode. We will learn more about this, in Chapter 8, Virtual Machine Concepts and Management. Unlike vSphere HA, FT is enabled on individual virtual machines. Although, FT had imposed a lot of restrictions on the scalability and the actions that can be performed on an FT-enabled virtual machine with the earlier versions of vSphere, with vSphere 6, it has been vastly improved and most of the restrictions don't exist anymore:

Features	vSphere 6.0 FT	vSphere 5.5 FT
Number of vCPUs	4-vCPU SMP	1-vCPU
Max. memory on the VM	64 GB	64 GB
Sync technology	Fast checkpointing	Record and replay
Virtual machine snapshots	Yes	No
VMDK types	All types	Eager-zeroed thick only

vSphere Distributed Resource Scheduler and Storage Distributed Resource Scheduler

VMware vSphere Distributed Resource Scheduler (DRS) is a series of algorithms devised to manage an aggregated pool of computing resources and distribute virtual machines among the ESXi hosts in a cluster in an effort to reduce any resource imbalance in the cluster. It also helps in reducing the power consumption in the data center using DRS's power management feature known as Distributed Power Management (DPM). VMware DPM can help reduce the energy consumption of a data center by vacating VMs from an underutilized host and putting that host in a power-off state.

Unlike DRS, which manages the compute resources, Storage DRS manages the storage resources. It is a mechanism to balance space utilization and the I/O load on data stores in a data store cluster by migrating (using Storage vMotion) the VMs. Storage DRS can only be enabled on a data store cluster. It also influences the initial placement of the VMs on the data stores, by generating placement recommendations. vSphere Storage DRS requires Enterprise Plus licensing.

Tip

To understand how the vSphere licensing editions compare refer to: https://www.vmware.com/products/vsphere/compare

vSphere Storage I/O Control and Network I/O Control

VMware vSphere Storage I/O Control (SIOC) is used to throttle the VMkernel device queue depth of a LUN, based on the shares set on the virtual machine disks contending for I/O bandwidth. SIOC can only be enabled on data stores (FC/ISCSI/NFS) and not on RDMs. It cannot be enabled on data stores with multiple extents. In this book, you will learn how to enable SIOC on a data store.

VMware vSphere Network I/O Control (NIOC) enables use and creation of Network Resource Pools. Much like with the compute resources of an ESXi cluster, you can use resource pools on a vSphere Distributed Switch (VDS) to configure Shares, Bandwidth Limitation, and Quality of Service (QoS) values. Such resource pools are referred to as Network Resource Pools (NRP). There are both System Defined and User Defined NRPs.

Tip

Both SIOC and NIOC requires vSphere Enterprise Plus licensing.

vSphere Standard Switch and Distributed Virtual Switches

VMware vSphere Standard Switch (vSwitch) is a software switching construct (in other words, a software-based network switch) local to each ESXi host. It provides a network infrastructure for the virtual machines running on that host. Unlike a physical switch, a vSphere Standard Switch is not a managed switch. It doesn't learn MAC addresses to build a MAC table, but it does know the MAC addresses of the virtual machine vNICs connected to it.

Unlike the standard switch, the vSphere Distributed Switch (VDS) spans across multiple ESXi hosts. It is not locally managed at the ESXi host. It requires VMware vCenter Server for configuration and management, though VDS is only available with the vSphere Enterprise Plus license. It has a control plane which resides at the vCenter Server and a data plane which resides on an ESXi host that is connected to the VDS.

vSphere Virtual Symmetric Multiprocessing

VMware vSphere Virtual Symmetric Multiprocessing (SMP) enables a virtual machine to use more than one logical processor simultaneously.

VMware Virtual Machine File System

Virtual Machine File System (VMFS) is VMware's proprietary cluster filesystem that can be used to format block storage units presented to an ESXi host. VMFS will let more than one host have simultaneous read/write access to the volume. To make sure that a virtual machine or its files are not simultaneously accessed by more than one ESXi host, VMFS uses an on-disk locking mechanism called distributed locking. The current version of VMFS is 5.

VMware Virtual Volumes

Virtual Volumes (VVols) is a newly introduced concept with vSphere 6.0. It is not intended to replace VMFS, but to take advantage of the hardware capabilities of the storage system. It requires a supported vSphere API for Storage Awareness (VASA) provider for its functioning. It is not a filesystem by any means. It is only a method to encapsulate files, backing a virtual machine into virtual volumes, and these are created automatically when you create or modify a virtual machine. ESXi does not have direct control over the VVols created, instead it interacts with a Protocol Endpoint, which again is provided by the storage vendor.

vSphere Storage APIs

VMware vSphere Storage API is an application programming interface framework from VMware that enables the storage and backup software vendors to enable or enhance integration with vSphere. The vSphere Storage APIs-Data Protection (VADP) is a framework that enables backup vendors to create backup and recovery solutions that integrate with vSphere. The vSphere Storage APIs-Storage Awareness (VASA) enables storage vendors to create storage providers which become an interface for vCenter to gather storage characteristics for the LUNs presented to the ESXi hosts. The vSphere Storage APIs-Array Integration (VAAI) enables ESXi to offload certain storage operations to a supported storage array. For instance, the process of zeroing the blocks of an eager-zeroed thick VMDK during its creation can be offloaded to the array to speed up the process. The availability of these APIs is dependent of the type of license in use. So, when you are designing an environment for performance it is important to understand what APIs are available with which VMware license editions.

VMware Virtual SAN

VMware Virtual SAN is a hyper-converged storage architecture that enables creating a shared storage platform using the local storage on the participating ESXi hosts. Since this ability is built into the hypervisor, there is no requirement to deploy appliances. All the management is from the vCenter Server. VSAN supports two types of configuration, an all-flash architecture and a hybrid architecture. In a hybrid-architecture, SSD and magnetic HDDs are mixed together to form the storage layer. The SSDs will be used for caching purposes to increase performance. In an all-flash architecture, both caching and storage are done on SDDs, hence delivering a very high performance storage platform.

Storage Thin Provisioning

vSphere Storage Thin Provisioning enables the creation of Virtual Machine Disks (VMDKs) that consume the space required for the data in it and not the actual size of the VMDK. Meaning, if the VMDK is of the size 50GB, but the data in it is only 15 GB, then only 15 GB worth of storage space is consumed from the data store. It is beneficial because not every disk created is fully consumed leading to wastage of storage space. Thin provisioning helps in over-allocation, but requires better reporting to manage the consumption of the storage resources.

vSphere Flash Read Cache

Flash storage (Solid State Disks-SSDs) disks offer higher I/O performance when comparted to the magnetic disks. Unfortunately, SSDs are far more expensive than the regular hard disks. With the vSphere Flash Read Cache mechanism, you can configure the available local SDD storage to act as a cache for virtual machines to use. VMkernel handles the assignment and allocation of the cache.

vSphere Content Library

With vSphere 6, VMware introduced a new feature called the Content Library. It is used to store templates and other files that can be shared across infrastructures, and it is backed by a data store. They can be local to a vCenter, published to be subscribed, or subscribed from a published library.

vSphere Auto Deploy

vSphere Auto Deploy is a web server component, which once configured can be used to quickly provision a large number of the ESXi hosts without the need to use the ESXi installation image to perform an installation on the physical machine. It can also be used to perform the upgrade or patching of the ESXi hosts without the need for VUM.

vSphere Host Profiles

A VMware vSphere Host Profile is a configuration template that is created from existing ESXi hosts. It could only be created using the vCenter GUI. Host Profiles can be attached to other ESXi hosts managed by the vCenter and can be used to track configuration changes by monitoring compliance of the attached hosts, or it can even be used to apply configuration changes to a large number of hosts, greatly reducing the amount of manual work which would otherwise be required.

vSphere Replication

vSphere Replication is a replication engine that can be leveraged to configure replication on individual virtual machines. It can replicate a virtual machine and its disks from one location to another without the need to incorporate an expensive array-based replication. What it really does is provide a mechanism to replicate a virtual machine using the existing Ethernet infrastructure and recover them when there is a need. It directly integrates with the vSphere platform and is available with Standard, Enterprise, and Enterprise Plus editions. It is storage agnostic, which means that a virtual machine or its disk files can be replicated to a data store, regardless of it being a VMFS volume or an NFS mount. You can learn more about vSphere Replication from the book Disaster Recovery using VMware vSphere Replication and vCenter Site Recovery Manager, Abhilash GB, ISBN 9781782176442, Packt Publishing.