What's new on vSphere 6.5 and vSphere limits?

Even if VMware vSphere seems a minor release, there are many improvements, and changes that could be considered in the same way as a new release with new functionalities and new limits.

What's new in 6.5?

VMware vSphere 6.5 became generally available (GA) on November 15, 2016, several years after the initial release of version 6.0 (GA was on March 12, 2012), bringing several new features and improvements to the vSphere platform.

At a high level, the new version focuses on the following four main areas of innovation:

Simplified customer experience: There are several improvements in vCenter Server Appliance (vCSA) features and capabilities, with a new modern and truly multi-platform client (finally HTML5-based) and a simple REST based API for automation and integration
Comprehensive built-in security: It uses the well-known policy-driven approach also used for security aspects at scale to secure data, infrastructure, and access
Universal app platform: Following the VMware vision, vSphere 6.5 could be a single platform to support any application on any cloud as discussed previously
Proactive data center management: It has become predictive in order to address potential issues before they can become serious issues

At a technical level, the different improvements are as follows:

Scale enhancements: There are new configuration maximums to support even the largest application environments (see the next paragraph for the different numbers).
VMware vCSA: This is now the preferred type of vCenter and the core building block for vSphere. Not only does it now have the same features as the Windows version, but it has new specific functions—a native vCenter Server high availability solution, native vCenter Server backup and restore, migration tool from existing vCenter Server (also in a previous version) to vCSA.
VMware vSphere Update Manager (VUM): In vSphere 6.5, it has been fully integrated with vCSA. This integration eliminates the additional resources required for another VM, OS license, and database dependencies of the previous architecture. Integrated VUM leverages the vPostgres installation that is part of vCSA, but the data is stored using a separate schema.
VMware Tools: There are several improvements (such as digital signed ISO), but also some changes in supported OS and supported levels, including a bifurcation of VMware Tools for legacy and current guests.
REST APIs: These are simple, modern developer-friendly APIs to integrate your vSphere environment in other management platforms. Also, other CLIs have been extended and improved.
vSphere Client: This is a new HTML5-based GUI, similar to the VMware Host Client, that ensures fast performance, cross-platform and multi-OS compatibility. Note that the vCenter installer is now also supported on Microsoft Windows, macOS, and Linux operating systems without the need for any plugins.
Content library: This was introduced in vCenter 6.0, but with some limitations. Now it's possible to mount an ISO directly from the content library, apply a guest OS customization specification during VM deployment, and update existing templates.
Security at scale: This is a new policy-driven security framework that makes securing infrastructure operationally simple using the same approach as the already existing policy-driven storage.
Encryption: VM-level encryption protects unauthorized data access both at rest and in motion.
Audit-quality logging: There is enhanced logging that provides forensic information about user actions.
Secure Boot: This protects both the hypervisor and guest operating system by ensuring images have not been tampered with and preventing loading of unauthorized components.
Proactive HA: There is high availability capability that utilizes server health information and migrates VMs from degraded hosts before a problem occurs.
Cross-Cloud vMotion: Live migrates workloads between VMware-based clouds.
Virtual Volumes Replication: Extends Virtual Volumes (vVols) support (introduced in v6.0) with native array vVols replication.
Virtual NVM Express (NVMe) and others new controllers: With hardware version 13, you can use NVMe, SATA, SCSI, and IDE controllers in a VM.

All these features will be discussed in upcoming chapters of this book.

For more information, see the following links:

What's new with vSphere 6.5 Update 1?

VMware vSphere 6.5 Update 1 was released on August 2, 2017, and it adds some bug fixes, new features, and also a key additional change in the support and license of vCenter Server:

vCenter Server Foundation can now manage and support four ESXi hosts: Although three hosts could be enough for smaller environments, VMware has received feedback that three host environments were too small in some small and medium-sized business (SMB) cases. For this reason, vSphere 6.5 Update 1 is now increasing the number of hosts that vCenter Server Foundation will support from three hosts to four.
vSphere 6.5 general support has been extended: VMware understands that upgrading infrastructure can be a lengthy process. One consideration for whether or not to upgrade is how long the new product will be supported. VMware wants to make the customer's decision to upgrade easier by extending general support for vSphere 6.5 for a full 5 years. This means that support for vSphere 6.5 will now end on November 15, 2021.
Upgrade path: Direct upgrade from vSphere 6.0 Update 3 is now a supported path (more details on the upgrade path will be provided in Chapter 11, Lifecycle Management, Patching, and Upgrade).
Adds full support for ESXi on Mac Pro 6,1 hardware: Many customers and home lab users like to use Mac hardware in order to virtualize macOS in an officially supported manner. VMware vSphere 6.5 Update 1 adds full support for ESXi on Mac Pro 6,1 hardware. So, if virtualizing macOS is your thing, you can now do it with the latest hardware and without workarounds.

Also, there are some interesting enhancements, including the following:

vSphere Client now supports 90% of general workflows and features; the new HTML5-based vSphere Client can now support up to 90% of general workflows. This is welcome news as VMware pushes towards 100% parity between the various clients (considering also that Adobe has recently announced the end of Flash Player support by the end of 2020).
There is a new version of vSAN 6.6.1 with new capabilities, such as VUM support in order to manage vSAN software upgrades easily.
VMware vSphere 6.5 Update 1 is required to enable VMware Cloud on AWS.
New limits for vCenter Server in Linked Mode (see next paragraph, in the numbers specific for vCenter Server).

For more information, consult the release notes at https://docs.vmware.com/en/VMware-vSphere/6.5/rn/vsphere-esxi-651-release-notes.html.

Configuration maximums

As usual, the maximum numbers are published in a specific document and grouped by types. The vSphere 6.5 configuration maximums guide has already been updated with the new 6.5 U1 numbers (https://docs.vmware.com/en/VMware-vSphere/6.5/vsp-esxi-vcenter-server-651-configuration-maximums-guide.pdf). The most interesting numbers, as compared to the previous versions are described in the following paragraphs.

VM

With this new version of vSphere, there is a new version of the VM virtual hardware (version 13). Some limits and features (virtual RAM, NVMe controllers) are available only with the new virtual hardware 13; other limits are also valid for previous versions of virtual hardware. For more details refer to VMware vSphere Virtual Machine Administration guide at https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-789C3913-1053-4850-A0F0-E29C3D32B6DA.htm.

The following table summarizes some of the maximums numbers for each VM in the different version of vSphere:

	vSphere 4.0	vSphere 4.1	vSphere 5.0	vSphere 5.1	vSphere 5.5	vSphere 6.0	vSphere 6.5
Virtual CPU	8	8	32	64	64	128	128
Virtual RAM	255 GB	255 GB	1 TB	1 TB	1 TB	4 TB	6128 GB
Max VMDK size	2 TB – 512 B	2 TB – 512 B	2 TB – 512 B	2 TB – 512 B	62 TB	62 TB	62 TB
Virtual SCSI adapters	4	4	4	4	4	4	4
Virtual SCSI target	60	60	60	60	60	60	60
Virtual NICs	10	10	10	10	10	10	10
Virtual NVMe adapters	NA	NA	NA	NA	NA	NA	4

Table 1.1: Maximums numbers for each VM in the different version of vSphere

Those limits can change if you are using VMware FT to protect your VMs and the real limits, in this case, will be lower. More details will be provided in Chapter 13, Advanced Availability in vSphere 6.5.

Host ESXi 6.5

ESXi hosts limits remain mostly the same from version 6.0 but, of course, new hardware and new devices are now supported. Scalability remains quite similar compared with version 6.0, as summarized in the following table :

	vSphere 4.0	vSphere 4.1	vSphere 5.0	vSphere 5.1	vSphere 5.5	vSphere 6.0	vSphere 6.5
Logical CPU	64	160	160	160	320	480	576
Physical RAM	1 TB	1 TB	2 TB	2 TB	4 TB	12 TB	12 TB
NUMA nodes	NA	NA	8 nodes	8 nodes	16 nodes	16 nodes	16 nodes
Virtual CPU	512	512	2048	2048	4096	4096	4096
VMs	320	320	512	512	512	1024	1024
LUNs (iSCSI/FC)	256	256	256	256	256	256	512*
NFS mounts	64	64	256	256	256	256	256
LUN size	64 TB	64 TB	64 TB	64 TB	64 TB	64 TB	64 TB

Table 1.2: Scalability comparison with different versions

* The official document reports 512, but the Disk.MaxLUN advanced settings report 1024 on a ESXi 6.5 host.

In most cases, there is 2x increase from vSphere 5.5.

vCenter Server 6.5

The vCSA version of vCenter Server will now have the same limits as the Windows installable version (also with the embedded DB). And finally, the vCSA is now the first choice (the Windows version will be deprecated in the next releases), including some new features available only for it. The following table summarizes the different numbers from the different versions of vCenter Servers:

	vSphere 4.0	vSphere 4.1	vSphere 5.0	vSphere 5.1	vSphere 5.5	vSphere 6.0	vSphere 6.5
Hosts per vCenter	300	1000	1000	1000	1000	1000	2000
Hosts per data center	100	400	500	500	500	500	200
Hosts per cluster	32	32	32	32	32	64	64
VMs per cluster	1280	3000	3000	4000	4000	8000	8000
Powered on VMs	3000	10000	10000	10000	10000	10000	25000
Registered VMs	4500	15000	15000	15000	15000	15000	35000
Linked vCenter Servers	10	10	10	10	10	10	15

Table 1.3: Different numbers from the different versions of vCenter Servers

There is a 2x increase as compared to previous vCenter Server 6.0.

Some numbers have been increased with vSphere 6.5 Update 1 when you are using more vCenter in Linked Mode (that defines a vSphere domain):

Maximum vCenter Servers per vSphere domain: 15 (increased from 10)
Maximum ESXi hosts per vSphere domain: 5,000 (increased from 4,000)
Maximum powered on VMs per vSphere domain: 50,000 (increased from 30,000)
Maximum registered VMs per vSphere domain: 70,000 (increased from 50,000)

Why you should upgrade to vSphere 6.5 and why not?

Despite the number, vSphere 6.5 does not represent a minor release of vSphere 6.0 but a new major release (the same considerations were possible with vSphere 5.0, 5.1, and 5.5). For this reason, you have to plan carefully whether to upgrade and how to upgrade. Finally, with vSphere 6.5 Update 1, it is possible to upgrade from vSphere 6.0 Update 3 (this was not supported in the previous version of 6.5).

Customers who are still on vSphere 5.5 will need to be at least on vSphere 5.5 Update 3b in order to upgrade to vSphere 6.5 U1.

There are several changes but also new features and scalability properties that make vSphere 6.5 interesting for new environments, but also for existing customers.

Existing 6.0 customers already have the right license keys, while 5.x customers will need to have an active subscription in order to upgrade their 5.x license keys to version 6.x.

The main consideration to make with regard to the upgrade is that each new product (it does not matter whether it is a major or a minor release) brings new features, new code, and potentially maybe also new bugs (and the history of vSphere 5.1 and 6.0 has demonstrated that early adopters came across some issues). Of course, upgrading might also fix some existing bugs. But the maturity of a new release might not be the same as the previous versions. So, before upgrading, evaluate the new release by first using it in a dev or test environment or wait a few months to see the first feedback from the community and reported issues, and how and when they have been solved. Some prefer to wait for the first Update 1 version and, finally, it is here. To be honest, the code of the initial 6.5 release already seemed to be more mature than that of version 6.0 in its infancy.

You also have to consider all third-party code, included drivers or services, and kernel modules (for example, PernixData FVP is not compatible with 6.5) or switch extensions (after vSphere 6.5 Update 1, customers using third-party virtual switches such as the IBM DVS 5000v, HPE 5900v, and Cisco Nexus 1000v will need to migrate off of those switches prior to upgrading to any future release), vCenter plugins or integration with external software, for example, backup products. For third-party switches see KB 2149722—Discontinuation of third party vSwitch program at https://kb.vmware.com/kb/2149722.

Remember also that vSphere may be just a foundation of a bigger solution and architecture (as described before); in this case, you have to check every piece of software and hardware to match the compatibility and supported version.

More details on the upgrade procedure will be provided in Chapter 11, Lifecycle Management, Patching, and Upgrade.

Why upgrade?

There can be several reasons to upgrade vSphere to the latest version:

Extend the support and the life cycle of the product: VMware vSphere 5.5 will have an extended support and will reach end of general support in September 2018, VMware vSphere 6.0 on March 2020, and vSphere 6.5 on November 15, 2021.
Have a new product: It provides new features but also new hardware (and other new software) may require or have some benefit from this version.
New infrastructure functions: Such as the new high availability features; we will discuss all those functions in upcoming chapters.
New security functions: Some are really cool and unique, but data at rest protection using encryption could also be possible not only at hypervisor level (in this case only with 6.5) but also at storage level (also vSAN now has this capability).
Storage benefits: If you are using vVols, you can now have a native replication support (of course, if your storage vendor supports it in vSphere 6.5). If you are using vSAN, the only way to upgrade it and have new features is to upgrade vSphere.
New web client (vSphere Client): Finally, we have an HTML5 client (not 100% complete, but very close, at least for operational tasks) for the vCenter Server graphical management. For the ESXi host, there was already (starting from 6.0 Update 2). Note that you can add both the HTML5 clients to the previous version using Flings software (https://labs.vmware.com/flings).
New vCSA: The new virtual appliance for vCenter is definitely the first choice, due to the full capabilities and also for the new functions.

Why shouldn't you upgrade?

There can be few reasons to skip the upgrade to vSphere 6.5 which are as follows:

Is it compatible? You may have a software or hardware part that does not support this version. Note that from the next version of vSphere, several generations of servers will probably no longer be supported (for example, if you install ESXi 6.5 on a Dell 11g, it reports that the next version of ESXi will no longer support that processor).
Does it support existing servers? vSphere 6.5 drops the support to some old hardware and software. vSphere 6.5 no longer supports the following processors—Intel Xeon 51xx series, Xeon 30xx series, Xeon 32xx series, Xeon 53xx series, Xeon 72xx/73xx series.
Do you really need the new functions? If you are involved in a digital transformation, you will probably need the new platform (AWS for vSphere or vSphere for integrated containers management require the new version). But for SMBs, most of the new functions are not usable or useful yet.
Can you really use the new functions? Most of the new features are only for the Enterprise Plus edition (see the next paragraph for more details about the different editions).
Is it mature and stable enough? As mentioned previously, vSphere 6.5 seems a better code compared to previous version 6.0 (or also 5.1) when it was released in GA. Also, it has already been used in production environments for more than 6 months, with few bugs.

Features and editions of vSphere 6.5

VMware vSphere is licensed in different ways and different packages and bundles, usually identified by a stock keeping unit (SKU) code. There are some bundles (like vSphere with Operations Management Enterprise Plus), OEM, ROBO and VDI specific SKU, or other license models (such as ELA) that we will not consider and describe.

For more details, you can consult the official VMware (vSphere) licensing page at https://www.vmware.com/support/support-resources/licensing.html.

For VMware vCenter Server, the licensing model is quite simple per instance and with three different editions:

Product feature	vCenter Essential	vCenter Foundation	vCenter Standard
Host manageable	Max 3 ESXi with Essential or Essential Plus	Max 4 ESXi Standard or Enterprise	Unlimited ESXi Standard or Enterprise
vCenter High Availability (HA)	Not available	Not available	Only for the vCSA
vCenter Backup and Restore	Not available	Not available	Only for the vCSA
Linked Mode	Not available	Not available	X

Table 1.4: vCenter features across different editions

For ESXi, the license is entitled per socket (except in some specific bundle or SKU, such as ROBO or VDI) and there are different editions with different features.

Note that since June 30, 2016, the ESXi Enterprise, vSphere with Operations Management Standard/Enterprise editions are no longer available. Customers who already own Enterprise versions are not yet affected (their current editions will continue to be supported through the EOA of vSphere 6):

Product feature	ESXi Essential Plus	ESXi Standard	ESXi Enterprise Plus
VMware Integrated OpenStack	Not available	Not available	Support is sold separately
VIC	Not available	Not available	X
vMotion	X	+Cross-vSwitch	+Cross-vSwitch / Cross-vCenter / Long Distance / Cross-Cloud
Storage vMotion	Not available	X	X
vSphere HA	X	X	X
Proactive HA	Not available		X
vSphere FT	Not available	2-vCPU	4-vCPU
vSphere Replication	X	X	X
Virtual Machine Encryption	Not available	Not available	X
Virtual Volumes	Not available	X	X
VAAI, 3rd part multipath	Not available	X	X
Storage Policy-Based Management	Not available	X	X
DRS, DPM, Storage DRS	Not available	Not available	X
SIOC, NIOC	Not available	Not available	X
Distributed virtual switches	Not available	Not available	X
Host profile, Auto-Deploy	Not available	Not available	X

Table 1.5: Product features with different vSphere editions

For ESXi, there are also the Free Hypervisor and the Essential editions; both are quite limited in function (no cluster function at all) but do not have specific limitations on the resources at the level of a single host.

The Free Hypervisor edition does not include the VDAP API, that means no native backup capability for all the backup software that uses that interface, and does not include either the vCenter Agent, that means no way to manage from vCenter.

For SMB, there are two specific bundles—vSphere Essential and Essential Plus Kit, that combine one instance of vCenter Essential and six licenses (usable on a maximum of three hosts) of ESXi Essential or Essential Plus. Both bundles have an interesting price, making virtualization also possible for companies with budget constraints.

For enterprises, depending on the size and the business requirements, for ESXi licensing, usually Standard or the Enterprise Plus can be used.