Objective 1.2 – Secure ESXi and vCenter Server 2

To increase the security of ESXi, vCenter, and other vSphere components, you will need to use different approaches, as follows:

• Protecting the physical layer: For example, for the networking part, use dedicated VLAN for different traffic.
• Securing network communications: This at least applies to infrastructural components. By default, management traffic is already encrypted. Note that one new feature of vSphere 6.5 is the ability to also encrypt vMotion traffic.
• Applying the minimum privileges: Limit all the services, permissions, access to minimize the attack surface.

Objective 1.2 for VCP65-DCV and VCP6-DCV is quite different, due to the security and hardening changes from vSphere 6.0 to vSphere 6.5.

Hardening is a process that enhances the security of a system, a service, or an entire infrastructure, by reducing the attack surface and minimizing the possible vulnerabilities and related risks.

VMware has built in a set of Security Hardening Guides (https://www.vmware.com/security/hardening-guides.html), including one related to the vSphere environment. The vSphere 6.5 Security Configuration Guide is a spreadsheet file with several possible hardening actions and guidelines, each classified with a risk profile. There are also some example scripts, for enabling security automation.

The vSphere 6.5 Security Configuration Guide isn't a compliance tool; it can be used to reach compliance, but it's not automatically enforced. It's a set of guidelines that attempts to explain security risks, but there are other solutions for mitigating them.

The Security Guide contains in-depth information on how to secure ESXi hosts (https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-A706C6C6-DF07-455B-99B9-5B8F8580F1EB.html) and vCenter components (https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-8C5F5839-37EC-409E-8C46-C8AD146CBC73.html):
https://docs.vmware.com/en/VMware-vSphere/6.5/vsphere-esxi-vcenter-server-652-security-guide.pdf

Configure encrypted vMotion

This is a new option in vSphere 6.5 (but only for the Enterprise Plus edition), in order to secure the vMotion network traffic.

The vMotion encryption feature isn't simply an encrypting of the entire network channel for the vMotion traffic; it's a per-VM setting. There aren't certificates to manage or import on the infrastructural side.

This will be discussed later, in Objective 1.4, because it's related to the VM options.

On the infrastructural side, you will need to configure the proper key servers (as described in Objective 1.3).

Describe ESXi Secure Boot

Unified Extensible Firmware Interface (UEFI) is a replacement for the traditional BIOS firmware, and is supported for VM from virtual hardware 7.

Secure boot is part of the UEFI firmware standard, where the UEFI firmware validates the digital signature of the operating system and its bootloader, to ensure that the bootstrap sequence starts only a properly signed system, including drivers and applications.

Starting with vSphere ESXi 6.5, it's possible to have secure boot for both ESXi and VMs.

For ESXi, the secure boot can verify each VIB by using its digital sign. At boot time, the already validated ESXi VMkernel will validate each VIB against the firmware-based certificate:

Figure 1.11: ESXi secure boot

For more information on how to enable this feature, and also some possible issues (during the upgrade process, for example), see https://blogs.vmware.com/vsphere/2017/05/secure-boot-esxi-6-5-hypervisor-assurance.html.

For the secure boot options for VMs, see Objective 1.4.

Harden ESXi hosts

To increase the protection of ESXi hosts against possible attacks and unauthorized access, consider the following options:

• Limit user privileges and access: One aspect is using the RBAC model (described in Objective 1.1) to limit user privileges. But, you also have to use a centralized authentication, limit the authorized users, restrict access to ESXi management adapter, and enforce security policies (such as password expiration and password complexity).
• Limit shell access: ESXi shell and ESXi SSH access have several privileged accesses, and permit executing several commands from the CLI. For this reason, this type of access must be closed or limited. Lockdown mode (as described later) can be effective for limiting that type of access.
• Limit services: By default, ESXi only runs essential services, and any services that are not needed are stopped. Note that third-party services, such as some hardware vendor agents, should be limited, or at least validated.
• Limit network connections: ESXi has a built-in firewall (starting from ESXi 5.0), and, by default, it is closed on most ports. When you enable a service, it also opens the right ports. The personal firewall does not protect from DoS attacks, so keep your ESXi VMkernel interfaces on protected networks, and continue to use perimeter firewalls.
• Use secure connections: By default, most of the communications are secured by the SSL layer, and all weak ciphers are disabled (this can vary in the different builds of vSphere). Also, VMware vSphere 6.0 introduced a certification authority (described in Objective 1.3) to help with certification management.
• Update your environment: VMware Update Manager (VUM) can simplify host patching. With the VCSA, the vCenter management can also be simplified (using VAMI).
• Check the VMware Security Advisories (https://www.vmware.com/security/advisories.html): This site has a list of possible security vulnerabilities for VMware products, and related remediation or mitigation.

Also, in order to mitigate security risks in ESXi, there are some built-in security settings, as follows:

• Shell access: ESXi Shell and SSH are disabled, by default. It is usually safe to keep both ESXi Shell and SSH access disabled, preventing direct access to the ESXi CLI. Note that in this case, you can still use esxcli remotely, as well as other remote CLI tools!
• Firewall: Usually, there are a few ports open by default, and ports are automatically open on the firewall if there are some services that need specific ports. Although you can manually open ports or build custom ESXi firewall rules, try to keep the management automatic.
• Services: Following the minimum privilege approach, ESXi only runs required services, and new services are automatically started if a specific feature requires them.
• Secure protocols: By default, weak ciphers are disabled, and communications from clients are secured by SSL. Starting with vSphere 6.5, the TLS protocol versions 1.0, 1.1, and 1.2 are enabled, by default. Also, see VMware KB 2147469 (https://kb.vmware.com/s/article/2147469)—Managing TLS protocol configuration for vSphere 6.5.
• Web server: A custom Tomcat web service is used to provide access from the web client. The service has been hardened to improve its security.
• Bugs: VMware usually releases security patches, in case of possible security issues affecting ESXi (or other components). With VUM, you can easily apply those patches.
• Secure Boot: VMware ESXi 6.5 supports secure booting, as described previously.

Enable/configure/disable services in the ESXi firewall

As stated previously, only the required ports are open on the ESXi firewall. But, using the vSphere Web Client, it's possible to manage incoming and outgoing firewall rules. Usually, firewall rules are related to specific ESXi services.

It's possible to manage service settings and/or firewall rules in the Security Profile menu, under the Configure tab of each host:

Figure 1.12 ESXi security profile

The first part (Firewall) shows all active incoming and outgoing rules, with their corresponding firewall ports.

Firewall rules can be modified by clicking on the Edit button in the Firewall section:

Figure 1.13 Editing the security profile—firewall rules

You can enable or disable a specific firewall rule, and you can also specify which logical network address is authorized to use the selected service.

Until vSphere 6.0, it was also possible to build custom firewall rules, but not from the UI. For more information, see KB 2008226 (https://kb.vmware.com/s/article/2008226)—Creating custom firewall rules in VMware ESXi 5.x.

The second part (Services) shows all of the configured services, and their statuses. It's possible to manage them with the Edit button, in the Services section:

Figure 1.14: Editing security profile—services

In the Service Details section, you can see the status, and also perform some tasks:

• Manage the services status: Use the Start, Stop, or Restart buttons
• Define how services are started: With the Startup Policy, you can choose how each service must be started, with one of the following three startup policies:
  • Start and stop with host
  • Start and stop manually
  • Start and stop with port usage

Starting a service automatically opens the network ports that are required by the service.

For more information, see the vSphere 6.5 Security Guide (https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-9C8A0CD0-1664-4F21-B75A-541C03E37233.html).

Change ESXi default account access

ESXi permissions management has a local RBAC model, with some predefined local system roles, such as the following:

• Administrator role: With all privileges, like the administrator role in vCenter.
• Read-only role: Allows for viewing objects associated with ESXi host, but not making any changes, like the read-only role in vCenter.
• No access role: No privileges at all, like the no access role in vCenter. This is the default role for new users.

As with vCenter permissions, it's possible to define new custom roles.

ESXi roles and local users can be managed through the Host UI interface; both are located in the Security & users tab, in the Host | Manage menu:

Figure 1.15: ESXi user and role management

You can find local user management in the Users menu, and local role management in the Roles menu.

Local permissions can be managed via the Host | Action button, in the Permissions menu:

Figure 1.16: ESXi permissions

If a user exists in both ESXi host and SSO (or one of its identity sources), there can be two different users with the same login on the different authentication sources. The full login name must be login@domain to avoid confusions.

On each ESXi, there are some default local users:

• Root user: This is a built-in user with the administrator role. You can remove or change the role of the root user, but be sure to first add another user with the administrator role. Note that this is the only built-in user that is reported in the ESXi user management interface, but there are also other users.
• vpxuser user: This user is used by vCenter to manage ESXi hosts, after they have been connected. Note that this user also has an administrative role. This user must be managed by the vCenter Server; don't change it in any way (such as changing its password or permissions).
• dcui user: The primary purpose of this user is to configure hosts for lockdown mode, from the Direct Console User Interface (DCUI).

For audit purposes, be sure to assign named accounts to all accounts with the administrator role.

For more information, see the vSphere 6.5 Security Guide (https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-2215AADC-D4CD-49DD-AF92-65BED243D851.html).

The next section will describe how to add an ESXi to an AD domain, in order to use external users and groups.

Also, remember that if your ESXi host is managed by a vCenter Server, it might be better to manage authentication and authorization tasks through the vCenter permissions.

Add an ESXi Host to a directory service

To use centralized users (and groups) instead of local accounts, it's possible to join an ESXi to an AD domain. When the host is added to AD, the ESX Admins domain group has the administrator role on the ESXi host.

It's possible to join an ESXi host to an AD domain if it is a standalone host, or if it's managed by a vCenter Server. But, if your ESXi is already managed by a vCenter Server, authentication and authorization with the vCenter permissions might be enough; there's no need to also use ESXi roles.

If ESXi is managed by vCenter, you can use the vSphere Web Client to join ESXi to an AD domain. Use the Join Domain button in the Authentication Services menu, under the Configure tab of a specific host, as in the following screenshot:

Figure 1.17: Joining an ESXi to an AD domain

Enter the full AD domain name (in a DNS format), and the credentials of an AD user with enough permissions to join a computer to the domain.

To add a standalone ESXi to an AD domain, you will need the Host UI. In this case, just select the Host | Manage menu, go to the Security & users tab, and select Authentication.

For more information, see the vSphere 6.5 Security Guide (https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-4FD32125-4955-439D-B39F-C654CCB207DC.html).

Apply permissions to ESXi Hosts using Host Profiles

Host profiles are only available in the Enterprise Plus edition, and are used to provide a standard configuration for multiple ESXi hosts. You can create a new host profile from a reference host and apply the host profile to all hosts that share similar characteristics with the reference host (like a homogeneous hardware configuration).

To apply permissions to ESXi using a host profile, you need to modify it with the vSphere Web Client in Home | Policies and Profiles | Host Profiles. Select a specific host profile and click on the Configure tab, and then click on the Edit Host Profile... button, or select Edit settings in the host profile contextual menu:

Figure 1.18 Editing a host profile

The security setting and the permissions rules are in the Security and Services menu.

Note that, in the same menu, you can also configure ESXi firewall rules and domain service integration, as discussed previously.

For more information, see the vSphere 6.5 Security Guide (https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-FD142F1A-FE26-473E-BF09-AC2F84B15318.html).

Objective 8.2 will provide more information on how to create, manage, and apply host profiles.

Enable Lockdown Mode

To harden ESXi connected to a vCenter Server, one option is to use lockdown mode, which disables a direct connection to ESXi host. The host will only be accessible through the vCenter Server, or, depending on the lockdown mode, through the DCUI.

It's possible to modify lockdown mode in the host settings or from the DCUI (the usual method).

In vSphere 6.x, lockdown mode has different levels of protection; the following are the different configuration options available:

• Disabled: Lockdown mode is disabled.
• Normal: Lockdown mode is enabled, DCUI is not blocked, but the Host UI, ESXi shell, or ESXi SSH is disabled.
• Strict: Lockdown mode is enabled, and all local services are disabled (including the DCUI that is stopped). ESXi is only accessible through the vCenter Server.

You can configure ESXi lockdown mode from the vSphere Web Client, when you add a new host. It's also possible to change the setting later; in that case, select the Security Profile menu in the Configure tab of the desired ESXi.

Find the Lockdown Mode area (after Services), and click on the Edit... button, as follows:

Figure 1.19: Configuring lockdown mode

In vSphere 6.x, there is a new feature for lockdown mode: the Exception Users list. Those users (or solutions) will be excluded from lockdown mode (if Normal mode is used). Exception users cannot be managed from the DCUI.

From the DCUI, press F2 and log in, then select Configure Lockdown Mode and press Enter:

Figure 1.20: Configuring lockdown mode from the DCUI

For more information, see the vSphere 6.5 Security Guide (https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-88B24613-E8F9-40D2-B838-225F5FF480FF.html).

Control access to hosts (DCUI/Shell/SSH/MOB)

Lockdown mode, as described previously, can be used to limit and control access to ESXi by using DCUI (in strict mode), or ESXi Shell and ESXi SSH.

There are also other ways to limit access to an ESXi host. For example, there is the Managed Object Browser (MOB) interface, which provides a view of the VMkernel objects. Usually, it's recommended to disable it in production systems.

Starting with vSphere 6.0, the MOB interface is disabled by default. However, for some specific cases, you may need to enable it again.

You can enable and disable the MOB interface by using the vSphere Web Client. Select the desired host, choose the Configure tab, and go to the Advanced System Settings menu (in the System section).

Check the value of Config.HostAgent.plugins.solo.enableMob, and change it as needed.

Harden vCenter Server

Depending on your type of vCenter deployment, you may have internal or external PSC (see Objective 1.3), and you may also have Windows-based servers or Linux-based virtual appliances.

Note that, in the new major release of vSphere, there will only be the virtual appliance (or vCenter Virtual Appliance (VCSA)) option, and the Windows-based installation will no longer be available.

There are some generic security best practices that are valid for both versions, as follows:

• Use named accounts, and minimize permissions
• Monitor privileges of vCenter Server administrator users
• Limit vCenter Server network connectivity
• Verify certificates
• Keep the vCenter OS and services updated

By using the VSCA, from VMware for vSphere 6.5 and later, you can use the same VM hardening suggestions (see Objective 1.4). But note that the VCSA is already based on a hardened operating system (in vSphere 6.5, the VCSA is based on the VMware PhotonOS platform).

By default, on VCSA, the shell access is disabled. A remote shell with SSH can be enabled during the deployment, but note that by default this shell has a limited set of commands (but it's also easy to enable the full shell).

The PSC security best practices are quite simple:

• Check password expirations: Remember that the default SSO password lifetime is 90 days.
• Use NTP for time sync: All systems must use the same time source. Time synchronization is essential for authentication, and also for certificate validity.

For more information, see the vSphere 6.5 Security Guide (https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.security.doc/GUID-8C5F5839-37EC-409E-8C46-C8AD146CBC73.html).

Control datastore browser access

Data store browsing is provided by different roles, but it is mainly provided by the Datastore.Browse privilege. It can be dangerous, because users with this privilege can view, delete, copy, upload, or download files directly from data stores.

Be sure to assign this privilege only to users or groups that really need it, in order to follow the minimum privilege principle.

VM file encryption (see Objective 1.4) can help to minimize some risks in data confidentiality if a user can browse the data store.

Create/Manage vCenter Server Security Certificates

Network communications between vSphere components are usually encrypted using TLS/SSL protocols. At a minimum, all management traffic is secured by default.

However, in vSphere 5.5 and earlier, the TLS/SSL communications were only authenticated with a username, password, and basic certification verification (thumbprint). Starting with vSphere 6.0, vCenter uses certificates for authentication, to increase the security of communications.

VMware vSphere 6.x supports the following certificate modes:

• VMware Certificate Authority (default): The PSC acts as a top-level CA (or as an intermediate CA) and provisions certificates to ESXi hosts and other endpoints that require them.
• Custom Certificate Authority: In this case, custom certificates signed by third-party or enterprise CAs are used. Unless you change the certificate mode to Custom Certificate Authority, the PSC might replace custom certificates.
• Thumbprint Mode: Certificates are checked for the correct format, but without verifying the validity of the certificate. This mode was used until vSphere 5.5, but it is still available as a compatible option in vSphere 6.x.

For more information about the VMware Certification Authority, see Objective 1.3.

Control MOB access

MOB access was already discussed in the section on hardening ESXi hosts. The Managed Object Browser (MOB) is a web-based server application that can permit access to some data and represent a security risk.

Remember that, starting with vSphere 6.0, the MOB is disabled by default.

Change vCenter default account access

User management was already discussed in Objective 1.1.

The default role for new users or new identity sources is No access. However, there are also the local users (of the vCenter OS), which can have some privileges and, for example in previous vCenter Server Windows based system local administrator was also added as a vCenter Administrator.

Starting with vSphere 6.0, the local administrator of the PSC (user root for VCSA, or administrator for Windows-based) does not have the full administrative vCenter permissions.

For the VCSA, the root user (the password of which is defined during the installation) can, of course, access the VAMI interface.

Restrict administrative privileges

Due to the minimum privilege principle, not all administrative users must have the administrator role. A good practice is to create a custom role with an appropriate set of privileges, and assign it to other administrators.

For more information and an example, see Objective 1.1.

Understand the implications of securing a vSphere environment

During the hardening process, you have to find the right balance between security and manageability. Some changes related to increasing the security of the vSphere environment can have an impact on its manageability (for example, see the Enabling lockdown mode section).