This book aims to be useful for both new and seasoned VMware NSX for vSphere administrators. It is intended to be used by those that have never deployed NSX and by those that have it deployed already but are looking to leverage newer or advanced functionality. Intermediate networking and virtualization knowledge is assumed and is essential to understanding deployment of NSX into your environment.

Before we begin serving the main recipes of our cookbook, we will first provide an overview of what VMware NSX for vSphere is and what functionality it provides over traditional networking models.

VMware NSX for vSphere is a core component of the VMware Software-Defined Data Center (SDDC); it is the component that enables network virtualization. Network virtualization provides a layer of abstraction over the physical network using a VXLAN network overlay. With NSX, network operations are now independent of the physical hardware, and functions such as logical firewalls, load balancers, logical routers, logical switches, and virtual private networks can be provisioned, modified, or torn down as part of an automated workflow.

VMware NSX has four licensing editions: standard, advanced, enterprise, and remote office/branch offices (ROBO). Each licensing tier provides distinctive functionality, available per CPU socket on a perpetual basis at the vSphere cluster level.

The standard and advanced editions are also available as per 100 users in a pack basis to align with virtual desktop deployments (vSphere for desktop). The enterprise edition is also available on per-VM term basis. You can upgrade from standard to advanced/enterprise and from advanced to enterprise.

Like vSphere licensing, VMware NSX is licensed per CPU socket. If you have a separate Management vSphere Cluster that is used for Infrastructure VMs and are not planning to protect it with the NSX Distributed Firewall or place NSX Edge Service Gateways onto it, you are not required to license the CPUs on that Management vSphere Cluster. The Compute vSphere cluster and Edge vSphere cluster need to be licensed.

From your vSphere inventory you will need to do the following:

1. Determine how many CPU sockets you need
2. Determine the NSX features required
3. If you are planning to integrate third-party partner solutions with NSX (http://www.vmware.com/products/nsx/technology-partners.html), check whether a specific NSX feature is required

4. Choose the NSX edition based on the features required

The following sub-sections will detail the different tiers of NSX licensing and the features available in each. From there, how to evaluate and purchase VMware NSX will also be detailed.

The four tiers of licenses are as follows:

• Standard edition
• Advanced edition
• Enterprise edition
• ROBO

The features available in each edition are as follows:

There are two ways to evaluate VMware products:

• Deploy NSX in your environment and use an evaluation license for a limited time

• Use VMware Hands-on Labs (http://labs.hol.vmware.com/) to experience VMware NSX in a virtual lab environment:
  • VMware NSX Hands-on Lab Intro (http://www.vmware.com/go/try-nsx-en)
  • VMware NSX Hands-on Lab Advanced (http://www.vmware.com/go/try-nsx-adv-hol)

There are support and subscription plan options that you can purchase in addition to the product:

• Basic support: 12 hours a day technical support during business hours
• Production support: 24 hours (Severity 1), seven days a week support

The production support plan is recommended for production and critical environments. If you need higher-level support above production grade, additional support options such as Business Critical Support (BCS) or Mission Critical Support (MCS) can be purchased on top of production support. For more information on VMware support offerings, see https://www.vmware.com/support/services.html.

VMware vRealize Log Insight is a log management engine that collects logs from a number of different sources and provides rich dashboards and search functionality.

Log Insight is available for NSX at no additional charge, you are entitled to one Log Insight CPU per NSX CPU license. The support and subscription is included with the NSX purchase. It is a fully functioning version of Log Insight but limited to vSphere and NSX data sources and content packs only. If you need more data sources and content packs, additional Log Insight licenses are required.

There are several tools for monitoring VMware NSX. Some of these tools are built directly into the NSX platform, and others are separate feature-rich VMware products. These tools are as follows:

• VMware NSX built-in tools
• vRealize Network Insight

For more information about the VMware NSX Neutron plugin license editions for VMware integrated OpenStack, see VMware KB 2145269 (https://kb.vmware.com/kb/2145269).

Similar to the requirements of a VMware vSphere solution, choosing the correct hardware is still an important part of any NSX deployment; therefore, you need to follow the same process that you did for vSphere to ensure the hardware you are deploying is on the VMware Compatibility Guide (http://www.vmware.com/resources/compatibility/search.php).

The compatibility guide does not only list the supported servers, but you need to also check if your network interface card (I/O devices) is supported and features such as VXLAN Offload and Receive Side Scaling are also supported.

VXLAN Offload is akin to TCP segmentation offload (TSO), but compared to TSO, which is designed for TCP packet headers, VXLAN encapsulates the original (source) packet from a virtual machine into a user datagram protocol (UDP) packet with its own unique header, known as the VXLAN header. Placing this additional header onto a packet invalidates traditional offloading mechanisms in-place and therefore increases load on the CPU as additional CPU cycles are needed to encapsulate and decapsulate every VXLAN packet. VXLAN is covered in greater detail in Chapter 2, Configuring VMware NSX Logical Switch Networks.

Receive Side Scaling (RSS) is a technique the Network Interface Card (NIC) employs to ensure that data processing for a particular connection is balanced across multiple CPU cores. Without RSS, all connections would be handled by a single CPU core, which can adversely affect network performance.

In this recipe, we will download the installation media for NSX for vSphere. The installation media comes in the form of an open virtual application (OVA) that is distributed through the VMware downloads site (https://my.vmware.com/web/vmware/downloads).

To download NSX for vSphere, the following prerequisites must be satisfied:

• Valid VMware software entitlements that enable you to download the installation media
• Access to the VMware downloads website
• Access to VMware software manager. Download and install VMware software manager first (https://www.vmware.com/go/download-software-manager-en)
• VMware product interoperability matrix has been consulted so you know which version is compatible with your environment

The following sections will explain how to check that your infrastructure supports the version of NSX you are implementing and how to obtain the download media.

In this section, we will check to make sure the version of NSX we are deploying is compatible with the other VMware solutions in our environment.

1. Navigate your web browser to the VMware product interoperability matrix webpage (http://www.vmware.com/go/interop)
2. Select your vSphere solution as the first solution
3. Add VMware NSX for vSphere as your second solution

4. Add any other solutions that are specific to your environment
5. Ensure all solution versions are compatible with one another before proceeding to download the NSX installation media

The following screenshot shows the official VMware product interoperability matrices that should be referenced before downloading NSX for vSphere:

In this section we will download the installation media from the VMware downloads website as follows:

1. From your web browser, navigate to the VMware downloads website (https://my.vmware.com/web/vmware/downloads).
2. Scroll down to the Networking & Security menu item and click on Download Product

3. Click on go to Downloads against your licensed tier for VMware NSX for vSphere 6.3.1 or whichever version is compatible with your environment
4. Click on Download Now

In this section, we will download the VMware NSX installation media using the VMware Software Manager, in contrast to a manual download via the downloads website:

1. Open the VMware Download Service application:

2. Click on the VMware vSphere software suite
3. Select VMware vSphere 6.5
4. Select the licensing tier of your vSphere environment
5. On the VMware NSX for vSphere menu pane, select the download button:

To make sure that your vSphere and NSX version is supported by VMware, check the VMware life cycle product matrix (http://www.vmware.com/go/lifecycle). This list contains a list of unsupported products as well.

Deploying the NSX Manager virtual appliance is the first step to enabling network virtualization in your vSphere environment. In this recipe, you will go through the steps to enable your environment for NSX.

The following diagram depicts the logical process of enabling your environment for network virtualization, and the first four steps will be covered in this chapter:

Before deploying NSX Manager, the following prerequisites need to be satisfied:

• Static IP address and portgroup for NSX Manager
• Firewall ports open between NSX Manager, vCenter server, and ESXi VMKernel 0 Interface on each host (refer to Appendix for a complete list of ports)
• Forward and reverse DNS entries for NSX Manager
• NTP server is accessible; minimum of four is recommended for accurate time
• Shared datastore for the appliance to be deployed onto
• Satisfy minimum requirements for NSX Manager
• Fill in the following table before deployment (removing prefilled data to reflect your environment):

The following steps will detail how to deploy the NSX Manager appliance:

1. Log into the vSphere Web Client
2. Select Hosts and Clusters, right-click on the target cluster and select Deploy OVF Template

3. Select Local File and locate the NSX Manager OVA downloaded earlier; click on Next
4. Type in the Name of the virtual appliance and click on Next
5. Select the vSphere cluster and resource where you want to deploy NSX Manager and select Next
6. Review details, Accept license agreements and click on Next
7. Select the shared datastore of where you want the virtual appliance to be deployment onto
8. Select the VLAN-backed portgroup as defined earlier and click on Next
9. Fill in the template details as highlighted in the preceding table and click on Next
10. Ensure all details are correct and click on Finish:

When you first deploy the NSX Manager, it creates a self-signed certificate. Using a self-signed certificate is generally not a recommended security practice. It is recommended to deploy a signed certificate from your internal certificate authority. NSX Manager supports two ways of deploying a signed certificate, which are as follows:

• Certificate signing request to a Certificate Authority (CA)
• Importing a PKCS#12 certificate archive (bundle) onto the NSX Manager, which includes the private and public key for NSX Manager and certificate chain of any subordinate CAs in your environment

In the following recipes, we will explore how you can create a certificate signing request on NSX Manager and how to import a PKCS#12 certificate bundle onto the NSX Manager.

A Certificate Signing Request (CSR) is the first part in a three-step process; this process involves the following steps:

1. The NSX Manager creating a CSR
2. The CSR is sent as a request to the certificate authority, which then signs the certificate and sends back a signed certificate
3. Importing the signed certificate into the NSX Manager

The procedure to complete a certificate signing request is as follows:

1. Log into NSX Manager via your web browser
2. Click on Manage Appliance Settings
3. Click on SSL Certificates

4. Click on Generate CSR and follow the prompts as per the following screenshot:

5. Click on OK and select Download CSR
6. Send the CSR file to your security administrator and get the certificate signed
7. With the returned certificate, click on Import so you can import the correct certificate into the NSX Manager
8. Reboot the NSX Manager to complete the process of importing a signed certificate into the NSX Manager

Importing PKCS#12 into the NSX Manager is used when the certificate signing was not completed using the CSR method outlined in the previous recipe. The PKCS#12 format is typically used in scripted installations of NSX Manager and other components. If a CSR was not generated by the NSX Manager itself, it is required that the PKCS#12 archive is imported into NSX Manager.

The PKCS#12 archive generally consists of the following:

• A signed server certificate
• A private key for the signed certificate
• Root and intermediate certificate authority public keys

The PKCS#12 is also password-protected, so it's important to have the password before attempting to import the PKCS#12 archive into NSX Manager.

In some cases, the received signed certificate may not be in the PCKS#12 format. In this event, you must convert the certificates into the PKCS#12 format for import into the NSX Manager. This can be achieved using openSSL (https://www.openssl.org/), and the command to achieve this is as follows:

openssl pkcs12 -export -out server.p12 -inkey server.key -in server.crt -certfile CACert.crt  

The procedure to import the PCKS#12 archive is as follows:

1. Log into the NSX Manager via your web browser
2. Click on Manage Appliance Settings
3. Click on SSL Certificates
4. Click on Upload PCKS#12 Keystore and browse to the file
5. Enter the password for archive and click on Import
6. Reboot the NSX Manager to complete the process of importing the signed certificate

Once the NSX Manager appliance has been deployed and is accessible via https://nsxmgr-01a.corp.local, the next step is to register the NSX Manager as a solution against your vCenter deployment. NSX Manager and a vCenter server have a 1:1 relationship, and it's important to ensure that no other NSX Manager has previously been registered.

The following are things you need to consider before pairing the NSX Manager with the vCenter server:

• Solution interoperability has been verified
• vCenter server and vSphere environment are in a healthy state
• Platform Services Controller (PSC) Fully Qualified Domain Name (FQDN) can be resolved
• vCenter server FQDN can be resolved
• vCenter and PSC time settings are verified
• A service account with administrator role in vCenter has been created for the NSX Manager to use for registration; for further information refer to Chapter 9, Managing User Accounts in VMware NSX
• TCP port 443 connectivity is required from the NSX Manager to the platform services controller and the vCenter server

The following section describes the steps to integrate NSX Manager with the vCenter server and the platform services controller, which are the first steps in enabling your environment for NSX.

The following are the steps to pair the NSX Manager with the vCenter server:

1. Log into the NSX Manager web administration page: https://nsxmgr-01a.corp.local
2. Navigate to Manage | NSX Management Services, and on the Lookup Service URL click on Edit

3. Type the Lookup Server Host as the PSC FQDN or vCenter Server FQDN if using an embedded PSC
   1. For SSO Administrator Use Name, use the service account credentials defined
   2. Click on OK to complete
   3. When presented with the Trust Certificate dialog box, verify the SSL certificate thumbprint and click on Yes:

In this section we will register the NSX Manager with the Platform Services Controller for Single Sign-On services:

1. Navigate back to the NSX management service web page on the NSX Manager web administration page: https://nsxmgr-01a.corp.local
2. On the vCenter Server menu, click on Edit:
   1. Type the vCenter Server FQDN
   2. Type the service account credentials for the vCenter Service account and click on OK:

1. 3. When presented with the Trust Certificate dialog box, verify the SSL certificate thumbprint and click on Yes

The NSX Manager registers the com.vmware extension. This extension is installed on the vSphere web server as a plugin. When the plugin is installed onto the vSphere web server, any users that were logged in during integration will need to log out of the vSphere Web Client before they can start to consume the Networking & Security interface.

If the event registration fails with the platform services controller, check the following commons issues first:

• NTP Synchronization (time) for NSX Manager, platform services controller, and vCenter server is correct and aligned
• DNS resolution for all components
• Firewall ports are open if the NSX Manager and the PSC/vCenter server are separated in different security zones

As described in choosing the right VMware NSX for vSphere edition, this section will describe the process of applying the license you have procured to utilize the features of NSX.

Things to verify before applying the NSX for vSphere license:

• Correct license procured for installation of NSX
• NSX has been integrated as a solution with your vSphere deployment

Perform the following steps to apply the NSX license to your installation:

1. Log into the vSphere Web Client and click on Administration
2. Click on Licenses under the Licensing section on the sidebar
3. Select the Licenses tab and click on the plus sign:
   1. Enter your license key and click on Next
   2. Create a descriptive name for your license and click on Finish

4. Next, select the Solutions tab and select the NSX Installation:
   1. Navigate to Actions | Assign License
   2. Select the license you added earlier and click on OK

The NSX controller cluster is an integral part of any NSX for vSphere deployment; the NSX controller cluster is responsible for:

• Managing the vSphere hypervisor routing and switching modules
• Managing the ARP table, MAC table, and VXLAN network identifier (VNI) information of the entire vSphere for NSX deployment
• Distributed Logical Router:
  • Interfaces
  • Layer 2 Bridging Tables
  • Routes

The following are things to consider before deploying the NSX controller cluster:

• The controller cluster has three controllers in total and must be deployed in a cluster of three.
• Each controller node should reside on a separate ESXi host; DRS anti-affinity rules should be used to enforce this rule. It is generally recommended to deploy controllers on a vSphere cluster with a minimum of four ESXi hosts.
• Sufficient resources (vCPU, memory, and storage) on the vSphere cluster.
• NSX controller nodes should be deployed onto shared storage that is highly available.
• Each NSX controller requires an IPv4 address; these addresses are allocated via the NSX IP pool construct.
• NSX controllers require connectivity to NSX Manager and vSphere management VMKernel IP addresses.
• NSX controller should reside on a VLAN-backed PortGroup.

The NSX Controller IP Pool requires the following details prior to configuration. You can change values to suit your environment:

In the following sub-sections, we proceed to start deploying the NSX controller cluster, which is required for the logical networking components in NSX.

Before deploying the NSX controller cluster, an IP pool must be configured to reserve three IPv4 addresses on the network:

1. In the vCenter Web Client, navigate to Networking & Security | NSX Managers | NSX Manager
2. Select IP Pools and click on the plus sign
3. Fill in the details as per the preceding table and click on OK

The IP Pool can be configured during deployment of the NSX controller cluster as well in the event it is not configured beforehand.

In this section, we will deploy each of three NSX Controllers on our vSphere cluster:

1. In the vCenter Web Client, navigate to Networking & Security | Installation
2. Under the NSX controller nodes menu pane, click on the plus sign
3. Fill in the NSX controller details for the first node as follows and then click on OK:

4. After the first controller is deployed, repeat steps 1 to 3 for the remaining two controllers

Once all the controllers have been deployed, you should see the following displayed under the Installation tab in Networking & Security, with the green boxes indicating healthy connectivity between each of the peers in the controller cluster:

DRS anti-affinity rules are required to ensure that the NSX controllers do not reside on the same physical host and are kept separate on dedicated ESXi hosts. This is to ensure in the event a ESXi host goes down where all three controllers are potentially running as guest VMs, the entire control plane for logical networking is not lost. If two controllers are lost, then the remaining controller goes into read-only mode until a cluster majority is restored.

It's important to note that the underlying infrastructure should still be designed for HA and resiliency, which includes compute/network/storage.

Configuring DRS anti-affinity rules via the vSphere web client:

1. In the vCenter Web Client, navigate to Hosts and Clusters | Management Cluster | Manage | Settings | VM/Host Rules
2. Click on Add...
3. Choose Type as Separate Virtual Machines

4. Add NSX controller virtual machines and click on OK:

You can also configure DRS anti-affinity rules using PowerCLI. To configure via PowerCLI, you will need to ensure PowerCLI has been deployed and installed locally on your system. Perform the following steps to configure the DRS rules via PowerCLI:

1. Open the PowerCLI terminal up.
2. Type Connect-VIServer -Server VCENTER_SERVER, which will connect your PowerCLI session to the vCenter server you are working on.

3. Next, we want to retrieve the NSX controller virtual machines and store it as a variable, $nsx_controllers, using the get-vm PowerCLI cmdlet. The following code snippet demonstrates the command:

$nsx_controllers = get-vm | ? {$_.name -like "NSX_Controller*"}  

4. Next, using the New-DRSRule cmdlet, we will configure the anti-affinity DRS rule on the RegionA01-MGMT01 vSphere cluster using the following command:

New-DrsRule -Name nsx-controller-anti-affinity -Cluster RegionA01-MGMT01 -KeepTogether $false -VM $antiAffinityVMs  

In the following sub-sections, placement of the NSX Controllers and Controller password configuration will be discussed in greater detail.

The controller cluster is deployed in a group of three. Each controller node can only be deployed onto a vSphere cluster that is part of the vCenter inventory that the NSX Manager you are configuring is paired with. In large environments with multiple vCenters, it is not uncommon for the vCenter server and NSX Manager to be deployed onto a dedicated vSphere cluster that is managed by an independent vCenter server that is deemed as management. In this scenario, the NSX controller cluster cannot be deployed onto the dedicated management vSphere cluster.

The NSX controller password must meet the following criteria:

• It must not contain the username as a substring
• A character must not be repeated consecutively more than three times
• It must be at least 12 characters long and must follow three of the following four rules:
  • It must have at least one uppercase letter
  • It must have at least one lowercase letter
  • It must have at least one number
  • It must have at least one special character

In the event that the NSX controller password is forgotten, it can be easily changed using the following steps:

1. Log into the vSphere Web Client
2. Click on the Networking & Security tab and then navigate to Installation | Management:
   1. Under the NSX Controller nodes menu, select Actions
   2. Click on Change Controller Cluster Password
   3. Type a new password following the preceding guidelines and click on OK:

Preparing a vSphere cluster for NSX does two things:

1. It installs NSX Kernel modules on each ESXi host, which is a member of the vSphere cluster
2. It builds the NSX control-plane and management-plane fabric

NSX Kernel modules are packaged as VMware installation bundles (VIBs) and provide functionality such as distributed routing, distributed firewall, and VXLAN bridging.

To get ready for installation, ensure that the following prerequisite tasks have been completed:

• DNS forward and reverse names have been created for all ESXi hosts and are resolvable
• Firewall Ports between all management components are open
• vCenter Update Manager Service, if in use, is operational
  • Ensure that the EAM service is operational
  • Ensure that the NTP settings are checked across all ESXi hosts and are updating time correctly

Perform the following steps to start the installation of the NSX VIBs onto your first vSphere cluster; we will be enabling it on vSphere Cluster RegionA01-COMP01 to begin with:

1. In the vCenter Web Client, navigate to Networking & Security | Installation | Host Preparation
2. Select the vSphere Cluster RegionA01-COMP01
3. Click on the COG wheel and select Install:

Each ESXi host in the cluster will now download the VIBs from vCenter Server, where they were downloaded from NSX Manager and cached when NSX was registered as a solution. Depending on the number of hosts in the vSphere cluster, this process will take a few minutes to complete. Once the installation has completed, you will be presented with a screen like the one shown in the following screenshot:

The following figure depicts the management, control, and data plane components that make up an NSX implementation. Each has an important part to play in enabling ESXi for the Distributed Firewall and VXLAN. In this section, we will explore the interaction among the various components:

• vCenter server: This is the management component of the vSphere environment and is where the networking and security components of an NSX environment are all managed from.
• NSX Manager: This is the management plane of the NSX implementation. It integrates directly with vCenter and manages both the NSX controller cluster and the ESXi hosts. The NSX Manager is also responsible for pushing distributed firewall rules to each host that is prepared for the distributed firewall. In addition, the NSX Manager is also the API entry point for NSX operations via the REST protocol.
• ESXi Agency Manager (EAM): This is part of the vCenter deployment; it is responsible for installing the VIBs to each of the hosts.

When you prepare a vSphere cluster for NSX, the VIBs are copied directly from NSX Manager and cached by EAM. The EAM will then track the installation of each VIB onto each host in the vSphere cluster. If the VIB is not present, it is installed without the ESXi host requiring a reboot, and if it is present, a reboot is required to complete the upgrade.

Once the installation of VIBs has been completed, each ESXi host will have active TCP connections to the NSX Manager and NSX controller cluster. The connection to the NSX Manager is from the vsfwd daemon running on the ESXi host via the RabbitMQ message bus. The connection to the NSX Controller cluster is from the netcpa daemon running on the ESXi host via an SSL connection (TCP Port 1234). It is important that both channels of communication are active and can be checked via the communication channel health from each host, which is covered in a subsequent section:

When enabling a vSphere cluster for NSX in a brownfield environment, it is important to be cognizant that any preconfigured DFW firewall rules have the potential to impact virtual machines on the newly-configured vSphere cluster. Therefore, it is extremely important to ensure that the default Distributed Firewall rule remains as allow any any. Changing to deny before defining rules for allowing legitimate traffic from/to virtual machines will cause traffic blackholing.

As a best practice, vCenter server and virtual machines that require promiscuous mode should be excluded from the DFW if you are not planning to protect them. To learn how to exclude virtual machines from the DFW, refer to Chapter 6, Configuring VMware NSX Distributed Firewall (DFW) and SpoofGuard.

Installation of NSX VIBs that enable the Distributed Firewall and VXLAN are essential for a working NSX environment. This section will investigate how to manually verify that each VIB is installed correctly and whether communication to both the NSX controller cluster and NSX Manager are present.

The first control plane communication that we are concerned with is from the NSX Manager to each ESXi host via TCP port 5671. This port is reserved for the Rabbit MQ Message bus to the vsfwd daemon running on each host after the VMware Service Insertion Platform (VSIP) VIB installation, which is the Distributed Firewall kernel module. The NSX Manager uses the message bus to publish firewall rules down to each ESXi host. The ESXi host then applies them to vNICs of virtual machines that are running on top of its hypervisor.

The second control plane communication that is expected from each ESXi host is an open connection to each of the NSX controllers deployed. The NSX controller cluster is responsible for control plane information for ARP/MAC/VTEP tables. It is also used to program routes received on the Distributed Logical Router Control VM to each host (more on this in Chapter 2, Configuring VMware NSX Logical Switch Networks). From each host, we expect the netcpa daemon to have an active connection to the controller cluster on TCP port 1234.

To manually verify control-plane communication and VIB installation, you will need the following access to the following NSX components:

• SSH access to NSX Manager
• SSH access to each NSX controller
• SSH access to ESXi hosts that were prepared for NSX

You would not be expected to check communication of each and every host in your environment, as this can become unwieldly. However, this section is included for you to understand what the expected communication is, but in large deployments you would check the communication channel health per vSphere cluster as depicted in the earlier section.

To check whether the NSX VIBs have been installed successfully is crucial. The upcoming section details how to do this manually on an ESXi host and how to check NSX component communication.

In this section we perform manual verification that the VIBs have been successfully installed.

1. SSH onto an ESXi host.
2. Check whether VXLAN VIB modules have been installed by executing the following command:

esxcli software vib get --vibname esx-vxlan 

3. You will receive an output similar to the following:

[root@vSphere:~]    esxcli software vib get --vibname esx-vxlan
VMware_bootbank_esx-vxlan_6.0.0-0.0.4987429
   Name: esx-vxlan
   Version: 6.0.0-0.0.4987429
   Type: bootbank
   Vendor: VMware
   Acceptance Level: VMwareCertified
   Summary: Vxlan and host tool
   Description: This package loads module and configures firewall for vxlan networking.
   ReferenceURLs:
   Creation Date: 2017-01-27
   Depends: esx-base >= 6.0, esx-base <= 6.5.0, nsx-api <= 2.1, vmkapi_2_3_0_0
   Conflicts: nsx-api = 2
   Replaces: esx-traceflow, esx-dvfilter-switch-security, esx-bfd
   Provides: com.vmware.vxlan = 1.0.0.0-nsx, com.vmware.switchsecurity = 1.0.0.0, com.vmware.traceflow = 1.0.0.0, com.vmware.bfd = 1.0.0.0
   Maintenance Mode Required: True
   Hardware Platforms Required:
   Live Install Allowed: True
   Live Remove Allowed: True
   Stateless Ready: True
   Overlay: False
   Tags:
   Payloads: esx-vxlan

4. If the module has been installed correctly, you should see open TCP connections on port 1234 with the following command:

esxcli network ip connection list | grep 1234

An example is included below that shows the connection as established to each of the three NSX controllers from the point of view of an ESXi host:

5. To see which NSX controllers the host is configured to communicate with, execute the following command:

cat /etc/vmware/netcpa/config-by-vsm.xml

The following screenshot provides a truncated output of the command and its expected output.

6. Check if VSIP VIB modules have been installed by executing the following command: esxcli software vib get --vibname esx-vsip:

7. You will receive an output similar to the following:

[root@vSphere:~] esxcli software vib get --vibname esx-vsip
VMware_bootbank_esx-vsip_6.0.0-0.0.4987429
   Name: esx-vsip
   Version: 6.0.0-0.0.4987429
   Type: bootbank
   Vendor: VMware
   Acceptance Level: VMwareCertified
   Summary: vsip module
   Description: This package contains DFW and NetX data and control plane components.
   ReferenceURLs:
   Creation Date: 2017-01-27
   Depends: esx-base >= 6.0, esx-base <= 6.5.0, nsx-api <= 2.1, vmkapi_2_3_0_0
   Conflicts: nsx-api = 2
   Replaces: esx-vdpi
   Provides: vsip = 1.0.0-0
   Maintenance Mode Required: True
   Hardware Platforms Required:
   Live Install Allowed: True
   Live Remove Allowed: True
   Stateless Ready: True
   Overlay: False
   Tags:
   Payloads: esx-vsip 

8. If the module has been installed correctly, you can open TCP connections on port 5671 with the following command:

esxcli network ip connection list | grep 5671

The following screenshot provides an example of the above command, the output shows a connection of established to the NSX Manager over TCP Port 5671:

9. To see which NSX Manager the host is configured to communicate with, execute the following command:

esxcfg-advcfg -g /UserVars/RmqIpAddress

The following screenshot shows the results of the execution of the command above, and the expected configuration parameter is the IP address of the NSX Manager:

To check the communication channel between NSX Manager, NSX controller cluster, control plane agent, and Distributed Firewall agent, follow the following procedure:

1. In the vCenter Web Client, navigate to Networking & Security | Installation | Host Preparation
2. Select your vSphere cluster or an ESXi host

3. Click on the Actions button and navigate to | Communication Channel Health:

The following screenshot displays the result of the preceding action and the ESXi communication health: