Installation of NSX VIBs that enable the Distributed Firewall and VXLAN are essential for a working NSX environment. This section will investigate how to manually verify that each VIB is installed correctly and whether communication to both the NSX controller cluster and NSX Manager are present.
The first control plane communication that we are concerned with is from the NSX Manager to each ESXi host via TCP port 5671. This port is reserved for the Rabbit MQ Message bus to the vsfwd daemon running on each host after the VMware Service Insertion Platform (VSIP) VIB installation, which is the Distributed Firewall kernel module. The NSX Manager uses the message bus to publish firewall rules down to each ESXi host. The ESXi host then applies them to vNICs of virtual machines that are running on top of its hypervisor.
The second control plane communication that is expected from each ESXi host is an open connection to each of the NSX controllers deployed. The NSX controller cluster is responsible for control plane information for ARP/MAC/VTEP tables. It is also used to program routes received on the Distributed Logical Router Control VM to each host (more on this in Chapter 2, Configuring VMware NSX Logical Switch Networks). From each host, we expect the netcpa daemon to have an active connection to the controller cluster on TCP port 1234.
To manually verify control-plane communication and VIB installation, you will need the following access to the following NSX components:
- SSH access to NSX Manager
- SSH access to each NSX controller
- SSH access to ESXi hosts that were prepared for NSX
You would not be expected to check communication of each and every host in your environment, as this can become unwieldly. However, this section is included for you to understand what the expected communication is, but in large deployments you would check the communication channel health per vSphere cluster as depicted in the earlier section.
To check whether the NSX VIBs have been installed successfully is crucial. The upcoming section details how to do this manually on an ESXi host and how to check NSX component communication.
In this section we perform manual verification that the VIBs have been successfully installed.
- SSH onto an ESXi host.
- Check whether VXLAN VIB modules have been installed by executing the following command:
esxcli software vib get --vibname esx-vxlan- You will receive an output similar to the following:
[root@vSphere:~] esxcli software vib get --vibname esx-vxlanVMware_bootbank_esx-vxlan_6.0.0-0.0.4987429 Name: esx-vxlan Version: 6.0.0-0.0.4987429Type: bootbankVendor: VMwareAcceptance Level: VMwareCertifiedSummary: Vxlan and host toolDescription: This package loads module and configures firewall for vxlan networking. ReferenceURLs:Creation Date: 2017-01-27Depends: esx-base >= 6.0, esx-base <= 6.5.0, nsx-api <= 2.1, vmkapi_2_3_0_0Conflicts: nsx-api = 2Replaces: esx-traceflow, esx-dvfilter-switch-security, esx-bfdProvides: com.vmware.vxlan = 1.0.0.0-nsx, com.vmware.switchsecurity = 1.0.0.0, com.vmware.traceflow = 1.0.0.0, com.vmware.bfd = 1.0.0.0Maintenance Mode Required: TrueHardware Platforms Required:Live Install Allowed: TrueLive Remove Allowed: TrueStateless Ready: TrueOverlay: FalseTags:Payloads: esx-vxlan
- If the module has been installed correctly, you should see open TCP connections on port
1234with the following command:
esxcli network ip connection list | grep 1234
An example is included below that shows the connection as established to each of the three NSX controllers from the point of view of an ESXi host:
- To see which NSX controllers the host is configured to communicate with, execute the following command:
cat /etc/vmware/netcpa/config-by-vsm.xmlThe following screenshot provides a truncated output of the command and its expected output.
- Check if
VSIP VIBmodules have been installed by executing the following command:esxcli software vib get --vibname esx-vsip:
- You will receive an output similar to the following:
[root@vSphere:~] esxcli software vib get --vibname esx-vsipVMware_bootbank_esx-vsip_6.0.0-0.0.4987429 Name: esx-vsipVersion: 6.0.0-0.0.4987429Type: bootbankVendor: VMwareAcceptance Level: VMwareCertifiedSummary: vsip moduleDescription: This package contains DFW and NetX data and control plane components.ReferenceURLs:Creation Date: 2017-01-27Depends: esx-base >= 6.0, esx-base <= 6.5.0, nsx-api <= 2.1, vmkapi_2_3_0_0Conflicts: nsx-api = 2Replaces: esx-vdpi Provides: vsip = 1.0.0-0Maintenance Mode Required: TrueHardware Platforms Required:Live Install Allowed: TrueLive Remove Allowed: TrueStateless Ready: TrueOverlay: FalseTags:Payloads: esx-vsip
- If the module has been installed correctly, you can open TCP connections on port
5671with the following command:
esxcli network ip connection list | grep 5671The following screenshot provides an example of the above command, the output shows a connection of established to the NSX Manager over TCP Port 5671:
- To see which NSX Manager the host is configured to communicate with, execute the following command:
esxcfg-advcfg -g /UserVars/RmqIpAddressThe following screenshot shows the results of the execution of the command above, and the expected configuration parameter is the IP address of the NSX Manager:
To check the communication channel between NSX Manager, NSX controller cluster, control plane agent, and Distributed Firewall agent, follow the following procedure:
- In the vCenter Web Client, navigate to
Networking & Security|Installation|Host Preparation - Select your vSphere cluster or an ESXi host
- Click on the
Actionsbutton and navigate to |Communication Channel Health:
The following screenshot displays the result of the preceding action and the ESXi communication health:
