Objective 6.3 – Troubleshoot Network Performance and Connectivity

Knowledge - Identify vCLI commands and tools used to troubleshoot vSphere networking configurations

• esxcfg-vswitch
• esxcfg-nics
• esxcfg-vmknic
• esxcfg-route
• esxcli network

Knowledge - Identify logs used to troubleshoot network issues

• DHCP issues – /var/log/dhclient.log
• Networking driver and device issues – /var/log/vmkernel.log
• vCenter issues – /var/log/vpxa.log

Skills and Abilities - Utilize net-dvs To Troubleshoot vSphere Distributed Switch Configurations
./usr/lib/vmware/bin/net-dvs

Lists a very verbose level of detail about packets passing through each vDS

Skills and Abilities - Utilize vSphere CLI Commands To Troubleshoot ESXi Network Configurations
Real world usages I have come across for the four primary vCLI network configuration tools, to note these tools can be used for many more things in addition to the below.

On newly built servers if NICs cabled incorrectly default vSwitch links to invalid uplink for this esxcfg-vswitch is useful to add and remove uplinks,  similarly if no native VLAN is set within a trunk esxcfg-vswitch can be used to set a portgroup VLAN to allow access.  If a network port is failing to negotiate correctly esxcfg-nics can be used to force various speeds and settings until successful negotiation occurs. If you want to add or manage VMkernel ports, esxcfg-vmknic can be useful.  To view or set a default gateway or static routing esxcfg-route I had occasion to use static routing once as my iSCSI \ NFS SAN was located on remote subnets (720 Mpps link but still routed) to ensure these routed correctly I defined static routes between end the networks.

Skills and Abilities - Troubleshoot Private VLANs
 
Skills and Abilities - Troubleshoot vmkernel related network configuration issues
/var/log/vmkernel.log

vmkping can be used to test the vmkernel networking

Skills and Abilities - Troubleshoot DNS and routing related issues
vSphere 5.0 and later rely heavily on both A and PTR DNS records being in place and correct. Prior to starting any troubleshooting ensure nslookup can resolve correct name to IP (A record) and IP to name (PTR record) nslookup is included as command on ESXi CLI.

Unlikely but possible,  DNS can be disabled via the ESXi Firewall.

 

For troubleshooting routing, esxcfg-route can be used to view the routing table also traceroute and ping can be used to generate traffic to establish the issue.

If you need to capture the traffic packets this can be done by following this kb

Skills and Abilities - Use esxtop/resxtop To Identify Network Performance Problems
The best reference for esxtop threshold Yellow Bricks summarized, to note network performance can manifest as CPU performance issue if incorrect MTU, speed or driver incorrect.  So to check CPU and network counters.

Skills and Abilities - Analyze Troubleshooting Data To Determine If The Root Cause For A Given Network Problem Originates In The Physical Infrastructure or vSphere Environment
 
Skills and Abilities - Configure and administer Port Mirroring

Skills and Abilities - Utilize Direct Console User Interface (DCUI) and ESXi Shell to troubleshoot, configure, and monitor ESXi networking
If your ESXi host networking has failed,  connecting to DCUI is useful way to get the configuration working you can

• Add \ remove uplinks from vSwitches
• Change IP settings
• Revert your from broken vDS to vSS
• Test management network connectivity
• Look at log files