I honestly don’t know why this is still a problem. Support for routed vmotion traffic was added back at vSphere6. Here we are vSphere7 and still have to set your gateway/routes for the vmotion stack through esxcli.
Either way, here is how it works
vMotion stack
Each tcp/ip stack can only have one gateway, that makes sense. And if you want to keep your management and vMotion traffic separated you need two tcp/ip stacks.
It’s nicely done through vSphere vCenter GUI and there is a KB for it. And you even have the option to override the default gateway and specify the right one for your vMotion stack.
[root@dc1esxcompx-xx:~] esxcli network ip route ipv4 list -N vmotion
Network Netmask Gateway Interface Source
---------- ------------- ------- --------- ------
10.1.115.0 255.255.255.0 0.0.0.0 vmk1 MANUALBut when looking at the routing table from esxcli it is not set. If you know, feel free to give me a kick and enlighten me.
ESXCLI add a static route
So for me to set an actually default route I have to do it as shown below
[root@dc1esxcompx-xx:~] esxcli network ip route ipv4 add -g 10.1.115.1 -n 0.0.0.0/0 -N vmotion
[root@dc1esxcompx-xx:~] esxcli network ip route ipv4 list -N vmotion
Network Netmask Gateway Interface Source
---------- ------------- ---------- --------- ------
default 0.0.0.0 10.1.115.1 vmk1 MANUAL
10.1.115.0 255.255.255.0 0.0.0.0 vmk1 MANUALPowerCLI
Need to do it on a cluster with multiple hosts? No problem LucD from VMware community got you covered. I only did a little customization and it works for my needs.
connect-viserver -Server
$stackName = 'vmotion'
$ipGateway = '10.1.115.1'
$ipDevice = 'vmk3'
$cluster = "computexx"
$vmhosts = get-cluster $cluster | get-vmhost
foreach($vmhost in $vmhosts)
{
$esx = Get-VMHost -Name $vmhost
$netSys = Get-View -Id $esx.ExtensionData.ConfigManager.NetworkSystem
$stack = $esx.ExtensionData.Config.Network.NetStackInstance | where{$_.Key -eq 'vmotion'}
$config = New-Object VMware.Vim.HostNetworkConfig
$spec = New-Object VMware.Vim.HostNetworkConfigNetStackSpec
$spec.Operation = [VMware.Vim.ConfigSpecOperation]::edit
$spec.NetStackInstance = $stack
$spec.NetStackInstance.ipRouteConfig.defaultGateway = $ipGateway
$spec.NetStackInstance.ipRouteConfig.gatewayDevice = $ipDevice
$config.NetStackSpec += $spec
$netsys.UpdateNetworkConfig($config,[VMware.Vim.HostConfigChangeMode]::modify)
}Conclusion
Manipulating the vmotion stack route table with either esxcli or PowerCLI is working great.
Need to know more? there are plenty of good bloggers and KBs out here.
