NSX-T – How to use all physical NICs/uplinks in your Bare Metal Edge
Problem: On a newly installed NSX-T 2.5 Bare Metal Edge node, you can only see 3 available physical NICs/uplinks, even though your physical server has 4 of them installed and connected. (Or 7 uplinks if you have 8 of them etc.)
When you list the physical uplinks available in the NSX Edge CLI or UI, you see something like this:
Explanation: This is because NSX by default automatically reserves one of the uplinks exclusively for the Management Interface. (This is the IP address that you and NSX Manager use to configure and manage the Edge Node.)
Solution: Configure the NSX Edge Node to put the Management Interface IP address in-band “behind” one of the uplinks instead of reserving one uplink exclusively for it.
This is done in the CLI. Preferably through the out-of-band KVM feature that your physical server vendor uses, like iLO, iDRAC etc:
The ‘clear’ commands above only need to be run if you’ve already set a management IP address for the Edge Node in question.
The VLAN number of the management interface is 100 in the example above, but should be replaced with your VLAN number. Same thing goes for the IP address, subnet mask and default gateway.
The MAC address of the physical management interface that you want to use is e4:43.. in the example above, but should be replaced with the MAC address of the physical NIC port that you want to use for in-band management.
If all goes well, you should now be able to ping the management IP address, and you should see all your physical uplinks:
After adding the Edge in the NSX Manager UI (by following this article but getting the cluster thumbprint, and not the api thumbprint in step 3), you can now see and map all 4 physical uplinks to NSX uplinks when configuring the Edge Node:
Read my other blog posts about NSX and the not-always-so-easy-to-design Edges here, here and here. Good luck!