Today, let’s talking about how to step by step to update Mellanox NIC firmware and driver for non-RDMA Storage Space Direct Cluster Servers. You can easy to do via follow steps.
Download the correct firmware and driver for NIC
-
Access follow link to download Mellanox Firmware Tools (MFT)
http://www.mellanox.com/page/management_tools
- Click 4.6.0, select Windows, select All, select X64, click WinMFT_x64_4_6_0_48.exe and then Save.
- Access follow link to download firmware.
http://www.mellanox.com/supportdownloader/
4. Click Adapter Cards, select ConnectX-3 Pro EN, select MCX312B-XCCT, select MT_120001111023, and then click Check for older versions, because fw-ConnectX3Pro-rel-2_40_7000-MCX312B-XCC_Ax-FlexBoot-3.4.746 has performance issue.
5. Select fw-ConnectX3Pro-rel-2_40_5030-MCX312B-XCC_Ax-FlexBoot-3.4.746.bin.zip and then Save.
6. Unzip fw-ConnectX3Pro-rel-2_40_5030-MCX312B-XCC_Ax-FlexBoot-3.4.746.bin.zip file to local drive.
7. Run WinMFT_x64_4_6_0_48.exe file to install Mellanox Firmware Tools.
8. On the Welcome page click Next.
9. On the License Agreement page, select I Accept the terms in the license agreement, click Next.
10. On the Destination Folder page, use the default path and then click Next.
11. Keep the default settings on the Custom Setup page, click Next.
12. Click Install on the Ready to Install the Program page.
13. Click Yes for the User Access Control.
14. On the InstallShield Wizard Completed page, click Finish.
15. Open Command Prompt as Administrator and type “C:\Program Files\Mellanox\WinMFT”.
16. Type mst status and check the device name.
17. Update NIC firmware, type flint -d mt4130_pciconf0 -I c:\post-install\Drivers\fw-Connectx3Pro-rel-2_40_5030-MCX312B-XCC_Ax-FilexBoot-3.4.746.bin\ fw-Connectx3Pro-rel-2_40_5030-MCX312B-XCC_Ax-FilexBoot-3.4.746.bin burn.
18. Run MLNX_VPI_WinOF-5_35_All_win2016_x64 .exe to install new driver for NIC.
19. Click Yes for User Account Control.
20. On the Welcome page, click Next.
21. On the License Agreement page, select I accept the term in the license agreement and then click Next.
22. Keep the default settings on the Custom Setup page, click Next.
23. On the Firmware Upgrade page, unselect Upgrade the HCA’s firmware version, click Next.
24. On the Maximum Performance page, select Check this box to configure your system for maximum performance (Recommended), click Next.
25. On the Setup type page, select Complete and then click Next.
26. Click Install on the Ready Install the Program page.
27. On the InstallShield Wizard Completed page, click Finish.
28. On the restart your system page, click No., we need to do some procedures with cluster services before reboot server.
29. Repeat steps 7-28 for other cluster node servers.
Rebuild Network Stack after NIC Driver Update
If you can arrange an outage maintenance or your failover cluster is still a non-production, you can follow steps and easy to make change without cluster services issues.
- Logon to the cluster Node 3 server.
- Open Failover Cluster Manager, right-click the cluster name.
- Select More Actions, click Shut Down Cluster.
4. Logon to cluster Node 1, Node 2 server and reboot servers.
5. Logon to the cluster Note 3 server and reboot server, after node 3 server reboot, the cluster service will be back
6. Logon Cluster Node 1 server.
7. Run follow PowerShell cmdlet to remove and rebuild Network Stack.
#Rebuild Network Stack post-Nic Driver Update
Get-VMNetworkAdapter -ManagementOS |where-object {$_.name -ne 'internal'} | Remove-VMNetworkAdapter
Get-VMSwitch | Where-Object {$_.name -ne 'internal'} | Remove-VMSwitch
Get-NetLbfoTeam | Remove-NetLbfoTeam
#Build NIC Team
New-NetlbfoTeam PRODVLAN "Ethernet", "Ethernet 3" -TeamingMode LACP –verbose
#Get the Status of the Network Adapters
Get-NetAdapter | Sort Name
#Create the new Hyper-V Vswitch VSW01
new-vmswitch "VSW01" -MinimumBandwidthMode Weight -NetAdapterName "PRODVLAN" -verbose
#Check the Bindings
Get-NetadapterBinding | where {$_.DisplayName –like "Hyper-V*"}
#Check the Adapter Settings
Get-NetAdapter | sort name
#Now Create the Converged Adapters
Add-VMNetworkAdapter –ManagementOS –Name "LM" –SwitchName "VSW01" –verbose
Add-VMNetworkAdapter –ManagementOS –Name "HB" –SwitchName "VSW01" –verbose
Add-VMNetworkAdapter –ManagementOS –Name "CLUSTERCSV" –SwitchName "VSW01" –verbose
#Review the NIC Configuration Again
Get-NetAdapter | Sort name
#Rename the HOST NIC
Rename-NetAdapter –Name "vEthernet (VSW01)" –NewName "vEthernet (Host)" –verbose
#Review the NIC Configuration Again
Get-NetAdapter | Sort name
#Set the weighting on the NIC's
Set-VMNetworkAdapter –ManagementOS –Name "CLUSTERCSV" –MinimumBandwidthWeight 40
Set-VMNetworkAdapter –ManagementOS –Name "LM" –MinimumBandwidthWeight 30
Set-VMNetworkAdapter –ManagementOS –Name "HB" –MinimumBandwidthWeight 20
Set-VMNetworkAdapter –ManagementOS –Name "VSW01" –MinimumBandwidthWeight 10
#Set-VMNetworkAdapterVlan -ManagementOS -VMNetworkAdapterName "LM" -Access -VLanID 2
Set-VMNetworkAdapterVlan -ManagementOS -VMNetworkAdapterName "CLUSTERCSV" -Access -VLanID 259
#Configure IPs for Converged Network
New-NetIPAddress -IPAddress 10.5.1.120 -PrefixLength 18 -InterfaceAlias "vEthernet (Host)" -DefaultGateway 10.5.0.1
Set-DnsClientServerAddress -InterfaceAlias "vEthernet (Host)" -ServerAddresses 10.5.1.2,10.5.1.1
New-NetIPAddress -IPAddress 10.5.1.130 -PrefixLength 18 -InterfaceAlias "vEthernet (HB)"
New-NetIPAddress -IPAddress 10.5.1.140 -PrefixLength 18 -InterfaceAlias "vEthernet (LM)"
New-NetIPAddress -IPAddress 172.16.21.10 -PrefixLength 24 -InterfaceAlias "vEthernet (CLUSTERCSV)"
8. Repeat step 6-7 for cluster Node 2 and Node 3.
 NoteIf your cluster servers are on production and cannot arrange the outage time, you need to follow steps to do Cluster-Aware update for Storage Space Direct Clusters.
|
Hope you enjoy this post.
Cary Sun @SifuSun
Hi Cary, nice article with a detailed procedure. I’m just wondering is it really necessary to rebuild network stack on cluster nodes as I can see that after update everything works well.