Dell EMC Solutions for Microsoft Azure Stack HCI - Network Configuration Options


Dell EMC Ready Solution for Microsoft Azure Stack HCI encompasses various configurations of R740xd, R740xd2, R640, R440 Storage Spaces Direct Ready Node to power the primary compute cluster deployed as Hyper-Converged Infrastructure (HCI). This HCI built by using the Ready Nodes uses a flexible solution architecture rather than a fixed component design. This flexibility in component choice brings in different choices for integrating the Ready Nodes as a solution and in the host network configuration.

This article describes the complete list of integration choices for each usage model (Scalable, Quick Start, and 2-Node back-to-back connected), host OS configuration options, and provides PowerShell commands to complete the host OS network configuration.

This article includes adapter specific configuration instructions related to RoCEv2 or iWarp configuration in the host OS. Instructions to configure Top-of-Rack networking switches are specified in the sample network switch configurations page.

Figure 1 depicts the options available based on the Dell EMC Ready Solution for Microsoft Azure Stack HCI usage model.

NetOptions.png

 

Figure 1 - Network Integration and Host Network Configuration Options

The following sections provide an overview of the integration choices and

Note: Deployment instructions for the host OS network configuration options are provided in the Deployment Guidance section.

Scalable architecture for Dell EMC Ready Solution for Microsoft Azure Stack HCI

The Dell EMC Ready Solution for Microsoft Azure Stack HCI scalable architectures support from 2 to 16 nodes in a cluster. These architectures support both fully-converged and non-converged network topologies. The non-converged integration choices available in the scalable architectures are supported with Quick Start architectures as well.

The Dell EMC Ready Solution for Microsoft Azure Stack HCI comprises of two top of rack network switches which provide high availability by connecting the nodes to two separate network switches and implementing Switch Embedded Teaming (SET). When you are using Remote Direct Memory Access over Converged Ethernet (RoCE) in a converged network topology, you must configure data center bridging (DCB). DCB provides enhancements to the Ethernet protocol, which improves the functionality of data center networks. To take advantage of the Mellanox RoCE network adapters, Priority Flow Control (PFC) and Enhanced Transmission Selection (ETS) are required. PFC and ETS are configured on all nodes and all network switches interconnecting the nodes.

Note: The scalable configurations support up to two network adapters (4 - ports) for storage.

Fully-Converged Network Configuration

Figure 2 depicts the integration of the fully-converged architecture.

2Port-FC_new1.jpg

 

Figure 2 - Fully-Converged network topology with 2 NIC ports

Figure 3 depicts the fully-converged network topology with 4 NIC ports.

4Port-FC.png

 

Figure 3 - Fully-Converged network topology with 4 NIC ports

In the fully-converged network topology both storage ports from the server are connected to the same network fabric. Within the host OS, the NIC ports are used for both storage and management/VM traffic. Figure 4 depicts this configuration in the host OS.

2Port-FC-HN.png

 

Figure 4 - Host OS network configuration in fully-converged topology with 2 NIC ports

In this host OS network configuration, a switch embedded team is created with two or four physical ports and virtual adapters in the host OS implemented for Storage 1, Storage 2 and Management traffic.

Note: The number of virtual storage adapters in the host OS should match the number of physical ports in the SET.

Figure 5 depicts using 4 NIC ports for the switch embedded team in fully-converged topology.

4Port-FC-HN.png

 

Figure 5 - Host OS network configuration in fully-converged topology with 4 NIC ports

The storage adapters in this configuration can use either a single VLAN or four different VLANs.

Non-Converged Network Configuration

In the non-converged network configurations, the storage traffic is separated from the management/VM traffic using dedicated storage network adapters. Figure 6 depicts this network integration option.

2Port-NC.png

 

Figure 6 - Network Integration in a non-converged configuration with 2 NIC ports

In the non-converged configuration, storage traffic can either be on the physical ports or implemented as virtual adapters in the host OS connected to a switch embedded team.

Figure 7 depicts using 4 NIC ports for storage in a non-converged topology.

4Port-NC.png

 

Figure 7 - Network Integration in a non-converged configuration with 4 NIC ports

Storage on physical adapters

Within the non-converged configuration, Dell EMC recommends the network integration option in which the storage traffic is on the physical storage network adapter ports and management/VM traffic through a SET created using network ports of the server rNDC.

Figure 8 depicts this configuration.

2Port-NC-HN.png

 

Figure 8 - Storage on Physical Links in a non-converged topology with 2 NIC ports

In this host OS network configuration, there is no SET or native team created for storage traffic. The physical adapter ports in the host OS are configured with the VLAN and IP configuration that is needed for storage communication across nodes in the cluster.

Figure 9 illustrates the non-converged topology with 4 NIC ports used as physical adapters.

4Port-NC-HN.png

 

Figure 9 - Storage on Physical Links in a non-converged topology with 4 NIC ports

Storage using Switch Embedded Teaming (SET)

In the second host OS network configuration option within non-converged topology, the storage traffic uses virtual adapters in the host OS connected to a SET.

2Port-NC-HN-SET.png

 

Figure 10 - Storage on SET in a non-converged topology with 2 NIC ports

As shown in Figure 10, SET gets implemented for both storage and management/VM traffic.

Figure 11 depicts storage on SET with 4 NIC ports.

4Port-NC-HN-SET.png

 

Figure 11 - Storage on SET in a non-converged topology with 4 NIC ports

Four Port config with Storage & Management/VM Traffic on 25-Gig Network

Within the non-converged topology, this configuration talks about having two 25-GbE NICs in which one port from each NIC is used for storage traffic while the second port from each NIC is used for Management and VM traffic. This topology helps in scenarios where you would require higher bandwidth for VMs whilst not compromising on bandwidth for storage traffic.

Bandwidth shaping can be used for allowing a minimum QoS for the Management traffic.

Set-VMNetworkAdapter -ManagementOS -Name "Management" -MinimumBandwidthWeight 10
Set-VMSwitch -Name S2DSwitch -DefaultFlowMinimumBandwidthWeight 90

The above two commands ensure minimum QoS weight of 10 for Management traffic while a weight of 90 is assigned for all other traffic (VM Network in this case) flowing via the SET team.

4-Port Topology.png

 

Figure 12 - 4-Port Topology

Eight Port config with separate Management, Storage & VM Traffic

This topology features separation between physical fabric , virtual machine workload and management/backup traffic. Furthermore, it satisfies high throughput requirements for Storage/VM/Management traffic by using additional ports.

8-Port Topology.png

 

Figure 13 - 8-Port Topology

Overview of benefits:

X 4 ports dedicated for storage RDMA traffic .

X 2 ports dedicated for management/backup traffic and VM traffic.

x 2 ports dedicated for Virtual Machine traffic.

Separate virtual switch for virtual machine traffic.

Separate management switch for management and backup traffic.

DP4400 Network Topology - Connecting to Azure Stack HCI

This section talks about networking tweaks to be made to a Dell Azure Stack HCI in order to connect a DP4000 to the environment

What is DP4400?

Dell EMC Integrated Data Protection Appliance DP4400 is a converged 2U appliance that offers backup, deduplication, replication, recovery & cloud readiness. More information on DP4400 can be found here. Getting started guides and product guides can be found here.

Networking changes to Azure Stack HCI Converged and Non-Converged topology

To accommodate DP4400 few changes to both converged and non-converged network topologies have to be made. DP4400 supports only a flat network for both management and backup traffic. Dell EMC Azure Stack HCI engineering recommends using a separate network for backup traffic thus segregating Management/VM/SMB traffic.

The converged topology (Figure 4) uses 25GbE network interface cards for both Management/VM/SMB traffic. DCB is configured on both hosts and switchports to ensure lossless traffic.

Non-converged topology (Figure 8) sees use of 10GbE rNDCs for Management and VM traffic while two or four 25GbE physical adapters are used for SMB traffic.

Configuring DP4400 in a Converged Topology

DP4400 does not support Data Center Bridging. Hence, using the SET team on 25GbE interface for backup traffic is not recommended.

Backup network is configured on 10GbE rNDCs using Switch Embedded Teaming. Use of SET ensures that, this network can be used for both host and VM Application backup as needed.

Backup network uses dedicated NICs on the host and DP4400 appliance, negating the need for QoS configuration on both hosts and switchports.

DP4400 - Converged Network.png

 

Figure 14 - DP4400 in a traditional Converged Topology

Configuring DP4400 in a Non-Converged Topology

In a Non-Converged Azure Stack HCI environment storage network is configured on physical adapters while rNDCs are used for Management and VM traffic. Since Management and VM traffic do not need Flow Control, backup network can be configured on same SET team as shown in the below diagram.

Bandwidth shaping can be configured to ensure appropriate QoS for egress traffic from the host.

DP4400 - Non-Converged Network.png

 

Figure 15 - DP4400 in a traditional Non-Converged Topology

 

Two Node back-to-back connected architecture for Dell EMC Ready Solution for Microsoft Azure Stack HCI

The 2-Node back-to-back connected architectures for Dell EMC Ready Solution for Microsoft Azure Stack HCI offer 2-node Storage Spaces Direct cluster configurations. These configurations use physical network adapters connected back-to-back for the storage traffic between the nodes. For the management and VM traffic, either the 1 Gb or the 10 GbE ports from the rNDC are used in a Switch Embedded Teaming (SET) configuration. This is the only recommended configuration for back-to-back connected 2-node architecture.

Figure 16 depicts this network integration in a back-to-back connected 2-node architecture.

2Node-B2B.png

 

Figure 16 - Network Integration in a back-to-back connected 2-node cluster

For the network integration shown in Figure 12, the host OS network configuration should be same as what is shown in Figure 8. The back-to-back connected 2-node architecture is validated with only one network adapter (2-ports) for storage traffic.

Deployment Guidance

This section provides the PowerShell commands to perform host OS network configuration choices described in the earlier sections. These commands must be run on each host that will be a part for the Azure Stack HCI cluster. The variables defined at the beginning of each command block should be updated based on what specific configuration you will need on a server node.

Fully-Converged

Applies to: 2 Node back to back, QuickStart, and Scalable architectures

This section provides the PowerShell commands to configure the fully-converged network topology in which a single SET switch is used for both storage and management/VM traffic.

Host OS configuration with 2-NIC ports

The following code snippet shows the fully-converged host OS network configuration with two NIC ports.

# Variables for the scenario
$NetAdapterName = @('SLOT 1 PORT 1','SLOT 1 PORT 2')
$SwitchName = 'S2DSwitch'

$ManagementAdapterName = 'Management'
$ManagementVlanId = 102
$ManagementGateway = '172.16.102.1'
$ManagementDns = '172.16.102.2'
$ManagementIPAddress = '172.16.102.51'
$ManagementAddressPrefix = 24

$Storage1AdapterName = 'Storage1'
$Storage1VlanId = 103
$Storage1IPAddress = '172.16.103.51'
$Storage1AddressPrefix = 24

$Storage2AdapterName = 'Storage2'
$Storage2VlanId = 104
$Storage2IPAddress = '172.16.104.51'
$Storage2AddressPrefix = 24

## Create a VM switch for management and Storage traffic
$null = New-VMSwitch -Name $SwitchName -AllowManagementOS 0 -NetAdapterName $NetAdapterName -Verbose

## Add VM Network Adapters and configure VLANs and IP addresses as needed
### Management Adapter
$managementAdapter = Add-VMNetworkAdapter -SwitchName $SwitchName -ManagementOS -Passthru -Name $ManagementAdapterName -Verbose

if ($ManagementVlanId -and ($ManagementVlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $managementAdapter -Access -VlanId $ManagementVlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($ManagementIPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -IPAddress $ManagementIPAddress -DefaultGateway $MangementGateway -PrefixLength $ManagementAddressPrefix -Verbose
   Set-DnsClientServerAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -ServerAddresses $ManagementDns -Verbose
}

## Add VM Network adapters and configure VLANs and IP addresses as needed
### Storage1 Adapter
$storage1Adapter = Add-VMNetworkAdapter -SwitchName $SwitchName -ManagementOS -Passthru -Name $Storage1AdapterName -Verbose
if ($Storage1VlanId -and ($Storage1VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage1Adapter -Access -VlanId $Storage1VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage1IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage1NetAdapterName)" -IPAddress $Storage1IPAddress -PrefixLength $Storage1AddressPrefix -Verbose   
}

### Storage 2 Adapter
$storage2Adapter = Add-VMNetworkAdapter -SwitchName $SwitchName -ManagementOS -Passthru -Name $Storage2AdapterName -Verbose
if ($Storage2VlanId -and ($Storage2VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage2Adapter -Access -VlanId $Storage2VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage2IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage2NetAdapterName)" -IPAddress $Storage2IPAddress -PrefixLength $Storage2AddressPrefix -Verbose   
}
Host OS configuration with 4-NIC ports

The following code snippet shows the fully-converged host OS network configuration with four NIC ports.

# Variables for the scenario
$NetAdapterName = @('SLOT 1 PORT 1','SLOT 1 PORT 2','SLOT 2 PORT 1','SLOT 2 PORT 2')
$SwitchName = 'S2DSwitch'

$ManagementAdapterName = 'Management'
$ManagementVlanId = 102
$ManagementGateway = '172.16.102.1'
$ManagementDns = '172.16.102.2'
$ManagementIPAddress = '172.16.102.51'
$ManagementAddressPrefix = 24

$Storage1AdapterName = 'Storage1'
$Storage1VlanId = 103
$Storage1IPAddress = '172.16.103.51'
$Storage1AddressPrefix = 24

$Storage2AdapterName = 'Storage2'
$Storage2VlanId = 104
$Storage2IPAddress = '172.16.104.51'
$Storage2AddressPrefix = 24

$Storage1AdapterName = 'Storage3'
$Storage1VlanId = 105
$Storage1IPAddress = '172.16.105.51'
$Storage1AddressPrefix = 24

$Storage2AdapterName = 'Storage4'
$Storage2VlanId = 106
$Storage2IPAddress = '172.16.106.51'
$Storage2AddressPrefix = 24

## Create a VM switch for management and Storage traffic
$null = New-VMSwitch -Name $SwitchName -AllowManagementOS 0 -NetAdapterName $NetAdapterName -Verbose

## Add VM Network Adapters and configure VLANs and IP addresses as needed

### Management Adapter
$managementAdapter = Add-VMNetworkAdapter -SwitchName $SwitchName -ManagementOS -Passthru -Name $ManagementAdapterName -Verbose

if ($ManagementVlanId -and ($ManagementVlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $managementAdapter -Access -VlanId $ManagementVlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($ManagementIPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -IPAddress $ManagementIPAddress -DefaultGateway $MangementGateway -PrefixLength $ManagementAddressPrefix -Verbose
   Set-DnsClientServerAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -ServerAddresses $ManagementDns -Verbose
}

## Add VM Network adapters and configure VLANs and IP addresses as needed
### Storage 1 Adapter
$storage1Adapter = Add-VMNetworkAdapter -SwitchName $SwitchName -ManagementOS -Passthru -Name $Storage1AdapterName -Verbose
if ($Storage1VlanId -and ($Storage1VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage1Adapter -Access -VlanId $Storage1VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage1IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage1NetAdapterName)" -IPAddress $Storage1IPAddress -PrefixLength $Storage1AddressPrefix -Verbose   
}

### Storage 2 Adapter
$storage2Adapter = Add-VMNetworkAdapter -SwitchName $SwitchName -ManagementOS -Passthru -Name $Storage2AdapterName -Verbose
if ($Storage2VlanId -and ($Storage2VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage2Adapter -Access -VlanId $Storage2VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage2IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage2NetAdapterName)" -IPAddress $Storage2IPAddress -PrefixLength $Storage2AddressPrefix -Verbose   
}

### Storage 3 Adapter
$storage1Adapter = Add-VMNetworkAdapter -SwitchName $SwitchName -ManagementOS -Passthru -Name $Storage3AdapterName -Verbose
if ($Storage3VlanId -and ($Storage3VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage3Adapter -Access -VlanId $Storage3VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage3IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage3NetAdapterName)" -IPAddress $Storage3IPAddress -PrefixLength $Storage3AddressPrefix -Verbose   
}

### Storage 4 Adapter
$storage4Adapter = Add-VMNetworkAdapter -SwitchName $SwitchName -ManagementOS -Passthru -Name $Storage4AdapterName -Verbose
if ($Storage4VlanId -and ($Storage4VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage4Adapter -Access -VlanId $Storage4VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage4IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage4NetAdapterName)" -IPAddress $Storage4IPAddress -PrefixLength $Storage4AddressPrefix -Verbose   
}

Non-Converged: Physical Storage and SET for management

Applies to: 2 Node back to back, QuickStart, and Scalable architectures

This section provides the PowerShell commands to configure the non-converged network topology in which the physical links are used for storage traffic and a SET is implemented for management/VM traffic.

Host OS configuration with 2-NIC ports

The following code snippet shows the non-converged host OS network configuration with two NIC ports connected as physical links.

# Variables for the scenario
$ManagementSwitchName = 'Management'
$ManagementNetAdapterName = @('NIC1','NIC2')
$ManagementAdapterName = 'Management'
$ManagementVlanId = 102
$ManagementGateway = '172.16.102.1'
$ManagementDns = '172.16.102.2'
$ManagementIPAddress = '172.16.102.51'
$ManagementAddressPrefix = 24

$Storage1NetAdapterName = 'SLOT 1 PORT 1'
$Storage1VlanId = 103
$Storage1IPAddress = '172.16.103.51'
$Storage1AddressPrefix = 24

$Storage2NetAdapterName = 'SLOT 1 PORT 2'
$Storage2VlanId = 104
$Storage2IPAddress = '172.16.104.51'
$Storage2AddressPrefix = 24

## Create a VM switch for management
$null = New-VMSwitch -Name $ManagementSwitchName -AllowManagementOS 0 -NetAdapterName $ManagementNetAdapterName -MinimumBandwidthMode Weight -Verbose

## Add VM Network Adapters and configure VLANs and IP addresses as needed
### Management Adapter
$managementAdapter = Add-VMNetworkAdapter -SwitchName $ManagementSwitchName -ManagementOS -Passthru -Name $ManagementAdapterName -Verbose

if ($ManagementVlanId -and ($ManagementVlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $managementAdapter -Access -VlanId $ManagementVlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($ManagementIPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -IPAddress $ManagementIPAddress -DefaultGateway $MangementGateway -PrefixLength $ManagementAddressPrefix -Verbose
   Set-DnsClientServerAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -ServerAddresses $ManagementDns -Verbose
}

## Storage1 Adapter
if ($Storage1VlanId -and ($Storage1VlanId -ne 0))
{
   Set-NetAdapterAdvancedProperty -Name $Storage1NetAdapterName -DisplayName 'VLAN ID' -DisplayValue $Storage1VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage1IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias $Storage1NetAdapterName -IPAddress $Storage1IPAddress -PrefixLength $Storage1AddressPrefix -Verbose   
}

## Storage2 Adapter
if ($Storage2VlanId -and ($Storage2VlanId -ne 0))
{
   Set-NetAdapterAdvancedProperty -Name $Storage2NetAdapterName -DisplayName 'VLAN ID' -DisplayValue $Storage2VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage2IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias $Storage2NetAdapterName -IPAddress       $Storage2IPAddress -PrefixLength $Storage2AddressPrefix -Verbose   
}
Host OS configuration with 4-NIC ports

The following code snippet shows the non-converged host OS network configuration with four NIC ports connected as physical links.

# Variables for the scenario
$ManagementSwitchName = 'Management'
$ManagementNetAdapterName = @('NIC1','NIC2')
$ManagementAdapterName = 'Management'
$ManagementVlanId = 102
$ManagementGateway = '172.16.102.1'
$ManagementDns = '172.16.102.2'
$ManagementIPAddress = '172.16.102.51'
$ManagementAddressPrefix = 24

$Storage1NetAdapterName = 'SLOT 1 PORT 1'
$Storage1VlanId = 103
$Storage1IPAddress = '172.16.103.51'
$Storage1AddressPrefix = 24

$Storage2NetAdapterName = 'SLOT 1 PORT 2'
$Storage2VlanId = 104
$Storage2IPAddress = '172.16.104.51'
$Storage2AddressPrefix = 24

$Storage3NetAdapterName = 'SLOT 2 PORT 1'
$Storage3VlanId = 105
$Storage3IPAddress = '172.16.105.51'
$Storage3AddressPrefix = 24

$Storage4NetAdapterName = 'SLOT 3 PORT 2'
$Storage4VlanId = 106
$Storage4IPAddress = '172.16.106.51'
$Storage4AddressPrefix = 24

## Create a VM switch for management
$null = New-VMSwitch -Name $ManagementSwitchName -AllowManagementOS 0 -NetAdapterName $ManagementNetAdapterName -MinimumBandwidthMode Weight -Verbose

## Add VM Network Adapters and configure VLANs and IP addresses as needed
### Management Adapter
$managementAdapter = Add-VMNetworkAdapter -SwitchName $ManagementSwitchName -ManagementOS -Passthru -Name $ManagementAdapterName -Verbose

if ($ManagementVlanId -and ($ManagementVlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $managementAdapter -Access -VlanId $ManagementVlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($ManagementIPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -IPAddress $ManagementIPAddress -DefaultGateway $MangementGateway -PrefixLength $ManagementAddressPrefix -Verbose
   Set-DnsClientServerAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -ServerAddresses $ManagementDns -Verbose
}

## Storage 1 Adapter
if ($Storage1VlanId -and ($Storage1VlanId -ne 0))
{
   Set-NetAdapterAdvancedProperty -Name $Storage1NetAdapterName -DisplayName 'VLAN ID' -DisplayValue $Storage1VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage1IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias $Storage1NetAdapterName -IPAddress $Storage1IPAddress -PrefixLength $Storage1AddressPrefix -Verbose   
}

## Storage 2 Adapter
if ($Storage2VlanId -and ($Storage2VlanId -ne 0))
{
   Set-NetAdapterAdvancedProperty -Name $Storage2NetAdapterName -DisplayName 'VLAN ID' -DisplayValue $Storage2VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage2IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias $Storage2NetAdapterName -IPAddress       $Storage2IPAddress -PrefixLength $Storage2AddressPrefix -Verbose   
}

## Storage 3 Adapter
if ($Storage3VlanId -and ($Storage3VlanId -ne 0))
{
   Set-NetAdapterAdvancedProperty -Name $Storage3NetAdapterName -DisplayName 'VLAN ID' -DisplayValue $Storage3VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage3IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias $Storage3NetAdapterName -IPAddress $Storage3IPAddress -PrefixLength $Storage3AddressPrefix -Verbose   
}

## Storage 4 Adapter
if ($Storage4VlanId -and ($Storage4VlanId -ne 0))
{
   Set-NetAdapterAdvancedProperty -Name $Storage4NetAdapterName -DisplayName 'VLAN ID' -DisplayValue $Storage4VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage4IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias $Storage4NetAdapterName -IPAddress       $Storage4IPAddress -PrefixLength $Storage4AddressPrefix -Verbose   
}

Non-Converged: SET Storage and SET for management

Applies to: QuickStart and Scalable architectures

This section provides the PowerShell commands to configure the non-converged network topology in which the SET switches are used for both storage traffic management/VM traffic.

Host OS configuration with 2-NIC ports

The following code snippet shows the non-converged host OS network configuration with two NIC ports for storage connected to switch embedded team .

# Variables for the scenario
$ManagementSwitchName = 'Management'
$ManagementNetAdapterName = @('NIC1','NIC2')
$ManagementAdapterName = 'Management'
$ManagementVlanId = 102
$ManagementGateway = '172.16.102.1'
$ManagementDns = '172.16.102.2'
$ManagementIPAddress = '172.16.102.51'
$ManagementAddressPrefix = 24

$StorageSwitchName = 'Storage'
$StorageNetAdapterName = @('SLOT 1 PORT 1','SLOT 1 PORT 2')
$Storage1AdapterName = 'Storage1'
$Storage1VlanId = 103
$Storage1IPAddress = '172.16.103.51'
$Storage1AddressPrefix = 24

$Storage2AdapterName = 'Storage2'
$Storage2VlanId = 104
$Storage2IPAddress = '172.16.104.51'
$Storage2AddressPrefix = 24

## Create a VM switch for management
$null = New-VMSwitch -Name $ManagementSwitchName -AllowManagementOS 0 -NetAdapterName $ManagementNetAdapterName -MinimumBandwidthMode Weight -Verbose

## Create a VM switch for storage
$null = New-VMSwitch -Name $StorageSwitchName -AllowManagementOS 0 -NetAdapterName $StorageNetAdapterName -Verbose

## Add VM Network Adapters and configure VLANs and IP addresses as needed
### Management Adapter
$managementAdapter = Add-VMNetworkAdapter -SwitchName $ManagementSwitchName -ManagementOS -Passthru -Name $ManagementAdapterName -Verbose

if ($ManagementVlanId -and ($ManagementVlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $managementAdapter -Access -VlanId $ManagementVlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($ManagementIPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -IPAddress $ManagementIPAddress -DefaultGateway $MangementGateway -PrefixLength $ManagementAddressPrefix -Verbose
   Set-DnsClientServerAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -ServerAddresses $ManagementDns -Verbose
}

## Add VM Network adapters and configure VLANs and IP addresses as needed
### Storage 1 Adapter
$storage1Adapter = Add-VMNetworkAdapter -SwitchName $StorageSwitchName -ManagementOS -Passthru -Name $Storage1AdapterName -Verbose
if ($Storage1VlanId -and ($Storage1VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage1Adapter -Access -VlanId $Storage1VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage1IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage1NetAdapterName)" -IPAddress $Storage1IPAddress -PrefixLength $Storage1AddressPrefix -Verbose   
}

### Storage 2 Adapter
$storage2Adapter = Add-VMNetworkAdapter -SwitchName $StorageSwitchName -ManagementOS -Passthru -Name $Storage2AdapterName -Verbose
if ($Storage2VlanId -and ($Storage2VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage2Adapter -Access -VlanId $Storage2VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage2IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage2NetAdapterName)" -IPAddress $Storage2IPAddress -PrefixLength $Storage2AddressPrefix -Verbose   
}
Host OS configuration with 4-NIC ports

The following code snippet shows the non-converged host OS network configuration with four NIC ports connected as physical links.

# Variables for the scenario
$ManagementSwitchName = 'Management'
$ManagementNetAdapterName = @('NIC1','NIC2')
$ManagementAdapterName = 'Management'
$ManagementVlanId = 102
$ManagementGateway = '172.16.102.1'
$ManagementDns = '172.16.102.2'
$ManagementIPAddress = '172.16.102.51'
$ManagementAddressPrefix = 24

$StorageSwitchName = 'Storage'
$StorageNetAdapterName = @('SLOT 1 PORT 1','SLOT 1 PORT 2','SLOT 2 PORT 1','SLOT 2 PORT 2')
$Storage1AdapterName = 'Storage1'
$Storage1VlanId = 103
$Storage1IPAddress = '172.16.103.51'
$Storage1AddressPrefix = 24

$Storage2AdapterName = 'Storage2'
$Storage2VlanId = 104
$Storage2IPAddress = '172.16.104.51'
$Storage2AddressPrefix = 24

$Storage3AdapterName = 'Storage3'
$Storage3VlanId = 105
$Storage3IPAddress = '172.16.105.51'
$Storage3AddressPrefix = 24

$Storage4AdapterName = 'Storage4'
$Storage4VlanId = 106
$Storage4IPAddress = '172.16.106.51'
$Storage4AddressPrefix = 24

## Create a VM switch for management
$null = New-VMSwitch -Name $ManagementSwitchName -AllowManagementOS 0 -NetAdapterName $ManagementNetAdapterName -MinimumBandwidthMode Weight -Verbose

## Create a VM switch for storage
$null = New-VMSwitch -Name $StorageSwitchName -AllowManagementOS 0 -NetAdapterName $StorageNetAdapterName -Verbose

## Add VM Network Adapters and configure VLANs and IP addresses as needed
### Management Adapter
$managementAdapter = Add-VMNetworkAdapter -SwitchName $ManagementSwitchName -ManagementOS -Passthru -Name $ManagementAdapterName -Verbose

if ($ManagementVlanId -and ($ManagementVlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $managementAdapter -Access -VlanId $ManagementVlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($ManagementIPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -IPAddress $ManagementIPAddress -DefaultGateway $MangementGateway -PrefixLength $ManagementAddressPrefix -Verbose
   Set-DnsClientServerAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -ServerAddresses $ManagementDns -Verbose
}

## Add VM Network adapters and configure VLANs and IP addresses as needed
### Storage 1 Adapter
$storage1Adapter = Add-VMNetworkAdapter -SwitchName $StorageSwitchName -ManagementOS -Passthru -Name $Storage1AdapterName -Verbose
if ($Storage1VlanId -and ($Storage1VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage1Adapter -Access -VlanId $Storage1VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage1IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage1NetAdapterName)" -IPAddress $Storage1IPAddress -PrefixLength $Storage1AddressPrefix -Verbose   
}

### Storage 2 Adapter
$storage2Adapter = Add-VMNetworkAdapter -SwitchName $StorageSwitchName -ManagementOS -Passthru -Name $Storage2AdapterName -Verbose
if ($Storage2VlanId -and ($Storage2VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage2Adapter -Access -VlanId $Storage2VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage2IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage2NetAdapterName)" -IPAddress $Storage2IPAddress -PrefixLength $Storage2AddressPrefix -Verbose   
}

### Storage 3 Adapter
$storage3Adapter = Add-VMNetworkAdapter -SwitchName $StorageSwitchName -ManagementOS -Passthru -Name $Storage3AdapterName -Verbose
if ($Storage3VlanId -and ($Storage3VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage3Adapter -Access -VlanId $Storage3VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage3IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage3NetAdapterName)" -IPAddress $Storage3IPAddress -PrefixLength $Storage3AddressPrefix -Verbose   
}

### Storage 4 Adapter
$storage4Adapter = Add-VMNetworkAdapter -SwitchName $StorageSwitchName -ManagementOS -Passthru -Name $Storage4AdapterName -Verbose
if ($Storage4VlanId -and ($Storage4VlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -VMNetworkAdapter $storage4Adapter -Access -VlanId $Storage4VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage4IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($Storage4NetAdapterName)" -IPAddress $Storage4IPAddress -PrefixLength $Storage4AddressPrefix -Verbose   
}

 

Non-Converged: Four Port config

PowerShell cmdlets to configure Network in a 4-Port config

# Variables for the scenario
$NetAdapterName = @('SLOT 1 PORT 2','SLOT 2 PORT 1')
$SwitchName = 'S2DSwitch'

$ManagementAdapterName = 'Management'
$ManagementVlanId = 102
$ManagementGateway = '172.16.102.1'
$ManagementDns = '172.16.102.2'
$ManagementIPAddress = '172.16.102.51'
$ManagementAddressPrefix = 24

$Storage1NetAdapterName = 'SLOT 1 PORT 1'
$Storage1AdapterName = 'Storage1'
$Storage1VlanId = 103
$Storage1IPAddress = '172.16.103.51'
$Storage1AddressPrefix = 24

$Storage2NetAdapterName = 'SLOT 2 PORT 2'
$Storage2AdapterName = 'Storage2'
$Storage2VlanId = 104
$Storage2IPAddress = '172.16.104.51'
$Storage2AddressPrefix = 24

## Create a VM switch for management traffic
$null = New-VMSwitch -Name $SwitchName -AllowManagementOS 0 -NetAdapterName $NetAdapterName -MinimumBandwidthMode Weight -Verbose

## Add VM Network Adapters and configure VLANs and IP addresses as needed
### Management Adapter
$managementAdapter = Add-VMNetworkAdapter -ManagementOS -Name $ManagementAdapterName -SwitchName $SwitchName -Passthru

if ($ManagementVlanId -and ($ManagementVlanId -ne 0))
{
   Set-VMNetworkAdapterVlan -Access -VlanId $ManagementVlanId Verbose
   Start-Sleep -Seconds 5
}

#Host Management Adapter
if ($ManagementIPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias "vEthernet ($ManagementAdapterName)" -IPAddress $ManagementIPAddress -DefaultGateway $ManagementGateway -PrefixLength $ManagementAddressPrefix -AddressFamily IPv4 Verbose
   Set-DnsClientServerAddress -InterfaceAlias 'vEthernet ($ManagementAdapterName)' -ServerAddresses $ManagementDns
}


## Storage1 Adapter
if ($Storage1VlanId -and ($Storage1VlanId -ne 0))
{
   Set-NetAdapterAdvancedProperty -Name $Storage1NetAdapterName -DisplayName 'VLAN ID' -DisplayValue $Storage1VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage1IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias $Storage1NetAdapterName -IPAddress $Storage1IPAddress -PrefixLength $Storage1AddressPrefix -Verbose   
}

## Storage2 Adapter
if ($Storage2VlanId -and ($Storage2VlanId -ne 0))
{
   Set-NetAdapterAdvancedProperty -Name $Storage2NetAdapterName -DisplayName 'VLAN ID' -DisplayValue $Storage2VlanId -Verbose
   Start-Sleep -Seconds 5
}

if ($Storage2IPAddress -ne 'DHCP')
{
   $null = New-NetIPAddress -InterfaceAlias $Storage2NetAdapterName -IPAddress       $Storage2IPAddress -PrefixLength $Storage2AddressPrefix -Verbose   
}


Set-NetAdapterAdvancedProperty -Name $Storage1NetAdapterName -DisplayName 'NetworkDirectTechnology' -DisplayValue 'iWarp'
Set-NetAdapterAdvancedProperty -Name $Storage2NetAdapterName -DisplayName 'NetworkDirectTechnology' -DisplayValue 'iWarp'

 

Non-Converged: Eight Port Config

PowerShell cmdlets to configure network in a 8-Port config

# Variables for the scenario
$MgmtNetAdapterName = @('Slot 7 Port 1','Slot 8 Port 2')
$VMNetAdapterName   = @('Slot 7 Port 2','Slot 8 Port 1')
$MgmtTeamName = 'MgmtNicTeam'
$SwitchName = 'MgmtVmSwitch'
$VMTeamName = 'VmNicTeam'
$VMNetSwitch = 'VMsVSwitch'

$ManagementAdapterName = 'Management'
$ManagementVlanId = 102
$ManagementGateway = '172.16.102.1'
$ManagementDns = '172.16.102.2'
$ManagementIPAddress = '172.16.102.51'
$ManagementAddressPrefix = 24

$Storage1NetAdapterName = 'SLOT 2 PORT 1'
$Storage1AdapterName = 'Storage1'
$Storage1VlanId = 103
$Storage1IPAddress = '172.16.103.51'
$Storage1AddressPrefix = 24

$Storage2NetAdapterName = 'SLOT 2 PORT 2'
$Storage2AdapterName = 'Storage2'
$Storage2VlanId = 104
$Storage2IPAddress = '172.16.104.51'
$Storage2AddressPrefix = 24

$Storage3NetAdapterName = 'SLOT 5 PORT 1'
$Storage3AdapterName = 'Storage3'
$Storage3VlanId = 103
$Storage3IPAddress = '172.16.103.52'
$Storage3AddressPrefix = 24

$Storage4NetAdapterName = 'SLOT 5 PORT 2'
$Storage4AdapterName = 'Storage4'
$Storage4VlanId = 104
$Storage4IPAddress = '172.16.104.52'
$Storage4AddressPrefix = 24

#MANAGEMENT/BKUP NETWORK Switch:
New-NetLbfoTeam -Name $MgmtTeamName -TeamMembers $MgmtNetAdapterName -LoadBalancingAlgorithm Dynamic -TeamingMode SwitchIndependent
New-VMSwitch $SwitchName -MinimumBandwidthMode weight -NetAdapterName $MgmtTeamName -AllowManagementOS 1
Set-VMNetworkAdapterVlan -ManagementOS -VMNetworkAdapterName $SwitchName -Access -VlanId $ManagementVlanId

#VIRTUAL MACHINE NETWORK Switch:
New-NetLbfoTeam -Name $VMTeamName -TeamMembers $VMNetAdapterName -LoadBalancingAlgorithm Dynamic -TeamingMode SwitchIndependent
New-VMSwitch $VMNetSwitch -MinimumBandwidthMode weight -NetAdapterName $VMTeamName -AllowManagementOS 1

#STORAGE PHYSICAL NIC:
Set-NetAdapter -Name $Storage1NetAdapterName -VlanID $Storage1VlanId
Set-NetAdapter -Name $Storage2NetAdapterName -VlanID $Storage2VlanId
Set-NetAdapter -Name $Storage3NetAdapterName -VlanID $Storage1VlanId
Set-NetAdapter -Name $Storage4NetAdapterName -VlanID $Storage2VlanId
Get-NetAdapter | select interfacealias,vlanid,LinkSpeed,Status

#Set RDMA QLOGIC to IWARP Mode:
Set-NetAdapterAdvancedProperty -Name $Storage1NetAdapterName -DisplayName 'NetworkDirect Technology' -DisplayValue 'iWarp'
Set-NetAdapterAdvancedProperty -Name $Storage2NetAdapterName -DisplayName 'NetworkDirect Technology' -DisplayValue 'iWarp'

Set-NetAdapterAdvancedProperty -Name $Storage3NetAdapterName -DisplayName 'NetworkDirect Technology' -DisplayValue 'iWarp'
Set-NetAdapterAdvancedProperty -Name $Storage4NetAdapterName -DisplayName 'NetworkDirect Technology' -DisplayValue 'iWarp'

# SET IPs
#Host Management Adapter set gateway and DNS:
New-NetIPAddress -InterfaceAlias vEthernet ($SwitchName) -IPAddress $ManagementIPAddress -DefaultGateway $ManagementGateway -PrefixLength $ManagementAddressPrefix -AddressFamily IPv4 Verbose
Set-DnsClientServerAddress -InterfaceAlias vEthernet ($SwitchName) -ServerAddresses $ManagementDns

#Set IPs for storage Nics:
New-NetIPAddress -InterfaceAlias $Storage1NetAdapterName -IPAddress $Storage1IPAddress -PrefixLength $Storage1AddressPrefix -AddressFamily IPv4 Verbose
New-NetIPAddress -InterfaceAlias $Storage2NetAdapterName -IPAddress 1$Storage2IPAddress -PrefixLength $Storage1AddressPrefix -AddressFamily IPv4 Verbose

New-NetIPAddress -InterfaceAlias $Storage3NetAdapterName -IPAddress $Storage3IPAddress -PrefixLength $Storage1AddressPrefix -AddressFamily IPv4 Verbose
New-NetIPAddress -InterfaceAlias $Storage4NetAdapterName -IPAddress 1$Storage4IPAddress -PrefixLength $Storage1AddressPrefix -AddressFamily IPv4 Verbose

 

Configuring DP4400 in a Converged Topology

PowerShell cmdlets to configure new backup switch/network adapter are as follows.

# Variables for the scenario
$BkpNetAdapterName = @('NIC1','NIC2')
$BkpSwitchName = 'BkpSwitch'
$BkpAdapterName = 'Backup'
$BkpVlanId = 100

New-VMSwitch -Name $BkpSwitchName -AllowManagementOS 0 -NetAdapterName $BkpNetAdapterName -MinimumBandwidthMode Weight
Add-VMNetworkAdapter -ManagementOS -Name $BkpAdapterName -SwitchName BkpSwitch -Passthru | Set-VMNetworkAdapterVlan -Access -VlanId $BkpVlanId Verbose

Configuring DP4400 in a Non-Converged Topology

Sample PowerShell cmdlets to configure a new switch and bandwidth shaping on rNDCs are given below

# Variables for the scenario
$BkpNetAdapterName = @('NIC1','NIC2')
$SwitchName = 'S2DSwitch'
$BkpAdapterName = 'Backup'
$BkpVlanId = 100
$ManagementAdapterName = 'Management'
$ManagementVlanId = 102

New-VMSwitch -Name $SwitchName -AllowManagementOS 0 -NetAdapterName $BkpNetAdapterName -MinimumBandwidthMode Weight

Add-VMNetworkAdapter -ManagementOS -Name $ManagementAdapterName -SwitchName $SwitchName -Passthru | Set-VMNetworkAdapterVlan -Access -VlanId $ManagementVlanId Verbose

Add-VMNetworkAdapter -ManagementOS -Name $BkpAdapterName -SwitchName S2DSwitch -Passthru | Set-VMNetworkAdapterVlan -Access -VlanId $BkpVlanId Verbose

Set-VMNetworkAdapter -ManagementOS -Name $ManagementAdapterName -MinimumBandwidthWeight 10
Set-VMNetworkAdapter -ManagementOS -Name $BkpAdapterName -MinimumBandwidthWeight 40
Set-VMSwitch -Name S2DSwitch -DefaultFlowMinimumBandwidthWeight 50
NetworkMinimum Bandwidth Weight
Management10
Backup40
VM Network/Other Network (Default Min Bandwidth)50

Additional guidance for all configurations

This section consists of guidance which will apply to all the configurations mentioned in this article

Change RDMA mode for Qlogic NICs - iWARP only

Qlogic 41262 NICs are configured to use iWARP. The Operating system driver defaults to RoCEv2 for RDMA. Change the setting by running the following cmdlet for each network storage network interface.

# Variables for the scenario
$StorageAdapters = @('SLOT 1 PORT 1','SLOT 1 PORT 2')
ForEach ($port in $StorageAdapters) {
    Set-NetAdapterAdvancedProperty -Name $port -DisplayName 'NetworkDirect Technology' -DisplayValue 'iWarp'
}

RDMA Configuration

Storage Spaces Direct hyperconverged Infrastructures with Mellanox ConnectX-4 LX, Mellanox ConnectX-5 EX, and QLogic FastLinQ 41262 adapters use converged network design and RDMA for storage traffic. Hence, it is important to ensure that the physical and virtual adapters used for storage traffic have RDMA enabled.

# Variables for the scenario
$StorageAdaptersAll = @('SLOT 1 PORT 1','SLOT 1 PORT 2','vEthernet (Storage1)','vEthernet (Storage2)')
Enable-NetAdapterRdma -Name $StorageAdaptersAll

QoS Policy Configuration

Microsoft Storage Spaces Direct Ready Nodes that are configured with Mellanox ConnectX-4 LX and Mellanox ConnectX-5 EX adapters require DCB configuration on the ToR switches and Quality of Service (QoS) configuration in the host OS. Ensure that the QoS policies are configured to prioritize the SMB traffic related to the storage adapters. Ensure that the QoS configuration in the host OS matches the QoS configuration that is performed in Network switch configuration.

The below table provides an overview of QoS priorities and the required state for the set of priorities .

QoS PriorityState
0,1,2,4,6,7Disabled
3Enabled
5Enabled

Perform the following steps to configure QoS in the host OS.

# Variables for the scenario
$StorageAdapters = @('SLOT 1 PORT 1','SLOT 1 PORT 2')
#New QoS policy with a match condition set to 445 (TCP Port 445 is dedicated for SMB)
#Arguments 3 and 5 to the PriorityValue8021Action parameter indicate the IEEE 802.1p #values for SMB and cluster traffic.
New-NetQosPolicy -Name 'SMB' NetDirectPortMatchCondition 445 PriorityValue8021Action 3
New-NetQosPolicy 'Cluster' -Cluster -PriorityValue8021Action 5

#Map the IEEE 802.1p priority enabled in the system to a traffic class
New-NetQosTrafficClass -Name 'SMB' Priority 3 BandwidthPercentage 50 Algorithm ETS
New-NetQosTrafficClass -Name 'Cluster' Priority 5 BandwidthPercentage 1 Algorithm ETS

#Configure flow control for the priorities shown in the above table
Enable-NetQosFlowControl Priority 3,5
Disable-NetQosFlowControl Priority 0,1,2,4,6,7

#Enable QoS for the Mellanox network adapter ports.
foreach ($port in $StorageAdapterName) {
    Enable-NetAdapterQos InterfaceAlias $port
}

#Disable DCBX Willing mode
Set-NetQosDcbxSetting -Willing $false

#Set Mellanox NIC DCBX mode to Host in charge
foreach ($port in $StorageAdapterName) {
Set-NetAdapterAdvancedProperty -Name $port -DisplayName 'Dcbxmode' -DisplayValue 'Host in charge'
}

Appendix

VLAN Information

Traffic ClassPurposeVLAN IDTagged/Untagged
BackupUsed for backup traffic (DP4400 scenarios)100Untagged
Out of BandUsed for OOB Management of nodes and ToR switchesUntagged
Host ManagementManagement of cluster and cluster nodes102Untagged/Tagged
Storage 1SMB Traffic103Untagged/Tagged
Storage 2SMB Traffic104Untagged/Tagged
Storage 3SMB Traffic105Untagged/Tagged
Storage 4SMB Traffic106Untagged/Tagged