Mellanox ConnectX-3 and EN6132 2-port 40Gb Ethernet Adapters for Flex SystemProduct Guide

Updated
11 Jul 2017
Form Number
TIPS0912
PDF size
11 pages, 233 KB

Abstract

The Flex System EN6132 2-port 40Gb Ethernet Adapter and Mellanox ConnectX-3 Mezz 40Gb 2-Port Ethernet Adapter in conjunction with the EN6131 40Gb Ethernet Switch offer the performance that you need to support clustered databases, parallel processing, transactional services, and high-performance embedded I/O applications, which reduces task completion time and lowers the cost per operation.

This product guide provides essential presales information to understand the Mellanox adapters and their key features, specifications and compatibility. This guide is intended for technical specialists, sales specialists, sales engineers, IT architects, and other IT professionals who want to learn more about the adapters and consider their use in IT solutions.

 

Change History

Changes in the July 11 update:

Introduction

The Flex System EN6132 2-port 40Gb Ethernet Adapter and Mellanox ConnectX-3 Mezz 40Gb 2-Port Ethernet Adapter in conjunction with the EN6131 40Gb Ethernet Switch offer the performance that you need to support clustered databases, parallel processing, transactional services, and high-performance embedded I/O applications, which reduces task completion time and lowers the cost per operation.

The EN6132 and ConnectX-3 adapters can help you achieve efficient computing by offloading protocol processing from the CPU and data movement impact, such as RDMA over Converged Ethernet and Send/Receive semantics, which allows more processor power for the application.

The EN6132 2-port 40Gb Ethernet Adapter is shown in Figure 1.

Flex System EN6132 2-port 40Gb Ethernet Adapter
Figure 1. Flex System EN6132 2-port 40Gb Ethernet Adapter

Did you know?

The 40 Gb Ethernet solution that is offered by Flex System can deploy more workloads per server without running into I/O bottlenecks. If there are failures or server maintenance, clients can move their virtual machines much faster by using 40 Gb interconnects within the chassis.

The 40 GbE switch and adapter are designed for low latency, high bandwidth, and computing efficiency for performance-driven server and storage clustering applications. They provide extreme scalability for low-latency clustered solutions with reduced packet hops.

Part number information

Table 1 shows the part numbers for ordering the adapters.

Table 1. Part numbers and feature codes for ordering
Part number Feature code Description
90Y3482 A3HK Flex System EN6132 2-port 40Gb Ethernet Adapter
7ZT7A00502 AVCU ThinkSystem Mellanox ConnectX-3 Mezz 40Gb 2-Port Ethernet Adapter

The part numbers includes the following items:

  • One adapter for use in ThinkSystem and Flex System compute nodes
  • Documentation

Benefits

The ConnectX-3 Mezz 40Gb 2-Port Ethernet Adapter and EN6132 2-port 40Gb Ethernet Adapter offer the following benefits:

  • 10 GbE and 40 GbE connectivity to servers and storage
  • Industry-leading throughput performance with low latency
  • Ideal for clients running Ethernet infrastructure in high speed trading, Web 2.0, virtualization, and cloud computing
  • Enables clients to maximize compute usage by removing I/O bottlenecks
  • Improved application productivity and efficiency with support for protocol offload and RDMA over Converged Ethernet
  • Support for industry-standard SR-IO Virtualization technology to deliver VM protection and granular levels of I/O services to applications
  • High-availability and high-performance for data center networking
  • Software compatibility with standard TCP/UDP/IP and iSCSI stacks
  • Low power, low cost, and high reliability

Features and specifications

The adapters have the following features and specifications:

  • Based on Mellanox ConnectX-3 technology
  • PCI Express 3.0 x8 host-interface
  • Two 40 Gb Ethernet ports that can operate at 10 Gbps or 40 Gbps speeds
  • CPU offload of transport operations
    • RDMA over Converged Ethernet (RoCE).
    • TCP/UDP/IP stateless offload.
      • TCP/UDP/IP checksum offload
      • TCP Large Send or Giant Send offload for segmentation
      • Receive Side Scaling (RSS)
  • End-to-end quality of service (QoS) and congestion control
    • Support for IEEE 802.1p and IP DSCP/TOS traffic processing based on the class of service.
    • Support for 802.3x flow control.
    • Congestion Notification (IEEE Std 802.3Qau) limits the transmission rate to avoid frame losses in case of congestion in the network.
    • Priority-Based Flow Control (PFC) (IEEE 802.1Qbb) extends 802.3x standard flow control to allow the switch to pause traffic based on the 802.1p priority value in each packet’s VLAN tag.
    • Enhanced Transmission Selection (ETS) (IEEE 802.1Qaz) provides a method for allocating link bandwidth based on the 802.1p priority value in each packet’s VLAN tag.
  • Hardware-based I/O virtualization
  • Jumbo frame support (up to 10 KB)
  • 802.1Q VLAN tagging
  • NIC teaming and failover (static and LACP)
  • Support for Wake on LAN (WoL)

Supported standards

The adapters support the following Ethernet standards:

  • IEEE 802.1p Class of Service (CoS) prioritization
  • IEEE 802.1Q Tagged VLAN
  • IEEE 802.1Qbb Priority-Based Flow Control (PFC)
  • IEEE 802.1Qaz Enhanced Transmission Selection (ETS)
  • IEEE 802.3ad Link Aggregation Control Protocol
  • IEEE 802.3ae 10GBASE-KR backplane 10 Gb Ethernet
  • IEEE 802.3ba 40GBASE-KR4 backplane 40 Gb Ethernet
  • IEEE 802.3Qau Congestion Notification
  • IEEE 802.3x Full-duplex Flow Control

Supported servers

The following table lists the ThinkSystem and Flex System compute nodes that support the adapters.

Table 2. Support for ThinkSystem and Flex System compute nodes
Part
number
Description
x240 (8737, E5-2600 v2)
x240 (7162)
x240 M5 (9532, E5-2600 v3)
x240 M5 (9532, E5-2600 v4)
x440 (7167)
x880/x480/x280 X6 (7903)
x280/x480/x880 X6 (7196)
SN550 (7X16)
SN850 (7X15)
90Y3482 Flex System EN6132 2-port 40Gb Ethernet Adapter N Y Y Y N N Y N N
7ZT7A00502 ThinkSystem Mellanox ConnectX-3 Mezz 40Gb 2-Port Ethernet Adapter N N N N N N N Y Y

Supported I/O modules

The EN6132 2-port 40Gb Ethernet Adapter supports the I/O modules that are listed in the following table.

Table 3. Supported I/O modules
Part number Feature code Description
90Y9346 A3HJ Flex System EN6131 40Gb Ethernet Switch

Other I/O modules: Other I/O modules that are not listed in Table 3 are not supported.

The supported I/O modules are installed in the I/O bays in the rear of the Flex System Enterprise Chassis, as shown in the following figure.

Location of the I/O module bays in the Flex System Enterprise Chassis
Figure 2. Location of the I/O module bays in the Flex System Enterprise Chassis

The connections between the EN6132 adapters that are installed in the compute nodes to the supported I/O modules that are installed in the chassis I/O bays are shown in the figure below. The figure also shows both half-wide compute nodes, such as the SN550 with two adapters, and full-wide compute nodes, such as the SN850 with four adapters.

Logical layout of the interconnects between EN6132 adapters and I/O modules
Figure 3. Logical layout of the interconnects between adapters and I/O modules

The adapters can be installed in slots A1 and A2 of the half-wide compute nodes (such as x240 M5 and SN550), and slots A1, A2, A3, and A4 of the full-wide compute nodes (such as x440 and SN850). A supported Ethernet I/O module must be installed in the corresponding I/O bay of the chassis (bays 1 and 2 when the adapter is installed slot A1 or A3 or bays 3 and 4 when the adapter is installed in slot A2 or A4).

In the compute nodes that have an integrated dual-port 10 GbE network interface controller (NIC), NIC ports are routed to bays 1 and 2 with a specialized periscope connector, and the adapter in slot A1 is not required. However, when needed, the periscope connector can be replaced with the EN6132 adapter. In such a case, integrated NIC is disabled.

The following table shows the connections between the adapters that are installed in the compute nodes to the supported I/O modules that are installed in the chassis.

Table 4. Adapter to I/O bay correspondence
I/O adapter slot
in the server
Port on the adapter Corresponding I/O module bay in the chassis
Bay 1 Bay 2 Bay 3 Bay 4
Slot 1 Port 1 Yes      
Port 2   Yes    
Slot 2 Port 1     Yes  
Port 2       Yes
Slot 3
(full-wide compute nodes only)
Port 1 Yes      
Port 2   Yes    
Slot 4
(full-wide compute nodes only)
Port 1     Yes  
Port 2       Yes

The adapters are installed in slots in a supported compute node. The following shows the locations of the slots in the x240 Compute Node. The positions of the adapters in the other supported compute nodes are similar.

Location of the I/O adapter slots in the Flex System x240 Compute Node
Figure 4. Location of the I/O adapter slots in the Flex System x240 Compute Node

Supported operating systems

The ThinkSystem Mellanox ConnectX-3 Mezz 40Gb 2-Port Ethernet Adapter supports the following operating systems:

  • Microsoft Windows Server 2012 R2
  • Microsoft Windows Server 2016
  • Red Hat Enterprise Linux 6 Server x64 Edition
  • Red Hat Enterprise Linux 7
  • SUSE Linux Enterprise Server 11 for AMD64/EM64T
  • SUSE Linux Enterprise Server 11 with Xen for AMD64/EM64T
  • SUSE Linux Enterprise Server 12
  • SUSE Linux Enterprise Server 12 with Xen
  • VMware vSphere 6.0 (ESXi)

The EN6132 2-port 40Gb Ethernet Adapter supports the following operating systems:

  • Microsoft Windows Server 2008 HPC Edition
  • Microsoft Windows Server 2008 R2
  • Microsoft Windows Server 2008, Enterprise x64 Edition
  • Microsoft Windows Server 2008, Standard x64 Edition
  • Microsoft Windows Server 2008, Web x64 Edition
  • Microsoft Windows Server 2012
  • Microsoft Windows Server 2012 R2
  • Microsoft Windows Server 2016
  • Red Hat Enterprise Linux 5 Server x64 Edition
  • Red Hat Enterprise Linux 6 Server x64 Edition
  • Red Hat Enterprise Linux 7
  • SUSE Linux Enterprise Server 10 for AMD64/EM64T
  • SUSE Linux Enterprise Server 11 for AMD64/EM64T
  • SUSE Linux Enterprise Server 12
  • VMware ESX 4.1
  • VMware ESXi 4.1
  • VMware vSphere 5.0 (ESXi)
  • VMware vSphere 5.1 (ESXi)
  • VMware vSphere 5.5 (ESXi)
  • VMware vSphere 6.0 (ESXi)
  • VMware vSphere 6.5 (ESXi)

For the latest OS support information including service packs and update levels, see the ServerProven page for Ethernet adapters for Flex System, http://static.lenovo.com/us/en/serverproven/flex/ioadapte/matrix.shtml

Warranty

The adapters have a 1-year, customer-replaceable unit (CRU) limited warranty. When installed in a supported server, these adapters assume the server’s base warranty and any Lenovo warranty service upgrade.

Physical specifications

The adapters have the following physical specifications:

Dimensions and weight (approximate):

  • Width: 100 mm (3.9 in.)
  • Depth: 80 mm (3.1 in.)
  • Weight: 13 g (0.3 lb)

Shipping dimensions and weight (approximate):

  • Height: 58 mm (2.3 in.)
  • Width: 229 mm (9.0 in.)
  • Depth: 208 mm (8.2 in.)
  • Weight: 0.4 kg (0.89 lb)

Agency approvals

The adapters conform to the following standards:

  • IEC/EN 60950-1:2006
  • ETSI EN 300 019-2-2
  • IEC 60068-2- 64, 29, 32

Related product families

Product families related to this document are the following: