Voltaire 40 Gb InfiniBand Switch Module for BladeCenter

Clients want to increase the use of their existing servers to reduce overall cost as well as the IT footprint within the organization. BladeCenter offers several choices, including servers, chassis, and the types of interconnect fabric. This solution offers end-to-end 40 Gb (4X QDR) InfiniBand connectivity. Clients can take advantage of the resiliency of BladeCenter as well as the performance of 4X QDR InfiniBand in a single package.

The Voltaire 40 Gb InfiniBand Switch Module for BladeCenter provides InfiniBand QDR connectivity between the server platform's high performance blade servers and external InfiniBand fabrics in non-blocking designs, all on a single device. Voltaire's high speed module also accommodates performance-optimized fabric designs utilizing a single BladeCenter chassis or stacking multiple BladeCenter chassis without requiring an external InfiniBand switch.

The InfiniBand switch module offers 14 Gb ports, one to each server, and 16 ports out of the chassis per switch. This solution offers a no compromise, congestion free solution to meet even the most performance hungry applications.

Figure 1 shows the switch module.

Figure 1. Voltaire 40 Gb InfiniBand Switch Module for BladeCenter

The module's HyperScale architecture also provides a unique inter-switch link or mesh capability to form highly scalable, cost effective, and low latency fabrics. Since this switch has 16 uplink ports, they can create a meshed architecture and still have unblocked access to data using the 14 uplink ports. This solution can scale from 14 to 126 nodes and offer latency less than 200 nanosecond allowing applications to operate at maximum efficiency.

Table 1 shows the part numbers to order the module and the supported cables.

Table 1. Part number and feature code for ordering

* Withdrawn from marketing

The Voltaire 40 Gb InfiniBand Switch Module part number includes the following items:

• One Voltaire 40 Gb InfiniBand Switch Module
• Documentation package

The Voltaire 40 Gb InfiniBand Switch Module includes the following features and functions:

• Form-factor
  • Double-height high-speed switch module
• Internal ports
  • 14 internal 4X QDR/DDR/SDR InfiniBand to the server blades
  • Two internal 4X InfiniBand ports to the bridge modules
• External ports
  • Up to 16 auto-sensing 4X QDR/DDR/SDR InfiniBand QSFP ports (40, 20, or 10 Gbps auto-negotiate)
• Characteristics
  • Full QDR rate InfiniBand switching
  • Based on the Infiniscale-IV device
  • Up to 40 Gbps performance for clusters and grids (bidirectional)
  • Hot swap support, with soft start and current limiting
  • Bay address and presence support
  • High temperature monitoring
  • Full BladeCenter H AMM interface via I2C
  • Fully non-blocking QDR blade
  • QSFP front panel connectors
  • On board CPU for chassis management capabilities
  • Ultra-low latency: under 100 nanoseconds blade to blade connectivity
  • Built-in high availability
  • Port status controlability

The Voltaire 40 Gb InfiniBand Switch Module is supported only in the BladeCenter H chassis as listed in Table 2.

Table 2. BladeCenter chassis that support the Voltaire 40 Gb InfiniBand Switch Module

I/O module	Part number
Voltaire 40 Gb InfiniBand Switch Module	46M6005	N	N	Y	N	N	N	N

The Voltaire 40 Gb InfiniBand Switch Module support the expansion cards listed in Table 3. This table also lists the chassis bays in which the switch module must be installed when used with each expansion card.

Table 3. Voltaire 40 Gb InfiniBand Switch Module and BladeCenter chassis I/O bays support

2-Port 40Gb InfiniBand Expansion Card (CFFh)	46M6001	N	N	N	N	N	N	Y	Y	Y	Y
InfiniBand 4X DDR Expansion Card (CFFh)	43W4423	N	N	N	N	N	N	Y	Y	Y	Y
Voltaire 4X InfiniBand DDR (CFFh)	43W4420	N	N	N	N	N	N	Y	Y	Y	Y

Note: The Voltaire 40 Gb InfiniBand Switch Module is a double-high switch module and occupies two adjacent high-speed bays (7 and 8 or 9 and 10).

The BladeCenter H chassis has four standard I/O bays (1, 2, 3, and 4), two bridge bays (5 and 6), and four high-speed bays (7, 8, 9, and 10). The Voltaire 40 Gb InfiniBand Switch Module is supported only in the BladeCenter H chassis and occupies two adjacent high-speed I/O bay (bays 7 and 8, or bays 9 and 10).

Figure 2 shows the use of Voltaire 40 Gb InfiniBand Switch Module to route two 4X InfiniBand ports from 2-Port 40Gb InfiniBand Expansion Card (CFFh) installed into each server. Two Voltaire 40 Gb InfiniBand Switch Modules are installed in bays 7/8 and bays 9/10 of the BladeCenter H chassis. All connections between the expansion cards and the switch modules are internal to the chassis. No cabling is needed.

Figure 2. A 40 Gb solution using two Voltaire 40 Gb InfiniBand Switch Modules

Table 4 lists the components that this configuration uses.

Table 4. Components used when connecting 2-Port 40 Gb InfiniBand Expansion Card (CFFh) to two Voltaire 40 Gb InfiniBand Switch Modules

Diagram reference	Part number/machine type	Description	Quantity
	Varies	BladeCenter HS22 or other supported server	1 to 14
	46M6001	2-Port 40 Gb InfiniBand Expansion Card (CFFh)	1 per server
	8852	BladeCenter H	1
	46M6005	Voltaire 40 Gb InfiniBand Switch Module	2
	49Y9980	3 m Copper QDR InfiniBand QSFP Cable	Up to 32*

* The Voltaire 40 Gb InfiniBand Switch Module has 16 external ports. To communicate outside of the chassis, you must have QSFP cables connected. You have the flexibility to expand bandwidth using from one to 16 connections per switch.

Figure 3 shows the left side of the Voltaire 40 Gb InfiniBand Switch Module front panel.

Figure 3. Left side of front panel of the Voltaire 40 Gb InfiniBand Switch Module*

* The port LEDs on the right side of front panel of the Voltaire 40 Gb InfiniBand Switch Module are the same as on the left side.

The front panel contains the following components:

• Three status LEDs display the status of the switch module and the network:
  • OK (indicating that the switch module has passed the power-on self-test (POST) with no critical faults and is operational)
  • Switch-module error (indicating that the switch module has failed the POST or detected an operational fault)
  • SM (reserved for future use)
• 16 QSFP port connectors to attach QSFP cables.
  • Each external QSFP port on the switch module contains IB physical link and IB logical link LEDs

The Voltaire 40 Gb InfiniBand Switch Module supports the following cable:

• 3 m Copper QDR InfiniBand QSFP Cable, part number 49Y9980

For more information, see the following documents:

• IBM US Announcement Letter for the Voltaire 40 Gb InfiniBand Switch Module
  http://ibm.com/common/ssi/cgi-bin/ssialias?infotype=dd&subtype=ca&&htmlfid=897/ENUS109-377
• Voltaire 40 Gb InfiniBand Switch Module Installation Guide
  http://www.ibm.com/support
• BladeCenter Interoperability Guide
  http://lenovopress.com/bcig
• BladeCenter Products and Technology, SG24-7523
  http://lenovopress.com/sg247523