2/4 Port Ethernet Expansion Card (CFFh) for IBM BladeCenter

The IBM 2/4 Port Ethernet Expansion Card (CFFh) for IBM BladeCenter is the perfect example of IBM's goal to meet user requirements in the BladeCenter environment. In addition to the two onboard Ethernet ports, this card allows the client to add up to four (in IBM BladeCenter H chassis) or two (in BladeCenter S) extra 1 Gb ports, thereby allowing them to take advantage of the benefits of 6 or 4 ports per blade respectively.

Figure 1 shows the 2/4 Port Ethernet Expansion Card (CFFh) for IBM BladeCenter.


Figure 1. 2/4 Port Ethernet Expansion Card (CFFh) for IBM BladeCenter
 

The use of the 2/4 Port Ethernet Expansion Card enables Ethernet redundancy in the BladeCenter S chassis. Two Ethernet ports are already enabled through the onboard Ethernet controller of each blade server when routed to the Ethernet switch module in bay 1 of the chassis. With this card in each server, two additional ports are enabled and are routed through a second redundant Ethernet switch module in bay 2. This combination provides four Gigabit Ethernet ports while still enabling redundancy.

 

Table 1. Ordering part number and feature code

The part number includes the following items:

• One 2/4 Port Ethernet Expansion Card (CFFh)
• Installation and User's Guide
• Documentation CD

The expansion card has the following features and benefits:

• CFFh form factor; Can be combined with a CFFv or CIOv expansion card in the same server
• Based on the Broadcom 5709 module
• PCI Express x4 host interface for high-speed connection
• Connectivity to either standard or high-speed I/O modules bays (depends on chassis)
• Multiple connections from the blade server to the external network
• Ability to function as a 2-port Ethernet NIC in BC S chassis or a 4-port Ethernet NIC in a BC H chassis
• Supports BladeCenter Open Fabric Manager (BOFM)

Performance features:

• Supports TCP offload engine (TOE)
• TCP, IP checksum offload
• TCP segmentation offload
• PXE 2.0 remote boot support

This is supported in the following environment:

• Temperature: 10 to 35 °C (50 to 95 °F)
• Relative humidity: 8% to 80% (non-condensing)

The 2/4 Port Ethernet Expansion Card (CFFh) for is supported in the IBM BladeCenter servers listed in Table 2.

Table 2. Supported servers

2/4 Port Ethernet Expansion Card (CFFh)	44W4479	Y	Y	Y	Y	Y	Y	Y	Y	N	N	Y	N

Figure 2 shows where the CFFh card is installed in a BladeCenter server.


Figure 2. Location on the BladeCenter server planar where the CFFh card is installed
 

IBM BladeCenter chassis support is based on the blade server type in which the expansion card is installed. Consult ServerProven to see which chassis each blade server type is supported in: http://ibm.com/servers/eserver/serverproven/compat/us/.

The 2/4 Port Ethernet Expansion Card (CFFh) is supported attached to blades installed in the BladeCenter chassis as listed in Table 3.

Table 3. I/O modules supported with the Broadcom 10Gb 2-Port and 4-Port Ethernet Expansion Cards

2/4 Port Ethernet Expansion Card (CFFh)	44W4479	Y	N	Y	N	Y	Y	Y

Details of this support is described in the following sections.

BladeCenter H and BladeCenter HT compatibility

When attached to a blade server that is installed in a BladeCenter H, the 2/4 Port Ethernet Expansion Card is used in conjunction with the Multi-Switch Interconnect Module (MSIM), and requires that supported Gigabit Ethernet Switch Modules be installed in the MSIM. In order for all four Ethernet ports to be used, two MSIMs and four Gigabit Ethernet Switch Modules are required. The same applies to the BladeCenter HT and the MSIM-HT.

Table 4 lists the standard format Ethernet Switch Modules and Pass-thru Modules that are supported in the MSIM and MSIM-HT in conjunction with the 2/4 Port Ethernet Expansion Card (CFFh). Both bays of the MSIM are supported.

Table 4. Standard I/O modules supported in the MSIM and MSIM-HT bays in conjunction with the 2/4 Port Ethernet Expansion Card (CFFh)

Ethernet switch modules	Part number	Supported in MSIM	Supported in MSIM-HT
Cisco Intelligent Gb Ethernet Switch Module	32R1892	Yes	Yes
Cisco Intelligent Gb Fiber Ethernet Switch Module	32R1888	Yes	No
Cisco Catalyst Switch Module 3110G	41Y8523	Yes	No
Cisco Catalyst Switch Module 3110X	41Y8522	Yes	No
Cisco Catalyst Switch Module 3012	43W4395	Yes	No
Server Connectivity Module	39Y9324	Yes	No
BNT Layer 2-7 Gb Ethernet Switch Module	32R1859	No	No
BNT Layer 2/3 Copper Gb Ethernet Switch Module	32R1860	Yes	Yes
BNT Layer 2/3 Fiber Gb Ethernet Switch Module	32R1861	Yes	No
BNT 10Gb Uplink Ethernet Switch Module	32R1783	Yes	No
BNT 1/10 Gb Uplink Ethernet Switch Module	44W4404	Yes	Yes
Optical Pass-thru Module	39Y9316	Yes	No
Copper Pass-thru Module	39Y9320	Yes	No
Intelligent Copper Pass-thru Module	44W4483	Yes	No

Table 5 lists the High Speed Switch Modules that support connectivity to the 2/4 Port Ethernet Expansion Card (CFFh) in the BladeCenter H. All four ports of the expansion card are available for use when four supported switch modules are installed in the BladeCenter H chassis.

Table 5. High Speed Switch Modules supported with the 2/4 Port Ethernet Expansion Card (CFFh)

I/O module	Part number	Supported
BNT Virtual Fabric 10Gb Switch Module	46C7191	Yes
10Gb Ethernet Pass-Thru Module	46M6181	No
BNT 6-port 10Gb Ethernet Switch Module	39Y9267	No
Cisco Nexus 4001I Switch Module	46M6071	Yes

In BladeCenter H, the ports of CFFh cards are routed through the midplane to I/O bays 7, 8, 9, and 10, as shown in Figure 3. The BladeCenter HT is similar in that the CFFh cards are also routed through the midplane to I/O bays 7, 8, 9, and 10.


Figure 3. IBM BladeCenter H I/O topology showing the I/O paths from CFFh expansion cards

BladeCenter S compatibility

When attached to a blade server that is installed in a BladeCenter S chassis, only two ports are used and both of these two ports are routed to bay 2. A supported Gigabit Ethernet Switch Module will need to be installed in bay 2 of the BladeCenter S chassis.

Table 6 lists the Ethernet Switch Modules and Pass-thru Modules that are supported in bay 2 of the BladeCenter S chassis to be used in conjunction with the 2/4 Port Ethernet Expansion Card (CFFh).

Table 6. I/O modules supported in bay 2 of the BladeCenter S chassis in conjunction with the 2/4 Port Ethernet Expansion Card (CFFh)

Ethernet switch modules	Part number	Bay 2 of BC-S
Cisco Intelligent Gb Ethernet Switch Module	32R1892	No
Cisco Intelligent Gb Fiber Ethernet Switch Module	32R1888	No
Cisco Catalyst Switch Module 3110G	41Y8523	No
Cisco Catalyst Switch Module 3110X	41Y8522	No
Cisco Catalyst Switch Module 3012	43W4395	Yes
Server Connectivity Module	39Y9324	Yes
BNT Layer 2-7 Gb Ethernet Switch Module	32R1859	Yes
BNT Layer 2/3 Copper Gb Ethernet Switch Module	32R1860	Yes
BNT Layer 2/3 Fiber Gb Ethernet Switch Module	32R1861	Yes
BNT 10Gb Uplink Ethernet Switch Module	32R1783	Yes
BNT 1/10 Gb Uplink Ethernet Switch Module	44W4404	Yes
Optical Pass-thru Module	39Y9316	Yes
Copper Pass-thru Module	39Y9320	Yes
Intelligent Copper Pass-thru Module	44W4483	Yes

In BladeCenter S, only two ports are used on the card and both of these two ports are routed to bay 2 as shown in Figure 4.


Figure 4. IBM BladeCenter S I/O topology showing the I/O paths from CFFh expansion cards


 

This section shows two configurations that use the 2/4 Port Ethernet Expansion Card (CFFh).


BladeCenter H configuration

The 2/4 Port Ethernet Expansion Card (CFFh) and the BladeCenter H chassis makes it possible to supply eight Gigabit Ethernet ports to every BladeCenter server. Figure 5 shows the eight Ethernet switch modules, four of which are connected to the 2/4 Port Ethernet Expansion Card (CFFh). The other four connections are via the onboard Ethernet controller and the CIOv or CFFv expansion card. All connections between the cards and the switch modules are internal to the chassis. No extra cabling is needed.


Figure 5. Eight Ethernet connections to each blade in the BladeCenter H chassis

Table 7 lists the components that are used in the eiight Ethernet ports per server configuration shown in Figure 5.

Table 7. Components used in the eight ports per server configuration

Diagram reference	Part number / machine type	Description	Quantity
	Varies	IBM BladeCenter HS22 or other server that supports CFFh cards	1 to 14
	None	Ethernet controller on the system board of the server	1 per server
	Varies	Ethernet CFFv or CIOv expansion card	1 per server
	44W4479	2/4 Port Ethernet Expansion Card (CFFh)	1 per server
	8852	BladeCenter H chassis	1
	Varies	Support Ethernet Switch Modules routing signals from the integrated controller (see Table 4)	2
	Varies	Supported Ethernet Switch Modules routing signals from the CFFv or CIOv card (see Table 4)	2
	Varies	Supported Ethernet Switch Modules routing signals from the CFFh card (see Table 4)	4
	39Y9314	Multi-Switch Interconnect Module	2

BladeCenter S configuration

Figure 6 shows a configuration using the 2/4 Port Ethernet Expansion Card (CFFh) in a BladeCenter S configuration. This solution enables a total of four Gigabit Ethernet connections from each blade server, two routed through the Ethernet switch module in bay 2 of the BladeCenter S chassis. The other two connections are via the onboard Ethernet controller.


Figure 6. Four Ethernet connections to each blade in the BladeCenter S chassis

The components used in this configuration are listed in Table 8.

Table 8. Components used when connecting Broadcom 4-port 10Gb Ethernet Expansion Card (CFFh) to four BNT 10Gb High Speed Switch Modules

Diagram reference	Part number/ machine type	Description	Quantity
	Varies	IBM BladeCenter HS22 or other supported server	1 to 14
	44W4479	2/4 Port Ethernet Expansion Card (CFFh)	1 per server
	8886	BladeCenter S	1
	Varies	Supported Ethernet switch module to connect to the 2/4 Port Ethernet Expansion Card (CFFh) (See Table 5)	1
	Varies	Supported Ethernet switch module to connect to the onboard Ethernet controller (See Table 5)	1

The 2/4 Port Ethernet Expansion Card (CFFh) supports the following operating systems:

• Microsoft Windows Server 2003, Web Edition
• Microsoft Windows Server 2003/2003 R2, Datacenter x64 Edition
• Microsoft Windows Server 2003/2003 R2, Enterprise Edition
• Microsoft Windows Server 2003/2003 R2, Enterprise x64 Edition
• Microsoft Windows Server 2003/2003 R2, Standard Edition
• Microsoft Windows Server 2003/2003 R2, Standard x64 Edition
• Microsoft Windows Server 2008 R2
• Microsoft Windows Server 2008, Datacenter x64 Edition
• Microsoft Windows Server 2008, Datacenter x86 Edition
• Microsoft Windows Server 2008, Enterprise x64 Edition
• Microsoft Windows Server 2008, Standard x64 Edition
• Microsoft Windows Server 2008, Web x64 Edition
• NetWare 6.5
• Red Hat Enterprise Linux 4 AS for AMD64/EM64T
• Red Hat Enterprise Linux 4 AS for iSeries and pSeries
• Red Hat Enterprise Linux 4 AS for x86
• Red Hat Enterprise Linux 4 ES for AMD64/EM64T
• Red Hat Enterprise Linux 4 ES for x86
• Red Hat Enterprise Linux 4 WS/HPC for AMD64/EM64T
• Red Hat Enterprise Linux 4 WS/HPC for x86
• Red Hat Enterprise Linux 5 for System i and System p
• Red Hat Enterprise Linux 5 Server Edition
• Red Hat Enterprise Linux 5 Server Edition with Xen
• Red Hat Enterprise Linux 5 Server with Xen x64 Edition
• Red Hat Enterprise Linux 5 Server x64 Edition
• Red Hat Enterprise MRG 1.0 Realtime (x64)
• SUSE LINUX Enterprise Real Time 10 AMD64/EM64T
• SUSE LINUX Enterprise Server 10 for AMD64/EM64T
• SUSE LINUX Enterprise Server 10 for IBM POWER
• SUSE LINUX Enterprise Server 10 for x86
• SUSE LINUX Enterprise Server 10 with Xen for AMD64/EM64T
• SUSE LINUX Enterprise Server 10 with Xen for x86
• SUSE LINUX Enterprise Server 11 for AMD64/EM64T
• SUSE LINUX Enterprise Server 11 for x86
• SUSE LINUX Enterprise Server 11 with Xen for AMD64/EM64T
• SUSE LINUX Enterprise Server 9 for AMD64/EM64T
• SUSE LINUX Enterprise Server 9 for x86
• VMware ESX 3.5
• VMware ESX 4.0
• VMware ESX Server 3.0
• VMware ESXi 4.0

Support for operating systems is based on the combination of the expansion card and the blade server in which it is installed. See IBM ServerProven for the latest information about the specific versions and service packs supported: http://ibm.com/servers/eserver/serverproven/compat/us/. Select the blade server, and then select the expansion card to see the supported operating systems.

 

For more information refer to the following documents:

• 2/4 Port Ethernet Expansion Card (CFFh) for IBM BladeCenter Installation and User's Guide
  http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5082171
• IBM U.S. Announcement Letter for the 2/4 Port Ethernet Expansion Card (CFFh) for IBM BladeCenter
  http://ibm.com/common/ssi/cgi-bin/ssialias?infotype=dd&subtype=ca&&htmlfid=897/ENUS108-601
• IBM BladeCenter Interoperability Guide
  http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073016
• IBM Redbooks publication IBM BladeCenter Products and Technology, SG24-7523
  http://www.redbooks.ibm.com/abstracts/sg247523.html