Cisco Nexus B22 Fabric Extender for Flex System

The Cisco Nexus B22 Fabric Extender for Flex System™ (Cisco Nexus model B22IBM) is designed to simplify data center server access architecture and operations. Clients looking for Cisco connectivity inside the Flex System chassis can now leverage the new module to reduce management and offer easy connectivity to existing Nexus infrastructure.

The Cisco Nexus B22 Fabric Extender eliminates the need to configure each networking device or individual ports, therefore reducing the number of management points. It provides a low latency, loop-free interface that does not rely upon Spanning Tree Protocols, therefore removing one of the greatest deployment and management complexities of a traditional switch.

The Cisco Nexus B22 Fabric Extender for Flex System is shown in the following figure.

Figure 1. Cisco Nexus B22 Fabric Extender for Flex System

The Cisco Nexus B22 Fabric Extender behaves like a remote line card for a parent Cisco Nexus switch, together forming a distributed modular system. This architecture simplifies data center access operations and architecture by combining the management simplicity of a single high-density access switch with the cabling simplicity of switches integrated into a chassis and top-of-rack (ToR) access switches.

The B22 Fabric Extender provides transparent Flex System connectivity to your existing Cisco Nexus network. It aggregates compute node ports by appearing as a simple pass-thru device, and the upstream network sees a "large pipe" of server traffic coming to and from the chassis. With the B22 Fabric Extender, your network administration team continues to use the same network management tools that are deployed in the network to manage the connectivity from the physical servers in the chassis to the upstream network.

Integrated or mezzanine Virtual Fabric adapters, combined with the Cisco Nexus B22 Fabric Extender for Flex System, offer network flexibility with virtual network interface controllers (NICs) (vNICs) supporting Ethernet, Internet Small Computer System Interface (iSCSI), and Fibre Channel over Ethernet (FCoE) connectivity.

The following table shows the part numbers and feature codes for ordering the Cisco Nexus B22 Fabric Extender for Flex System.

The part number for the module includes the following items:

• One Cisco Nexus B22 Fabric Extender for Flex System
• 16 Cisco Fabric Extender Transceivers (Cisco part number FET-10G) (included only in 94Y5355)
• Documentation package

Note: Cisco Nexus B22 Fabric Extender for Flex System, part number 94Y5350, comes without SFP+ (small form-factor pluggable plus) transceivers or cables; they must be ordered separately (see Table 2).

The following table (Part 1 and Part 2) lists the supported cables and transceivers.

With the flexibility of the Cisco Nexus B22 Fabric Extender for Flex System, you can take advantage of the technologies that are required for multiple environments:

• For 10 GbE, you can use direct-attached cables (DACs), which come in lengths up to 10 m (32.8 ft.). These DACs are a cost-effective and low-power alternative to transceivers, and are ideal for all 10 Gb Ethernet connectivity within the rack, or even connecting to an adjacent rack. For longer distances, there is a choice of SFP+ transceivers (short reach (SR), long reach (LR), or extended reach (ER)).
• With direct-attach breakout cables, you can consolidate four 10 GbE SFP+ ports to a single 40 Gb upstream port on the Cisco Nexus, helping reduce the number of upstream ports.

The B22 Fabric Extender for Flex System offers the following key benefits for Cisco Nexus environments:

• Highly scalable, consistent access layer
  • Scalability: A deployment of Cisco Nexus B22 Fabric Extenders connected to a Cisco Nexus parent switch supports highly scalable 10 Gigabit Ethernet environments under a single managed entity.
  • Layer 2 dependability: Reliance on Spanning Tree Protocol is eliminated between the fabric extender and the parent switch, therefore enabling a large, multipath, loop-free topology. Use of a single management entity to support a large server domain allows policy to be enforced more efficiently and enhances Layer 2 data center access scalability. Use of the virtual PortChannel (vPC) feature allows fast convergence and effective utilization of bandwidth in Layer 2 environments.
• Simplified operations and management
  • Single point of management with reduction in management points: Cisco Nexus B22 Fabric Extenders are remote line cards for a Cisco Nexus parent switch. All device configurations are managed on the Cisco Nexus parent switch, and configuration information is downloaded to the Cisco Nexus B22 using in-band communication.
  • Software maintenance simplification: The Cisco Nexus B22 software is embedded in the Cisco Nexus parent switch software. The fabric extender is a "plug-and-play" device that automatically downloads the software image from the Cisco Nexus parent switch in the same way that a line card downloads software from the supervisor engine in a modular chassis. In-Service Software Upgrade (ISSU) on the fabric extenders provides the capability to perform transparent software upgrades, reducing downtime and allowing clients to integrate the newest features and functions with little or no effect on network operation for Ethernet, storage, and converged network environments.
  • Switch feature consistency across a large number of servers: The Cisco Nexus B22 forwards all traffic to the parent Cisco Nexus switch over 10 Gigabit Ethernet fabric uplinks. Passing all traffic to the parent switch allows traffic to be switched according to policies established on the parent Cisco Nexus switch with a single point of management. Standardizing on the Cisco Nexus switches allows data centers to support the same switch features across the entire access layer with a single point of management.
• Business benefits
  • Cost-effective 10 Gigabit Ethernet solution: The Cisco Nexus B22 is an excellent platform for migration from 1 Gigabit Ethernet to 10 Gigabit Ethernet. Scalable 10 Gigabit Ethernet provides 10 times the bandwidth of 1 Gigabit Ethernet for the less price of a 10 GbE port compared to the price of ten 1 GbE ports.
  • Consolidation: The Cisco Nexus B22 protects investment into the future, supporting evolving data center needs by providing an easy migration path to low-latency 10 Gigabit Ethernet, high-performance computing (HPC), virtual machine-aware networks. In addition, the combination of the Cisco Nexus 5000 Series and the Cisco Nexus B22 provides a unified network fabric that supports LAN and SAN consolidation. Another benefit of the Cisco fabric extender architecture is the capability to collapse data center access and aggregation layers into a single layer.
  • Investment protection: The Cisco Nexus B22 Fabric Extender can derive new functions from upstream Cisco Nexus switches, resulting in the capability to add new functions without the need for a major equipment upgrade.
  • Cabling reduction with unified fabric: The Cisco Nexus B22 can pass Ethernet and Fibre Channel over Ethernet (FCoE) traffic over the same set of uplinks onto the Cisco Nexus fabric, reducing the total number of uplinks required.
  • The Cisco Nexus B22 supports an optimal cabling strategy that simplifies network operations and prepares for future technologies. Low-cost connections of up to 10 meters (32.8 ft.) to the Cisco Nexus parent switch can be made with copper Twinax cable, and longer connections of up to 100 meters (328 ft.) can use the Cisco Fabric Extender Transceiver (FET). Standard 10 Gbps optics such as short reach (SR) and long reach (LR) are also supported for distances over 100 meters (328 ft.).
  • Effective bandwidth utilization: Through vPC feature support, each server can bond its network adapters together into a 20 Gbps or higher PortChannel, achieving both high bandwidth and link redundancy.
  • Reduced power and cooling: Cost-effective 10 Gigabit Ethernet solutions, optimal cabling, device consolidation, and efficient bandwidth utilization all contribute to a significant reduction in power and cooling needs in the data center.

The Cisco Nexus B22 Fabric Extender for Flex System has the following features and specifications:

• Internal ports:
  • Fourteen internal full-duplex auto-sensing 1/10 Gigabit Ethernet ports.
• External ports:
  • Eight ports for 10 Gb Ethernet SFP+ transceivers (support for 10GBASE-SR, 10GBASE-LR, or 10GBASE-ER) or SFP+ copper direct-attach cables (DAC). SFP+ modules and DACs are not included and must be purchased separately.
• Scalability and performance:
  • Wire-speed forwarding of traffic with aggregated throughput of 400 Gbps.
  • Static and EtherChannel link aggregation, up to 80 Gb (160 Gb full-duplex) of total fabric uplink bandwidth per module.
  • Support for jumbo frames (up to 9,216 bytes).
  • PortChannel on server ports.
• Availability and redundancy:
  • Redundant uplinks through Cisco EtherChannel hashing or static port pinning.
  • vPCs for dual-homed active-active connectivity across two Cisco Nexus parent switches.
  • vPCs for dual-homed straight-through NIC connectivity across two Cisco Nexus B22 Fabric Extenders.
  • In-Service Software Upgrade (ISSU).
• Security:
  • Access control lists (ACLs).
• Quality of service (QoS):
  • Support for IEEE 802.1p traffic classification and processing.
  • Eight hardware queues per port.
  • Per-port QoS configuration.
  • Local traffic policing.
  • Egress strict-priority queuing.
  • Egress port-based scheduling: Weighted Round Robin (WRR).
• Virtualization:
  • 802.1Q VLAN tagging support.
  • Switch Independent Virtual NIC (vNIC2): Ethernet, iSCSI, or FCoE traffic.
• Converged Enhanced Ethernet:
  • Priority-Based Flow Control (PFC) (IEEE 802.1Qbb) extends 802.3x standard flow control to allow the switch to pause traffic based on the 802.1p priority value in each packet's VLAN tag.
  • Enhanced Transmission Selection (ETS) (IEEE 802.1Qaz) provides a method for allocating link bandwidth based on the 802.1p priority value in each packet's VLAN tag.
  • Data Center Bridging Capability Exchange Protocol (DCBX) (IEEE 802.1AB) allows neighboring network devices to exchange information about their capabilities.
• Fibre Channel over Ethernet (FCoE):
  • FC-BB5 FCoE specification compliant.
  • FCoE transit switch operations.
  • FCoE Initialization Protocol (FIP) support.
• Manageability:
  • Fabric Extender management using in-band management.
  • Simple Network Management Protocol (SNMP v1, v2, and v3).
  • XML (NETCONF) support.
  • Cisco Discovery Protocol versions 1 and 2.
  • CiscoWorks support.
  • Cisco Data Center Network Manager (DCNM); the Cisco Nexus B22 is managed through the parent Cisco Nexus switch using Cisco DCNM and standard Simple Network Management Protocol (SNMP), XML interfaces, and the command-line interface (CLI).
• Monitoring:
  • Switch light-emitting diodes (LEDs) for external port status and switch module status indication.
  • Change tracking and remote logging with syslog feature.
  • Remote monitoring (RMON).
  • Cisco Switched Port Analyzer (SPAN) source on server ports.
  • Power-on self-test (POST) diagnostic tests.

The Cisco Nexus B22 Fabric Extenders support the following standards:

• IEEE 802.1AB Data Center Bridging Capability Exchange Protocol (DCBX)
• IEEE 802.1p Class of Service (CoS) prioritization
• IEEE 802.1Q Tagged VLAN (frame tagging on all ports when VLANs are enabled)
• IEEE 802.1Qbb Priority-Based Flow Control (PFC)
• IEEE 802.1Qaz Enhanced Transmission Selection (ETS)
• IEEE 802.3 Ethernet
• IEEE 802.3ae 10GBASE-SR short range fiber optics 10 Gb Ethernet
• IEEE 802.3ae 10GBASE-LR long range fiber optics 10 Gb Ethernet
• IEEE 802.3ae 10GBASE-ER extended range fiber optics 10 Gb Ethernet
• IEEE 802.3ap 10GBASE-KR backplane 10 Gb Ethernet
• IEEE 802.3x Full-duplex Flow Control
• SFF-8431 10GSFP+Cu SFP+ Direct Attach Cable

The I/O modules are installed in switch bays in the rear of the Flex System Enterprise Chassis, as shown in the following figure. I/O modules are normally installed in pairs because ports on the I/O adapters that are installed in the compute nodes are routed to two switch bays for redundancy and performance.

Figure 2. Location of the switch bays in the Flex System Enterprise Chassis

Figure 3. Logical layout of the interconnects between I/O adapters and B22 Fabric Extenders

The Cisco Nexus B22 Fabric Extender for Flex System can be installed in bays 1, 2, 3, and 4 of the Enterprise Chassis. A supported adapter must be installed in the corresponding slot of the compute node (slot A1 when interconnect modules are installed in bays 1 and 2 or slot A2 when the modules are in bays 3 and 4). With four-port and eight-port adapters, only two ports are used for communications, and the remaining ports (two or six ports respectively) are not used.

In compute nodes that have an integrated dual-port 10 GbE network interface controller (NIC), NIC ports are routed to bays 1 and 2 with a specialized periscope connector, and the adapter in slot A1 is not required. However, when needed, the periscope connector can be replaced with the adapter. In such a case, the integrated NIC is disabled.

x222 compute node: The x222 compute nodes are not supported with the Cisco Nexus B22 Fabric Extender for Flex System.

Table 3. Adapter to I/O bay correspondence with the Cisco Nexus B22 Fabric Extender for Flex System

I/O adapter slot in the server	Port on the adapter	Corresponding I/O bay in the chassis
		Bay 1	Bay 2	Bay 3	Bay 4
Slot 1 (or integrated dual-port LOM - ports 1 and 2)	Port 1	Yes
	Port 2		Yes
Slot 2	Port 1			Yes
	Port 2				Yes
Slot 3 (full-wide compute nodes only)	Port 1	Yes
	Port 2		Yes
Slot 4 (full-wide compute nodes only)	Port 1			Yes
	Port 2				Yes

The following table lists the I/O adapters that are supported by the Cisco Nexus B22 Fabric Extender for Flex System.

* The Embedded 10Gb Virtual Fabric Adapter is included in some models of the x240 and x440.
# With four-port or eight-port adapters, only two adapter ports can be used, and the remaining ports are not used.

Figure 4. Location of the I/O adapter slots in the Flex System x240 Compute Node

The Cisco Nexus B22 Fabric Extender for Flex System supports connectivity to the following Cisco Nexus top-of-rack switches:

• Cisco Nexus 7700 Series (Requires NX-OS software version 7.2(0)D1(1) or later)

  • Cisco Nexus 7718 Switch
  • Cisco Nexus 7710 Switch
  • Cisco Nexus 7706 Switch
  • Cisco Nexus 7702 Switch
• Cisco Nexus 7000 Series (Requires NX-OS software version 7.2(0)D1(1) or later)
  • Cisco Nexus 7018 Switch
  • Cisco Nexus 7010 Switch
  • Cisco Nexus 7009 Switch
  • Cisco Nexus 7004 Switch
• Cisco Nexus 6000 Series
  • Cisco Nexus 6004 Switch
  • Cisco Nexus 6001 Switch
• Cisco Nexus 5600 Series
  • Cisco Nexus 56128P Switch
  • Cisco Nexus 5696Q Switch
  • Cisco Nexus 5672UP Switch
  • Cisco Nexus 5624Q Switch
• Cisco Nexus 5500 Series
  • Cisco Nexus 5596T Switch
  • Cisco Nexus 5596UP Switch
  • Cisco Nexus 5548P Switch
  • Cisco Nexus 5548UP Switch

The following figure shows the front panel of the Cisco Nexus B22 Fabric Extender for Flex System.

Figure 5. Front panel of the Cisco Nexus B22 Fabric Extender for Flex System

The front panel contains the following components:

• LEDs that display the status of the module and the network:
  • The OK LED indicates that the interconnect module passed the power-on self-test (POST) with no critical faults and is operational.
  • Identify: This blue LED can be used to identify the module physically by illuminating it through the management software.
  • The error LED (switch module error) indicates that the module failed the POST or detected an operational fault.
• Eight external SFP+ ports for 10 Gb connections to external Cisco Nexus devices.
• An Ethernet link OK LED and an Ethernet Tx/Rx LED for each external port.

The network cables that can be used with the Cisco Nexus B22 Fabric Extender for Flex System are shown in the following table.

The Cisco Nexus B22 Fabric Extender for Flex System carries a 1-year, customer-replaceable unit (CRU) limited warranty. When installed in a chassis, these I/O modules assume your system's base warranty and any Lenovo warranty service upgrade. Software support for the B22 Fabric Extender is offered via a support package for the upstream Cisco Nexus switch to which the B22 Fabric Extender is connected.

Dimensions and weight of the Cisco Nexus B22 Fabric Extender for Flex System (approximate):

• Height: 30 mm (1.2 in.)
• Width: 402 mm (15.8 in.)
• Depth: 297 mm (11.7 in.)
• Weight: 3.2 kg (7.0 lb)

Shipping dimensions and weight (approximate):

• Height: 114 mm (4.5 in.)
• Width: 508 mm (20.0 in.)
• Depth: 432 mm (17.0 in.)
• Weight: 4.1 kg (9.1 lb)

The Cisco Nexus B22 Fabric Extender for Flex System conforms to the following regulations:

• UL 60950
• CAN/CSA-C22.2 No 60950
• IEC/EN 60950
• FCC 47 CFR Part 15, Class A
• AS/NZS CISPR 22, Class A
• CISPR 22, Class A
• CE Mark (EN55022 Class A, EN55024, EN61000-3-2, EN61000-3-3)
• ICES003, Class A
• VCCI, Class A
• KN22, Class A

The most common connectivity topologies for the Cisco Nexus B22 Fabric Extender for Flex System that can be used with Cisco Nexus upstream network devices are shown in Figure 6 and Figure 7.

Figure 6. B22 Fabric Extender connectivity topology - Link Aggregation

In this sample loop-free redundant topology, each Cisco Nexus B22 Fabric Extender for Flex System is physically connected to a separate Top-of-Rack (ToR) switch with static or EtherChannel aggregated links. Compute node NICs are configured in an active/active NIC teaming.

Figure 7. B22 Fabric Extender connectivity topology - Virtual Link Aggregation

In this loop-free topology, aggregation is split between two physical switches, which appear as a single logical switch, and each Cisco Nexus B22 Fabric Extender for Flex System is connected to both ToR switches through static or EtherChannel aggregated links. Compute Node NICs can be configured in an active/passive or active/active NIC teaming.

With both topologies, each port of the compute node's NIC can be divided in up to four virtual NICs (vNICs).

Cisco Nexus B22 Fabric Extender for Flex System supports Data Center Bridging (DCB), and it can transport FCoE frames by using FCoE Initialization Protocol (FIP) snooping. These B22 Fabric Extenders provide an inexpensive solution for transporting encapsulated FCoE packets to the Fibre Channel Forwarder (FCF), which is functioning as both an aggregation switch and an FCoE gateway, such as the Cisco Nexus 5000 Series. The following figure shows a sample scenario.

Figure 8. B22 Fabric Extender in the FCoE network with the Cisco Nexus 5548/5596 as an FCF

Table 6. B22 Fabric Extender with the Cisco Nexus 5548/5596 as an FCF (Figure 8)

Diagram reference	Description	Part number	Quantity
	Flex System FCoE solution
	Flex System x240 Compute Node with Embedded Virtual Fabric Adapter	Varies	Varies
	Flex System Embedded 10Gb Virtual Fabric Upgrade	90Y9310	1 per server
	Cisco Nexus B22 Fabric Extender with FET bundle for Flex System	94Y5355*	2 per chassis
	Cisco Nexus 5548/5596 Switch
	Cisco MDS SAN fabric
	FC storage targets
	IBM System Storage DS3000 / DS5000
	IBM System Storage DS8000
	IBM Storwize V7000 / SAN Volume Controller
	IBM XIV
	IBM System Storage tape backup offerings

* Includes 16 Cisco Fabric Extender Transceivers (FET-10G): eight for the B22 Fabric Extender and eight for the upstream Cisco Nexus switch.

Lenovo provides extensive FCoE testing to deliver network interoperability. For a full listing of supported FCoE and iSCSI configurations, see the System Storage Interoperation Center (SSIC) website:
http://ibm.com/systems/support/storage/ssic

For more information, see the following Cisco Nexus B22 Fabric Extender for Flex System product publication, which is available from the Flex System Information Center at
http://publib.boulder.ibm.com/infocenter/flexsys/information:

• Installation and User Guide

The Cisco Nexus B22 Blade Fabric Extender Data Sheet is available from:
http://www.cisco.com/go/nexusb22

The following documents are other useful references:

• Flex System Enterprise Chassis Product Guide
  http://lenovopress.com/tips0865
• Lenovo Press publication Flex System Products and Technology for x86 Systems, SG24-8255
  http://lenovopress.com/sg248255
• Flex System Interoperability Guide
  http://lenovopress.com/fsig