Cisco Nexus 2000 (Fabric Extender or FEX)
•
Nexus
2000 Series Fabric Extenders behave logically like remote line cards for a
parent Cisco Nexus 5000 or 7000 Series Switch. They simplify data centre access
operations and architecture as well as management from the parent switches.
They deliver a broad range of connectivity options, including 40 Gigabit
Ethernet, 10 Gigabit Ethernet, 1 Gigabit Ethernet, 100 MB and Fibre Channel
over Ethernet (FCoE).
•
The
Cisco Nexus 2000 Series Fabric Extenders work in conjunction with a Cisco Nexus
parent switch to deliver cost-effective and highly scalable Gigabit Ethernet
and 10 Gigabit Ethernet environments while facilitating migration to 10 Gigabit
Ethernet, virtual machine–aware, and unified fabric environments.
The Cisco
Nexus 2000 Series has extended its portfolio to provide more server
connectivity choices and to support Cisco Nexus switches upstream. With more
flexibility and choice of infrastructure, we gain the following benefits:
•
Architectural flexibility :
- Provides a comprehensive range of
connectivity options—100 Megabit Ethernet, Gigabit Ethernet, and 10 Gigabit
Ethernet server connectivity and unified fabric
environments—and supports copper and fibre Gigabit Ethernet and10 Gigabit
Ethernet connectivity options with 1GBASE-T, SFP and SFP+, and CX1 over copper
and fibre cables
− Supports various server form
factors: rack and blade servers
− Offers space optimized for both
ToR and EoR topologies
− Provides
port-density options: 24, 32, and 48 ports
−
Enables quick expansion of network capacity
•
Highly
scalable server access
•
−
Provides highest density per rack unit
•
−
Allows transparent addition of network capacity as needed, reducing initial
capital expenditures (CapEx)
•
−
Enables quick expansion of network capacity by rolling in a prewired rack of
servers with a ToR fabric extender and transparent connectivity to
an upstream Cisco Nexus parent switch
•
Simplified
operations
•
−
With Cisco Nexus 5000 or 7000 Series, provides a single point of management and
policy enforcement
•
The
Cisco Nexus 2000 Series Fabric Extender forwards all traffic to its parent
Cisco Nexus 5000 Series
•
device
over 10-Gigabit Ethernet fabric uplinks, allowing all traffic to be inspected
by policies established
•
on
the Cisco Nexus 5000 Series device. No software is included with the Fabric
Extender. Software is
•
Automatically
downloaded and upgraded from its parent switch. The Nexus 2248T will allow
100/1000
•
Connectivity
and can be dual attached to the Nexus 5000. By dual attaching the Nexus 2248Ts
to the 5000, it will allow for the most resilient connections for single
attached servers.
•
The
Cisco Nexus 2000 Series provides two types of ports: ports for end-host
attachment (host interfaces) and uplink ports (fabric interfaces). Fabric
Interfaces are differentiated with a yellow colour (as shown in above figure)
for connectivity to the upstream parent Cisco Nexus switch.
•
Each
fabric extender module should be assigned a unique number (between 100-199).
This unique number enables the same fabric extender to be deployed in
single-attached mode to one CiscoNexus 5000 Series Switch only or in fabric
extender vPC mode (that is, dual-connected to different Cisco Nexus 5000 Series
Switches).
•
Nexus
2000 Fabric Extenders are not independent manageable entities; the Nexus 5000
manages the fabric extender through in-band connectivity.
•
Nexus
2000 Series can be attached to the Nexus 5000 Series in two different
configurations:
•
Static
pinning: The front host ports on the module are divided across the fabric ports
(that is, the uplinks connecting to the Nexus 5000).
•
Port-Channel:
The fabric ports form a Port-Channel to the Cisco Nexus 5000.
•
FEX
Configuration Example
#sh feature
| in en
•
N5K(config)#feature
fex
•
N5K(config)#interface
Ethernet 1/13-14
•
N5K(config-if-range)#channel-group
100
•
N5K(config-if-range)#no
shutdown
•
N5K(config-if-range)#interface
port-channel 100
•
N5K(config-if)#switchport
mode fex-fabric
•
N5K(config-if)#
fex associate 100
•
N5K(config-if)#fex
100
•
N5K(config-fex)#description
FEX 100 Eth1/13-14
•
While checking fex links, we got the
" Identity-Mismatch" error as shown below in "sh int
fex" output:
•
Nexus-5k-1# sh int fex-fabric
•
Fabric
Fabric Fex
FEX
•
Fex Port
Port State Uplink Model
Serial
•
---------------------------------------------------------------
•
103 Eth1/17
Active 1 N2K-C2248TP-1GE
JAX1122AAA
•
103 Eth1/18
Active 2 N2K-C2248TP-1GE JAX1122AAA
•
103 Eth1/19
Active 3 N2K-C2248TP-1GE
JAX1122AAA
•
103 Eth1/20
Active 4 N2K-C2248TP-1GE
JAX1122AAA
•
105 Eth1/23
Active 1 N2K-C2248TP-1GE
MLX1122BBB
•
105 Eth1/24
Active 2 N2K-C2248TP-1GE MLX1122BBB
•
105 Eth1/25 Identity-Mismatch 4
N2K-C2248TP 1GE PQR3344DDD
<<<Notice this
•
105 Eth1/26
Active 4 N2K-C2248TP-1GE
MLX1122BBB
•
Nexus-5k-2# sh int fex-fabric
•
Fabric
Fabric Fex
FEX
•
Fex Port
Port State Uplink Model
Serial
•
---------------------------------------------------------------
•
102 Eth1/17
Active 1 N2K-C2248TP-1GE
LMN2244CCC
•
102 Eth1/18
Active 2 N2K-C2248TP-1GE
LMN2244CCC
•
102 Eth1/19
Active 3 N2K-C2248TP-1GE
LMN2244CCC
•
102 Eth1/20
Active 4 N2K-C2248TP-1GE
LMN2244CCC
•
104 Eth1/23
Active 1 N2K-C2248TP-1GE
PQR3344DDD
•
104 Eth1/24
Active 2 N2K-C2248TP-1GE PQR3344DDD
•
104 Eth1/25
Active 3 N2K-C2248TP-1GE
PQR3344DDD
•
104 Eth1/26 Identity-Mismatch 3
N2K-C2248TP 1GE MLX1122BBB <<<Notice this
•
Basically this error is related to
incorrect cabling..
•
As we know that a nexus 2k switch or
FEX is connected to its parent Nexus 5k over fex links.
•
One Fex (2k) can be dual homed to two
Nexus 5k switches. and when a nexus 2k is connected to Nexus 5k, a unique fex
associate number is assigned to that particular 2k to identify it uniquely
•
So, i had four nexus 2k switches
whose serial numbers are JAX1122AAA,MLX1122BBB, PQR3344DDD and
LMN2244CCC.
•
JAX1122AAA and ,MLX1122BBB are
FEX switches for Nexus5k1.
•
and PQR3344DDD and
LMN2244CCC are part of Nexus-5k-2.
•
JAX1122AAA has been given FEX
associate number 103 and MLX1122BBB has been given 105,
•
LMN2244CCC is assigned 102 and
PQR3344DDD is assigned 104.
•
Each fex is connected to its parent
switch via 4 fex links.
•
Idealy, all 4 fex links which are
under same FEX ASSOCIATE NUMBER should be going to same 2k,however, one of our
onsite engineer incorrectly cabled one of the fex link from 103 on Nexus-5k-1
to another 2k which was part of FEX number 104 on Nexus-5k-2 and we started
getting identity mismatch. As you can see in above output,under FEX 105 on Nexus-5k-1,
the Eth1/25 is showing PQR3344DDD serial number,however, all other
interfaces showing MLX1122BBB and vice versa on Nexus-5k-2 for
Eth1/26.
•
In order to verify cabling and make
sure right fex or 2k is connected to correct parent 5k switch with respective
to its FEX associate number, we can use "show interface fex-fabric"
command and verify the same using serial number that all are correct switches.
•
once the cable were swapped, we
started getting right serial number for Eth1/25.
Nexus-5k-1# sh int fex-fabric
Fabric Fabric
Fex
FEX
Fex Port
Port State Uplink Model
Serial
---------------------------------------------------------------
103 Eth1/17
Active 1 N2K-C2248TP-1GE JAX1122AAA
103 Eth1/18
Active 2 N2K-C2248TP-1GE
JAX1122AAA
103 Eth1/19
Active 3 N2K-C2248TP-1GE
JAX1122AAA
103 Eth1/20
Active 4 N2K-C2248TP-1GE
JAX1122AAA
105 Eth1/23
Active 1 N2K-C2248TP-1GE MLX1122BBB
105 Eth1/24
Active 2 N2K-C2248TP-1GE
MLX1122BBB
105 Eth1/25
Active 3 N2K-C2248TP-1GE
MLX1122BBB -->>>>Correct now
105 Eth1/26
Active 4 N2K-C2248TP-1GE
MLX1122BBB
