Implementing Cisco UCS Solutions

Chapter 1. Cisco UCS Physical Architecture and Installing UCS Hardware

In previous decades, computers evolved at a dramatic pace. Moore's law, which predicted that the density of transistors and integrated circuits would double every two years as computing components kept on shrinking in size while improving in computational capacity, has truly prevailed. This technological evolution led to three distinct generations of computing devices.

We witnessed the era of the following generations:

Gigantic mainframe computers
Commoditized personal computers and tower and rack servers (also known as pizza-box servers)
Blade servers (also known as modular servers)

Mainframes were monolithic systems often with proprietary hardware and software. With their enormous computing power, mainframes were able to run multiple applications; however, their major limitations were cost and many single points of failure. Due to their high cost and management complexity, mainframes remained mainly confined to military use, universities, and some very large organizations.

Tower and rack-mounted servers usually have limited computational resources as compared to mainframes; however, these are very cost effective. Because of the limited resources available on these servers, a one-to-one server-to-application ratio is usually the way to go. Because of this one server one application design, rack and tower servers need more rack space, separate cabling, individual power supplies, and more cooling in the datacenter, which makes management of the infrastructure quite complex. However, this second generation of computers is generally very cost effective. This led to the mass adoption of computers everywhere.

The latest trend in the ever evolving computer architecture space is to move away from tower and rack-mounted servers in favor of blade servers. In today's highly demanding enterprise applications, blade server architecture provides excellent features when compared with rack and tower servers. These features include the following:

Less rack space usage
Less cabling
Shared power
Consolidated I/O
Easy management
Excellent heating and cooling

Contemporary datacenters are facing unprecedented growth in computational demands alongside the need for reducing implementation and operational costs. Considering these factors, blade servers are designed to minimize the use of resources and space. Components in a blade chassis are either removed or shared between blade servers.

The minimum form factor of a rack server is 1 Rack Unit (RU). 1 RU is equal to 1.75 inches, and the most common server rack height is usually 42 RU. It is therefore possible to fit only 42 pizza-box servers in a standard rack. With blade servers, it is possible to achieve higher densities of servers per rack.

In a blade server, data connectivity interfaces and power supplies are also shared. Thus, blade servers also require less cabling, and hence less management.

In this chapter, we will discuss physical components of the Unified Computing System (UCS) equipment. The list of the topics covered in this chapter is as follows:

A quick look at the UCS equipment
Understanding the physical architecture of UCS
Understanding Fabric Interconnects (FIs)
Cisco UCS 5100 series blade server chassis
IOM modules
Blade servers and rack-mount servers
Getting started with mezzanine cards
Power capacity and power plugs
Installing UCS chassis components
Cabling FI and IOM

Looking at the UCS equipment

With the ever increasing demand on datacenters, vendors started focusing on different aspects of server and networking hardware consolidation; however, most of the ad hoc solutions were based on gluing together the existing products which were not designed grounds up to provide a cohesive infrastructure and failed to address the requirements of the datacenter as a whole. Hence, management of these amalgamated solutions was a nightmare for IT administrators.

Cisco entered into the blade server market with a holistic approach to the blade server design. With a strong background in networking and storage products, Cisco developed a cohesive solution consolidating the computing, network, and storage connectivity components along with centralized management of these resources. The purpose of Cisco UCS is to reduce the Total Cost of Ownership (TCO) and improve scalability and flexibility.

Salient features and benefits of the UCS solution include the following:

Stateless computing
Rapid provisioning of servers
Simplified troubleshooting
Virtualization readiness
Choice of industry-standard form factors
Extended memory technology for increased density

Stateless computing

Cisco introduced the idea of stateless computing with its blade server design. Cisco blade servers do not have any initial configuration. Universally Unique Identifiers (UUIDs) for blades, Network Interface Cards (NICs), Media Access Control (MAC) addresses, storage World Wide Node (WWN) numbers, firmware, and BIOS settings are all abstracted from Unified Computing System Manager (UCSM), the management software running on the FIs.

Rapid provisioning of servers

Provisioning of servers dramatically improves as the servers can be provisioned using the UCSM software even before they are physically available. Once the server is physically installed, it will abstract its identity from UCSM. Using server configuration templates, it is therefore possible to create a server template only once and apply the template to hundreds of servers.

Simplified troubleshooting

Replacement of servers also becomes very easy. Since the servers are stateless, as soon as a replacement server is installed, it will abstract all the configuration of the old server and will be available for use. Servers can also be easily migrated for different roles and workloads.

Virtualization readiness

Virtualization in the form of modern bare metal hypervisors is a major breakthrough for optimal utilization of computational resources. Cisco UCS solution supports all major hypervisor platforms including VMware ESX/ESXi, Microsoft Hyper-V, and Citrix Xen server. Support and integration with VMware vSphere solution is very strong. UCSM can be integrated with vCenter to abstract and manage features at the individual Virtual Machine (VM) level. By leveraging the benefits of virtualization and increasing the density of the physical server, the UCS solution can scale up to thousands of VMs.

Choice of industry-standard form factors

Cisco UCS servers are available in two categories: B-series blade servers and C-series rack-mount servers. Both form factors are designed using the same industry-standard components and can address different computational requirements. Both B-series blade servers and C-series rack-mount servers are designed using Intel Xeon CPUs. B-series servers are managed through UCSM, whereas C-series servers can either be individually managed or can be integrated to UCSM.

Extended memory technology for increased density

Cisco also introduced a patented extended memory technology for two CPU socket servers to increase the total amount of memory support; this could be more than double the amount of memory as compared to the industry standards for two socket servers. Virtualized workloads can leverage this extra memory to support an even greater density of VMs in a reduced physical footprint, resulting in reduced Capital Expenditure (CAPEX) and Operational Expenditure (OPEX) costs. Extended memory technology is available in both B-series blade servers and C-series rack-mount servers.

Understanding the physical architecture of UCS

The Cisco UCS solution can be divided into the following three main categories:

The Cisco UCS FIs
The Cisco UCS blade servers
The Cisco UCS rack-mount servers

The Cisco UCS FIs

The Cisco UCS FIs provide network connectivity and management for the connected servers. The UCS FIs run the UCSM control software and consist of the following components:

6100 series and 6200 series FIs
Transceivers for network and storage connectivity
Expansion modules for both FI series
UCSM software

The Cisco UCS blade servers

The Cisco UCS blade servers require a mother chassis where these servers can be installed. The UCS blade server solution consists of the following components:

A blade server chassis, used for installing blades
B-series blade servers
IOM modules for connectivity to the FIs
Transceivers for connectivity to the FIs
Mezzanine cards for network and storage connectivity

The Cisco UCS rack-mount servers

The Cisco UCS rack-mount servers are standalone servers that can be installed and controlled individually. Cisco provides Fabric Extenders (FEXs) for the rack-mount servers. FEXs can be used to connect and manage rack-mount servers from FIs. The UCS rack-mount server solution consists of the following components:

UCS C-series rack-mount servers
FEXs for connecting rack-mount servers to FIs

In this chapter we will provide details about hardware options available for both blade servers and rack-mount servers. Most of the field deployments of UCS servers are based on blade servers. Therefore, our main focus will be on blade server configuration in this book. However, if proper connectivity is established between rack-mount servers and FIs, rack-mount servers can also be managed in the same fashion.

The following figure depicts the FIs running the UCSM software, a blade server chassis with IOM modules, and the main components of a blade server:

In this chapter, we will go into the details of various UCS components and will focus on their physical specifications and installation in the subsequent sections.

Note

The Cisco interactive 3D model for Cisco 5100 series chassis and blades is an excellent resource for exploring Cisco UCS components physically. It is also available for iPhone/iPad (search for UCS Tech Specs in the marketplace). More details on this are available at http://www.cisco.com/en/US/prod/ps10265/ps10279/ucs_kaon_model_preso.html.

Understanding FIs

An FI is the core component of a UCS solution. FIs are typically configured as highly available clustered pairs in production environments. It's possible to run a single FI-based design as a proof of concept test deployment before actually implementing it in production. FIs provide the following two capabilities:

Network connectivity to both LAN and SAN
UCS infrastructure management through the embedded management software, UCSM, for both hardware and software management

FIs are available in two generations, namely Cisco UCS 6100 series and Cisco UCS 6200 series. The core functionality is the same in both generations; however, UCS 6200 series has a newer generation Application Specific Integrated Circuit (ASIC), higher throughput, and increased number of physical ports. Both generations can be upgraded to the latest UCSM software.

FIs provide converged ports. Depending on the physical Small Form Factor Pluggable (SFP) transceivers and FI software configuration, each port can be configured in different ways. Cisco 6200 series FI ports can be configured as Ethernet ports, Fiber Channel over Ethernet (FCoE) ports, or Fiber Channel (FC) ports. On the other hand, 6100 series converged ports only support Ethernet and FCoE (they also support FC, but only in the expansion slot).

In production, FIs are deployed in clustered pairs to provide high availability. Cisco-supported implementation requires that clustered FIs be identical. The only possibility for having different FIs in a cluster is during a cluster upgrade.

Tip

Larger enterprises may consider deploying the Cisco UCS central software, which can manage multiple UCS domains across globally distributed datacenters.

The following are the specifications of all the available UCS FIs.

The Cisco 6296UP FI

The Cisco 6296UP FI (UP represents Unified Ports) is a 2 RU device with a maximum of 96 converged ports. Ports can be configured as 1 GB Ethernet, 10 GB Ethernet, 10 GB FCoE, and 2/4/8 GB FC. The specifications of this FI are as follows:

A maximum of 20 blade server chassis per FI
A fabric throughput of 1920 Gbps
A 48-port base unit with three expansion slots (each expansion slot can add 16 ports)

The Cisco 6248UP FI

The Cisco 6248UP FI is a 1 RU device with a maximum of 48 converged ports. Ports could be configured as 1 GB Ethernet, 10 GB Ethernet, 10 GB FCoE, and 2, 4, or 8 GB FC. The specifications of this FI are as follows:

A maximum of 20 blade server chassis per FI
A fabric throughput of 960 Gbps
A 32-port base unit with one expansion slot that can provide 16 extra ports

The Cisco 6140UP FI

The Cisco 6140UP FI is a 2 RU device with a maximum of 48 converged ports. Fixed ports can be configured as 10 GB Ethernet and 10 GB FCoE. Only the first 16 ports can be configured as 1 GB Ethernet. The FC is only supported in the expansion module ports. The specifications of this FI are as follows:

A maximum of 20 blade server chassis per FI
A fabric throughput of 1040 Gbps
A unit of 40 fixed ports with two expansion slots where each expansion slot can provide eight FC ports of 2, 4, or 8 Gbps or six 10 Gbps SFP+ ports

The Cisco 6120UP FI

The Cisco 6120UP FI is a 1 RU device with a maximum of 20 fixed 10 GB ports. Fixed ports can be configured as 10 GB Ethernet and 10 GB FCoE. Only the first eight ports can be configured as 1 GB Ethernet. The FC is only supported in the expansion module ports. The specifications of this FI are as follows:

A maximum of 20 blade server chassis per FI
A fabric throughput of 520 Gbps
A unit of 20 fixed ports with one expansion slot, which can provide eight 2/4/8 Gbps FC or six 10 Gbps ports

Note

Standard pricing of FIs provides a limited number of port licenses. To enable more ports, extra licenses can be purchased per port.

Exploring connectivity transceivers for FIs

A variety of SFP transceivers are available for both FI series. These transceivers provide south-bound IOM connectivity and north-bound network and storage connectivity. They are based on industry-standard SFP+ specifications.

Transceivers can be selected depending on the technology, for example, Ethernet or FC, and also according to the distance requirements. For shorter distances between FIs, IOMs, and north-bound network switches, twinax cables with integrated SFP is an economical alternative as compared to fiber optic SFP.

The most commonly used transceivers include the following:

Cisco SFP-10G-SR: This is a multimode optical fiber 10 Gbps Ethernet SFP that can be used for distances up to 400 meters.
Cisco SFP-10G-LR: This is a single-mode optical fiber 10 Gbps Ethernet SFP that can be used for distances up to 10 Km.
Cisco SFP-10G-FET: This is a low power consuming multimode fiber optic 10 Gbps Ethernet SFP that can be used for distances up to 100 meters.
Cisco SFP-H10GB-CUxM: These are the twinax cables providing low cost 10 Gbps Ethernet connectivity and are available in 1, 3, 5, and 7 meter fixed length configurations. The actual transceivers are named SFP-H10GB-CU1M, SFP-H10GB-CU3M, SFP-H10GB-CU5M, and SFP-H10GB-CU7M according to the length each twinax cable provides.
Cisco SFP-H10GB-ACU10M: This is a 10-meter-long twinax cable providing 10 Gbps Ethernet. At a length of 10 meters, this cable requires active transceivers at both ends.
DS-SFP-FCxG-xW: These are multi-mode and single-mode fiber optic FC transceivers that are available at 2, 4, and 8 Gbps transfer speeds. The actual transceivers are named DS-SFP-FC4G-SW and DS-SFP-FC8G-SW according to the speed and distance covered by the transceiver.

Note

A detailed list of FI-compatible SFPs is available at http://www.cisco.com/en/US/prod/collateral/ps10265/ps11544/data_sheet_c78-675245.html.

Distance and other detailed specifications of the Cisco SFPs are available at http://www.cisco.com/en/US/prod/collateral/modules/ps5455/data_sheet_c78-455693.html.

The Cisco UCS 5100 series blade server chassis

The Cisco 5100 series blade server chassis is a vital building block of the Cisco UCS solution. Currently, there is only one generation of UCS blade chassis, which is Cisco UCS 5108. The chassis form factor is 6 RU and it can host the following:

A maximum of eight half-width blade servers
A maximum of four full-width blade servers
Any other combination of half-width blade and full-width blade servers is also possible

A look at the chassis front

The UCS chassis front is used to insert blade servers into the chassis. The front of the chassis also holds UCS power supplies. The UCS chassis front can hold the following hardware:

Eight half-width empty slots for a maximum of eight half-width blade servers with a removable divider in the middle; this can be removed for installing a maximum of four full-width blades or any other combination of half-width and full-width servers
Four slots for single power supplies; these slots can be configured as nonredundant, N+1 redundant, and grid redundant

This has been demonstrated in the following image:

A look at the chassis back

The UCS chassis back provides slots for IOM modules, fan units, and power connectors. It provides the following connectors:

Two slots for IOM modules that act as remote line cards to the FIs
Eight fan units
Four power supply connectors

This has been demonstrated in the following image:

Environmental requirements

The UCS chassis is designed for the industry-standard rack form factor. The following are the environmental requirements for the UCS chassis:

An industry-standard, four-post, 19 inch rack or cabinet is required as the chassis cannot be mounted onto a two-post relay rack because of its weight and length
It requires a 6 RU rack space
Ensure proper cooling and ventilation for the rack as chassis air flow is front to back and should not be obstructed
The operating temperature of the UCS is 10 to 35 degrees centigrade
The acoustic power level is 85 dBA under normal operation
The weight of an empty chassis is 136 kg, which requires special consideration for floor loading

Note

Due to the weight and size of the chassis, at least two persons are required to mount it to the rails. It is highly recommended to mount the chassis first and then insert the power supplies, fan units, and blade servers.

The use of a server lift is highly recommended. Chassis side handles are only for moving and adjusting the chassis in the rack.

Blade servers and rack-mount servers

While blade servers are the dominant deployment of the UCS solution, Cisco has strategically extended its offering to include the industry-standard rack-mount form factor as well in order to provide users a choice against the competing rack-mount server vendors such as HP, IBM, and Dell.

Learning more about blade servers

Blade servers are at the heart of the UCS solution and come in various system resource configurations in terms of CPU, memory, and hard disk capacity. All blade servers are based on Intel Xeon processors and there is no AMD option available. Small and Medium Businesses (SMBs) can choose from different blade configurations as per business needs.

The B22 M3 blade server

The B22 M3 blade server is a new addition to the blade server series. It is a half-width blade server and has been shown in the following image:

The specifications of the B22 M3 blade server are as follows:

Two Intel® Xeon® E5-2400 series processor family CPUs
A maximum of 192 GB as the total memory capacity with a total of 12 memory slots
Two drive bays for internal SAS/SATA/SSD hard drives for up to 2 TB of maximum storage capacity with built-in RAID 0,1 controllers
Up to 80 Gbps of I/O throughput with a supported mezzanine card

The B22 series entry level blade server provides an excellent price-to-productivity balance. It is an excellent choice for IT, web infrastructure, and scale-out applications.

B200 M1/M2/M3 blade servers

B200 M1/M2/M3 blade servers are currently in the third generation. The B200 series is a half-width blade server. The latest B200 M3 series blade server further extends the capabilities of the B200 M2 servers. There are slight differences in the physical construction of M1 and M2/M3. Have a look at the following image for more details:

The specifications of the latest B200 M3 blade server are as follows:

Two Intel® Xeon® E5-2600 series processor family CPUs (a single processor is no longer supported with B200 M3)
A maximum of 768 GB in total memory capacity (using 32 GB Dual Inline Memory Modules (DIMMs)) with a total of 24 memory slots
Two drive bays for internal SAS/SATA/SSD hard drives for up to 2 TB of maximum storage capacity with built-in RAID 0,1 controllers
Up to 80 Gbps of I/O throughput with a supported mezzanine card

B200 series entry-level blade servers are an excellent choice for web server farms, distributed databases, and CRM applications.

B230 M1/M2 blade servers

B230 M1/M2 series servers differ from B200 series servers in terms of computational capabilities. This is also a half-width blade server with two CPUs; however, the B230 server is based on a slightly different CPU family. The B230 series memory slot density is higher as compared to that of the B200 series blade servers. Have a look at the following image for more details:

The specifications of the B230 M2 blade server are as follows:

Two Intel® Xeon® processors of the E7-2800/8800 product family CPUs
A maximum of 512 GB as the total memory capacity with a total of 32 memory slots
Two SSD drive bays with a maximum of 600 GB of storage and built-in RAID 0,1 controllers
Up to 20 Gbps of I/O throughput with a supported mezzanine card

B230 series servers are an excellent choice for virtualized loads and databases in compact form factor.

The B420 M3 blade server

B420 M3 series blade servers are full-width blade servers with the highest memory density ideal for demanding enterprise application deployment.

The specifications of the B420 M3 blade server are as follows:

Four Intel® Xeon® processors of the E4600 product family CPUs
A maximum of 1.5 TB as the total memory capacity with a total of 48 memory slots
Four drive bays for internal SAS/SATA/SSD hard drives for a maximum storage capacity of up to 4 TB with built-in RAID 0,1,5,10 controllers
Up to 160 Gbps of I/O throughput with supported mezzanine cards

B440 M1/M2 blade servers

B440 M1/M2 series blade servers are full-width blade servers with four CPUs and high memory density, ideal for enterprise applications deployment.

The specifications of the B440 M2 blade server are as follows:

Four Intel® Xeon® processors of the E7-4800/8800 product family CPUs
A maximum of 1 TB as the total memory capacity with a total of 32 memory slots
Four drive bays for internal SAS/SATA/SSD hard drives for up to 3.6 TB as the maximum storage capacity with built-in RAID 0,1, 5, 10 controllers
Up to 40 Gbps of I/O throughput with a supported mezzanine card

Note

For a quick comparison of B-series blade servers, please visit http://www.cisco.com/en/US/products/ps10280/prod_models_comparison.html. Select all the servers and click on the Compare button.

Cisco also had UCS 250 M1/M2 blade servers which were discontinued in November 2012. You may still see these servers deployed in the field.

Learning more about rack-mount servers

A majority of server vendors manufacture rack-mount servers (also known as pizza-box servers). Cisco UCS C-series servers are available in the industry-standard rack-mount form which can be deployed for SMB remote offices and applications requiring dedicated equipment. Like B-series blade servers, these servers are also based on Intel Xeon processors and there is no AMD option available.

C-series servers can be managed independently through an out of band (OOB) web interface or can be integrated with the UCSM software. The embedded CIMC provides the following services:

Remote KVM to server console
Remote power management
Remote virtual media for operating system installation
Industry-standard IPMI support for monitoring
Standard SNMP traps for monitoring

Connecting and managing C-series rack servers through UCSM requires a connection through Nexus 2200 series FEXs, which act as line cards to the FIs.

C-series servers are available in various CPUs, memory, I/O, and storage configurations to address the needs of differently sized organizations. The following sections cover the specifications of these servers as available on the Cisco website.

The C22 M3 rack-mount server

The C22 M3 rack-mount server is an entry-level rack-mount server. It is based on the Intel E2400 CPU product family.

The specifications of the C22 M3 server are as follows:

Two Intel® Xeon® processors of the E5-2400 product family of CPUs
A maximum total memory capacity of 192 GB with a total of 12 memory slots
Up to eight small form factor (SFF) or four large form factor (LFF) internal storage SAS/SATA/SSD hard disks with an 8 TB (SFF) or 12 TB (LFF) storage capacity and an optional RAID controller
Two 1 Gbps I/O ports with optional 10 Gbps unified fabric ports
Two PCIe third-generation expansion slots

The C22 series rack-mount server is an economical choice for SMBs, branch offices, departments for entry level virtualization, and web farms.

The C24 M3 rack-mount server

The C24 M3 rack-mount server is also an entry level rack-mount server with storage extensibility. It is based on the Intel E2400 CPU product family.

The specifications of the C24 M3 server are as follows:

Two Intel® Xeon® processors of the E5-2400 product family of CPUs
A maximum total memory capacity of 192 GB with a total of 12 memory slots
Up to 24 SFF or 12 LFF internal storage SAS/SATA hard disks with a 24 TB (SFF) or 26 TB (LFF) storage capacity and an optional RAID controller
Two 1 Gbps I/O ports with optional 10 Gbps unified fabric ports
Five PCIe third-generation expansion slots

The C24 M3 series rack-mount server is an economical choice for SMBs, branch offices, departments for entry level virtualization, and web farms.

The C220 M3 rack-mount server

The C220 M3 rack-mount server is based on the Intel E2600 CPU product family.

The specifications of the C220 M3 rack-mount server are as follows:

Two Intel® Xeon® processors of the E5-2600 product family of CPUs
A maximum total memory capacity of 512 GB with a total of 16 memory slots
Up to eight SFF or four LFF internal storage SAS/SATA/SSD hard disks with an 8 TB (SFF) or 12 TB (LFF) storage capacity and an optional RAID controller
Two 1 Gbps I/O ports with optional 10 Gbps unified fabric ports
Two PCIe third-generation expansion slots

The C220 M3 series server has great performance and density for distributed database applications and web farms.

The C240 M3 rack-mount server

The C240 M3 rack-mount server is based on the Intel E2600 CPU product family.

The specifications of the C240 M3 rack-mount server are as follows:

Two Intel® Xeon® processors of the E5-2600 product family of CPUs
A maximum total memory capacity of 768 GB with a total of 24 memory slots
Up to 24 SFF or 12 LFF internal storage SAS/SATA/SSD hard disks with a 24 TB (SFF) or 26 TB (LFF) storage capacity and an optional RAID controller
Four 1 Gbps I/O ports with optional 10 Gbps unified fabric ports
Five PCIe third-generation expansion slots

The C240 M3 series server has great performance along with storage extensibility for distributed database applications and web farms.

The C260 M2 rack-mount server

The C260 M2 rack-mount server is based on the Intel E2800 CPU product family.

The specifications of the C260 M2 rack-mount servers are as follows:

Two Intel® Xeon® processors of the E7-2800 product family of CPUs
A maximum total memory capacity of 1 TB with a total of 64 memory slots
Up to 16 LFF internal storage SAS/SATA/SSD hard disks with a 16 TB (LFF) storage capacity and no RAID controller
Two Gigabit Ethernet (GE) LAN on Motherboard (LoM) I/O ports with two optional 10 Gbps unified fabric ports
Six PCIe third-generation expansion slots

The C260 M2 series server provides one of the highest densities of computational resources for business-critical applications.

The C420 M3 rack-mount server

The C420 M3 rack-mount server is based on the Intel E4600 CPU product family.

The specifications of the C420 M3 rack-mount server are as follows:

Four Intel® Xeon® processors of the E5-4600 product family of CPUs
A maximum total memory capacity of 1.5 TB with a total of 48 memory slots
Up to 16 LFF internal storage SAS/SATA/SSD hard disks with a 16 TB (LFF) storage capacity and an optional RAID controller
Two 1 Gbps I/O ports with two optional 10 Gbps unified fabric ports
Four PCIe third-generation expansion slots

The C420 M3 series server provides computational resources for business-critical applications such as large databases both in bare metal and virtualized environments.

The C460 M2 rack-mount server

The C460 M2 rack-mount server is based on the Intel E4800 CPU product family.

The specifications of the C460 M2 rack-mount servers are as follows:

Four Intel® Xeon® processors of the E7-4800 product family of CPUs
A maximum total memory capacity of 2 TB with a total of 64 memory slots
Up to 12 LFF internal storage SAS/SATA/SSD hard disks with a 12 TB (LFF) storage capacity and no RAID controller
Two GE LoM I/O ports with two optional 10 Gbps unified fabric ports
10 PCIe third-generation expansion slots with an optional LSI MegaRAID controller in the eleventh slot

The C460 M2 series server provides exceptional computational resources for business-critical applications such as large databases both in bare metal and virtualized environments.

Note

For a quick comparison of C-series servers, please visit http://www.cisco.com/en/US/products/ps10493/prod_models_comparison.html. Select all the servers and click on the Compare button to get the results.

Getting started with mezzanine adapters

A huge variety of mezzanine adapters, also known as Virtual Interface Cards (VICs), is available from Cisco for both B-series blade servers and C-series rack servers. Older adapters are of the fixed port type and are not optimized for contemporary virtualized server environments. There are some older third-party network cards also available as an option. Newer adapters are optimized for virtualization and can provide 128 or 256 dynamic virtual adapters. The number of virtual adapters is dependent on the VIC model. These virtual adapters can be configured as Ethernet (vNIC) or fiber channel (vHBA) devices. All virtualization-optimized VICs also support the VM-FEX technology. Our focus will be on those mezzanine adapters that are virtualization optimized.

VICs for blade servers

VICs are available in the form of a mezzanine card. All new VICs provide dynamic vNIC or vHBA interfaces for server-side connectivity.

VIC 1280

The specifications of VIC 1280 are as follows:

256 dynamic vNIC (Ethernet) or vHBA (FC) interfaces
VM-FEX support for virtualized environments
Hardware failover without driver need
80 Gbps network throughput
Mezzanine form factor
Compatible with UCS M2 (B230 and B440) and all M3 blade servers

VIC 1240

The specifications of VIC 1240 are as follows:

256 dynamic vNIC (Ethernet) or vHBA (FC) interfaces
VM-FEX support for virtualized environments
Hardware failover without driver need
40 Gbps network throughput with optional 80 GB throughput using optional port expander in the mezzanine slot
LoM form factor
Compatible with all M3 blade servers

VIC M81KR

The specifications of VIC M81KR are as follows:

128 dynamic vNIC (Ethernet) or vHBA (FC) interfaces
VM-FEX support for virtualized environments
Hardware failover without driver need
20 Gbps network throughput
Compatible with UCS M2 blade servers

VICs for rack-mount servers

VICs are available as PCIe cards as well. All new VICs provide dynamic vNIC or vHBA interfaces for server-side connectivity.

VIC 1225

The specifications of VIC 1225 are as follows:

256 dynamic vNIC (Ethernet) or vHBA (FC) interfaces
VM-FEX support for virtualized environments
Hardware failover without driver need
20 Gbps network throughput
Compatible with UCS M2 (C460 and C260) and all M3 rack-mount servers

VIC P81E

The specifications of VIC P81E are as follows:

128 dynamic vNIC (Ethernet) or vHBA (FC) interfaces
VM-FEX support for virtualized environments
Hardware failover without driver need
20 Gbps network throughput
Compatible with UCS M2 (C460 and C260) and all M3 rack-mount servers

Note

For a quick comparison of mezzanine card specifications, please visit http://www.cisco.com/en/US/products/ps10277/prod_models_comparison.html#~tab-a.

Cisco VICs are also famous by their code name, Palo.

Installing UCS chassis components

Now that we have a better understanding of the various components of the Cisco UCS platform, we can delve into the physical installation of the UCS chassis, that is, the installation of blade servers, IOM modules, fan units, power supply units, SFP+ modules, and physical cabling.

Care must be taken during the installation of all components as failure to follow the installation procedure may result in component malfunction and bodily injury.

Tip

UCS chassis don'ts

Do not try to lift even an empty chassis alone. At least two persons are required to handle the UCS chassis.

Do not handle internal components such as CPU, RAM, and mezzanine cards without the Electro Static Discharge (ESD) field kit.

Before the physical installation of the UCS solution, it is also imperative to consider other datacenter design factors including the following:

Building floor load bearing capacity
Rack requirements for UCS chassis and FIs
Rack airflow, heating, and ventilation (HVAC)

Physical installation is divided into the following three sections:

Blade server component (CPU, memory, hard drives, and mezzanine cards) installation
Chassis component (blade servers, IOMs, fan units, and power supply units) installation
FI installation and physical cabling

Blade server installation

Cisco UCS blade servers are designed on industry-standard components with some enhancements. Anyone with prior server installation experience should be comfortable installing internal components using the guidelines provided in the blade server manual and following the standard safety procedures. ESD transient charges may result in thousands of volts of charge building up, which can degrade or permanently damage electronic components.

Note

The Cisco ESD training course may be referred to at http://www.cisco.com/web/learning/le31/esd/WelcomeP.html.

All Cisco UCS blade servers have similar cover design with a button at the top front of the blade; this button needs to be pushed down. Then, there is a slight variation among models in the way that the cover slides; this could be either towards the rear and upwards or towards self and upwards.

Installation and removal of CPU

The following is the procedure to mount a CPU onto a UCS B-series blade server:

Make sure you are wearing an ESD wrist wrap grounded to the blade server cover.
To release the CPU clasp, first push it down and then to the side away from the CPU socket.
Move the lever up and remove the CPU blank cover. Keep the blank cover in a safe place just in case you need to remove a CPU.
Pick up the CPU with the plastic edges and align it with the socket. The CPU can only fit one way.
Lower the mounting bracket with the side lever and secure the CPU into the socket.
Align the heat sink with its fins in a position allowing unobstructed airflow from front to back.
Gently tighten the heat sink screws on to the motherboard.

CPU removal is the reverse of the installation process. It is critical to place the socket blank cover back over the CPU socket. Damage could occur to the socket without the blank cover.

Note

For UCS B440, air blocker must be installed if you permanently remove a CPU.

Installation and removal of RAM

The following is the procedure to install RAM modules into the UCS B-series blade server:

Make sure you are wearing an ESD wrist wrap grounded to the blade server cover.
Move away the clips on the side of the memory slot.
Hold the memory module with both edges in an upright position and firmly push straight down, matching the notch of the module to the socket.
Close the side clips to hold the memory module.

Memory removal is the reverse of the installation process.

Note

The memory modules must be inserted in pairs and split equally between each CPU if all the memory slots are not populated. Refer to the server manual for identifying memory slot pairs and slot-CPU relationship.

Installation and removal of internal hard disks

UCS supports SFF serial attached Small Computer System Interface (SCSI) (SAS) hard drives. Blade servers B200, B240, and B440 support regular thickness (15 mm) hard drives whereas B230 supports thin (7 mm) hard drives.

To insert a hard disk into the B200, B250, and B440 blade servers, carry out the following steps:

Make sure you are wearing an ESD wrist wrap grounded to the blade server cover.
Remove the blank cover.
Press the button on the catch lever on the ejector arm.
Slide the hard disk completely into the slot.
Push the ejector lever until it clicks to lock the hard disk.

To remove a hard disk press the release button, pull the catch lever outward, and slide the hard disk out.

To insert or release a thin hard drive into or from the B230 server, release the catch by pushing it inside while inserting or removing the hard disk.

Note

Do not leave a hard disk slot empty. If you do not intend to replace the hard disk, cover it with a blank plate to ensure proper airflow.

Installation of mezzanine cards

UCS B200 and B230 support single mezzanine cards whereas B250 and B440 support two cards. The procedure for installing these cards is the same for all servers, which is as follows:

Make sure you are wearing an ESD wrist wrap grounded to the blade server cover.
Open the server top cover.
Grab the card with its edges and align the male molex connector, the female connector, and the motherboard.
Press the card gently into the slot.
Once the card is properly seated, secure it by tightening the screw on top.

Mezzanine card removal is the reverse of the installation process.

Installation of blade servers on the chassis

Installation and removal of half-width and full-width blade servers is almost identical with the only difference being the use of one ejector arm for half-width blade servers whereas for full-width blade servers, there are two ejector arms. Carry out the following steps:

Make sure you are wearing an ESD wrist wrap grounded to the chassis.
Open one ejector arm for the half-width blade servers or both ejector arms for full-width blade servers.
Push the blade into the slot. Once firmly in, close the ejector arm on the face of the server and tighten the screw with your hands.

The removal of a blade server is the opposite of the installation process.

Note

In order to install a full-width blade, it is necessary to remove the central divider. This can be done with a Philips screwdriver to push two clips, one in the downward and the other in the upward direction, and sliding the divider out of the chassis.

Cabling FI and IOM

UCS is an integrated solution that handles both network traffic and management control. All management and data movement intelligence for chassis components and blade servers is present in the FIs and IOM modules (which are line cards for the FIs). Therefore, proper cabling between FIs and IOM modules is an important design consideration.

IOM – FI cabling topology

IOM modules are used to connect blade server chassis to the FIs and act as line cards to them. It is therefore necessary to maintain proper connectivity between IOMs and FIs. Since an IOM module becomes part of the FI, multiple links from a single IOM can only be connected to a single FI and not across to the other FI. Depending on the IOM model, there can be one, two, four, or eight links from IOM to a single FI. These links can be configured in the port channel for bandwidth aggregation. The chassis discovery process is initiated as soon as an IOM is connected to an FI.

In the following figure, on the left-hand side, all links from IOM 0 are connected to a single FI and can be combined into a single port channel. The figure on the right shows a configuration in which links from a single IOM are connected to different FIs. This is an invalid topology, and hence chassis discovery will fail.

Note

Chassis discovery will also fail if a high availability cluster is not established between FIs.

IOM – FI physical cabling

IOMs provide connectivity to the individual blade servers through I/O multiplexer and connectivity to the FI. IOM interface connectivity to blade servers does not require user configuration.

IOM to FI connectivity, however, requires physical cabling. Both IOM and FI have SFP+ slots. There are a variety of possibilities in terms of physical interfaces. Some of the common configurations include the following:

10 GB FET SFP+ interface (special optical multimode fiber SFP+ module which can only be used with UCS and Nexus equipment)
10 GB CU SFP+ (copper twinax cable)
10 GB SR SFP+ (short range multimode optical fiber SFP+ module for up to 300 m)
10 GB LR SFP+ (long-range, single-mode optical fiber SFP+ module for above 300 m)

The following figure shows eight connections from IOM 0 to Fabric Interconnect A and eight connections from IOM 1 to Fabric Interconnect B. Depending on the bandwidth requirements and model, it is possible to have only one, two, four, or eight connections from IOM to FI.

Although large numbers of links provide higher bandwidth for individual servers, as each link consumes a physical port on the FI, they also decrease the total number of UCS chassis which can be connected to the FIs.

As shown in the preceding figure, IOM to FI only supports direct connection. However, FI to north-bound Nexus switch connectivity can be direct and may use regular port channel (PC), or the connections from a single FI may traverse two different Nexus switches and use virtual PortChannel (vPC).

The following figure shows a direct connection between FIs and Nexus switches. All connections from FI A are connected to the Nexus Switch 1 and all connections from FI B are connected to Nexus Switch 2. These links can be aggregated into a PC.

The following are two other connections that need to be configured:

Cluster heart beat connectivity: Each FI has two fast Ethernet ports. These ports should be connected using a CAT6 UTP cable for cluster configuration
Management port: This is also a fast Ethernet port that can be configured using a CAT6 UTP cable for remote management of the FI

The following figure shows the FI to Nexus switch connectivity where links traverse Nexus switches. One network connection from FI A is connected to Nexus Switch 1 and the other to Nexus Switch 2. Both these connections are configured via vPC. Similarly, one connection from FI B is connected to Nexus Switch 2 and the other to Nexus Switch 1. Both these connections are also configured via vPC. It is also imperative to have vPC on a physical connection between both the Nexus switches. This is shown as two physical links between Nexus Switch 1 and Nexus Switch 2. Without this connectivity and configuration between Nexus Switches, vPC will not work.

Physical slots in Nexus switches also support the same set of SFP+ modules for connectivity that FIs and IOMs do.

Note

A complete list of SFP+ modules is available in Table 3 at http://www.cisco.com/en/US/prod/collateral/ps10265/ps10276/data_sheet_c78-524724.html.

Filter reviews by

All

Amazon verified reviews

Mohammed Abdul Sep 13, 2015

This book is really help full & all the topics are in very simple language to understand.

Amazon Verified review

Kindle Customer Mar 15, 2014

If you read this book together with the Cisco Unified Computing System (UCS) by Cisco Press, you will end up with an excellent understanding of UCS. This book covers a lot of material that the other older book does not cover.

Team SS Aug 19, 2014

Really good for the beginning user. Maybe somewhat redundant for an experienced pro, but has good screen captures and walks through many of the basic processes. Good reference guide to remind yourself of what to do, even if you have done it before. I learned a lot and got stronger on my fundamentals.

Khamilia Khan May 22, 2016

Wonderful book

Owen Campbell May 06, 2016

Very good information

Implementing Cisco UCS Solutions: Cisco Unified Computer System is a powerful solution for data centers that can raise efficiency and lower costs. This tutorial helps professionals realize its full potential through a practical, hands-on approach written by two Cisco experts.

What do you get with eBook?

Stateless computing

Rapid provisioning of servers

Simplified troubleshooting

Virtualization readiness

Choice of industry-standard form factors

Extended memory technology for increased density

The B22 M3 blade server

B200 M1/M2/M3 blade servers

B230 M1/M2 blade servers

The B420 M3 blade server

B440 M1/M2 blade servers

The C22 M3 rack-mount server

The C24 M3 rack-mount server

The C220 M3 rack-mount server

The C240 M3 rack-mount server

The C260 M2 rack-mount server

The C420 M3 rack-mount server

The C460 M2 rack-mount server

VIC 1280

VIC 1240

VIC M81KR

VIC 1225

VIC P81E

Installation and removal of CPU

Installation and removal of RAM

Installation and removal of internal hard disks

Installation of mezzanine cards

Installation of blade servers on the chassis

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with eBook?

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs