QuantaMesh BMS T7128-IX5 is a new type of chassis switch that supports 4 line cards with up to 128 QSFP28 ports and 4 fabric cards in a 6U height mechanical enclosure for hyper scale data center networking deployments. Not like the traditional chassis switches with a central controller of all line cards and fabric cards, the T7128-IX5 is a unique distributed design consisting of a set of x86 CPU controls for each switching ASIC. Through this distributed design, the T7128-IX5 creates ease for software porting to support high density port counts. This kind of design breaks out the proprietary chassis design and can be a more open hardware platform for the switch industry.
The architecture of the T7128-IX5 includes a data plane, control plane, management plane. and power plane.
Data Plane
- Line Card
Each Line card can support 32 QSFP28 ports for 10/25/40/50/100Gbps. Each line card has two sets of x86 CPU – Intel XEON D-1527 and switching ASIC – Broadcom Tomahawk 3.2T. Each switching ASIC is designed to have half of the bandwidth face to the front panel and half of the bandwidth to the back connecting to the fabric card to support its non-blocking hardware design.
- Fabric Card
There are two types of fabric cards – active and passive. For the active fabric card design, one set of x86 CPU – Intel XEON D-1527 and one set of switching ASIC – Broadcom Tomahawk 3.2T are supported from each fabric card. All the switching ASIC bandwidth is designed to connect with all the line cards using orthogonal connectors. The passive fabric card is designed as a backplane for all the line cards in a back-to-back connection. The two types of fabric cards provide flexibility for hyper scale data center networking deployments that need two levels of spine/leaf network.
Control Plane
QuantaMesh T7128-IX5 adopts the Intel XEON D-1527 CPU as the controller for each switching ASIC. Each switching unit is an independent switching element to perform a single runtime image. Between each switching unit, the standard protocols such as OSPF, BGP, ECMP can be run to communicate with each other.
Management Plane
The T7128-IX5 is designed to have two central monitor cards – a redundancy design with master and slave roles for the whole system hardware control. Besides the powerful x86 CPU from each switching unit, the T7128-IX5 adds a baseband management controller (BMC) – Aspeed AST2520 for each switching unit to assist hardware management. The BMC can be responded for local thermal sensor report, power report and I2C report to the central controller and monitor card. When the central monitor card gets the report from the BMC distributed from each switching unit, it can control the fan speed for system cooling, power management, and system health monitoring. The monitor card also provides a communication channel with the runtime from each switching unit via open API.
Power Plane
The T7128-IX5 provides a 2+2 PSU design. Each PSU supports 2200 watts power.
Figure 1: T7128-IX5 front view and rear view with FRU numbering
People may doubt the distributed switch unit design from the T7128-IX5, and believe it causes complexities for management and troubleshooting. However, the T7128-IX5 configures the dedicated console port and shared management port with an SOL feature to release management concerns.
- Dedicated console
QuantaMesh T7128-IX5 allocates the console port for each switching unit for direct console connections. There are two console port designs in each line card for the two switch units. The console port of the fabric card resides in the monitor card one-to-one mapping to the 4 fabric cards in the rear panel.
- Serial OVER LAN
With the assistance of BMC built-in from each switching unit, BMC assist does the Serial over LAN (SOL) via IPMI protocol. This design redirects the console data from each switch unit to remote central management and monitor station which helps for any network accident.
- Out-of-band management
The out-of-band management port for T7128-IX5 resides in the monitor card. The management port supports 10/100/1000Gbps in a RJ-45 type standard. The management path of each switching unit is shared via this port as well. The T7128-IX5 supports ONIE from each switching unit. The ONIE installation can be through this management path for installation.
Figure 2: T7128-IX5 front panel with dedicated console port and shared OOB port
The T7128-IX5 can provide optimized cooling for each QSFP28 port at 550C (ambient temperature 450C) with a 7+1 hot-swappable fan tray design. Considering the use of an open hardware platform for ease of software porting, the thermal management jobs are all managed via the design of each local BMC and central monitor card.
For the hyper scale data center, a chassis switch is necessary when the scalability is more than 2K 100G nodes. A traditional chassis switch is a good choice but is a proprietary design which is not able to support the expectation of a self-controlled concept by hyper scale data center customers. Therefore, the T7128-IX5 can support up to 8192 100G connections by a CLOS topology deployment and is an open hardware platform for easy NOS porting and hardware and software disaggregation.
We do believe the QuantaMesh BMS T7128-IX5 can bring the changes to the data center networking industry and provide the best solution not only for the data center developer but to benefit the end users in the cloud networking world.