QCT made a splash this year as our Distributed Asynchronous Object Storage (DAOS) solution ranked in the IO500 for the first time. The integration of Intel DAOS with our QCT systems brought world-class IOPS and bandwidth for QCT as we ranked 16th on the IO500 list and placed 12th in the 10-Node Challenge released during SC21, which are HPC storage benchmarks with growing popularity. Our Taiwania 2 and 3 supercomputers also remain on the newly updated Top500 and Green500 lists revealed at SC21.

DAOS is an open-source project developed by Intel that provides high bandwidth, low latency, and high IOPS storage to HPC applications, designed from the ground-up to support Storage Class Memory and NVMe storage. Team QCT has worked with Intel’s engineers to integrate DAOS into our QuantaPlex D53X-1U and QuantaGrid D53XQ-2U, our latest server additions that feature up to 12 and 24 NVMe SSDs, and run on 3rd Gen Intel Xeon Scalable Processors to deliver optimized performance for time-sensitive contemporary HPC/AI workloads.

Aiming to fully utilize the strength of high-speed storage devices and interfaces, DAOS delivers more IOPS, bandwidth, and better scalability than its all-flash counterpart Lustre file system. Lower latency with higher memory and PCIe bandwidth enable DAOS to deliver a combination of peak IOPS for metadata and small files and unparalleled throughput for large files. Furthermore, DAOS performance improves significantly when running on QCT servers with third-gen Intel Xeon processors and Intel Optane technology. (see results below)

Fig. 1. DAOS performances 3-5 times better in sequential/random read/write compared to Lustre

Fig. 2. DAOS performs 10-100 times better in small file operations compared to Lustre

Fig. 3. Linear scalability in DAOS read performance; DAOS performs 3-5 times better in bandwidth compared to Lustre 

In addition to the benefits of high I/O throughput and minimal latency, DAOS also supports the traditional POSIX interface, simplifying transitions from legacy code and operation scripts. With native support of the HDF5 interface, a popular I/O library in weather models, via the HDF5 plugin, DAOS can perform at its full potential. Many of our benchmarks and benefits of this breakthrough technology can be seen in our video here.