Unleashing the Performance of Multi-Actuator Drives

This paper looks back at the analysis that has been done for the drive wear-out issue on for the different E-series array systems running at different customers’ sites and uses that data to give more specific guidance on thresholds for a preemptive drive removal. Motivation of DHM: Customers with old ventage or Refurbished drive replacement may experience a data loss event and continues to see a high drive failure rate >5% AFR (Annual Failure Rate). Storage vendors expect to see high fallout rates across the hard drive population as they age. The high utilization and the age of the drives are likely to continue and possibly increase the drive failure rate.

Failures are as such:

• Outages: Loss of access
• Performance impact
• Possible data loss

Mahmoud Jibbe

Technical Director

NetApp

Charles Binford

Software Architect

NetApp

For the last eight years Backblaze has collected daily operational data from the hard drives in our data centers. This includes daily SMART statistics from over 250,000 hard drives, and SSDs, totaling nearly two exabytes of storage, and totaling over 200 million data points. We'll use this data to examine the following: - the lifetime failure statistics for all the hard drives we have ever used. - how has temperature effects the failure rate of hard drives. - a comparison the failure rates of helium filled hard drives versus air-filled hard drives. - the SMART stats we use to indicate whether a Hard Drive may fail and if you can use SMART stats to predict hard drive failure. - how SSD failure rates compare to HDD failure rates. The data we use is available for anyone to download and use for their own experimentation and education.

Andrew Klein

Staff Software Engineer

Intel

Delivering high throughput and/or high IO rates while minimizing command execution latency is a common problem to many storage applications. Achieving low command latency to implement a responsive system is often not compatible with high performance. The ATA NCQ IO priority feature set, supported today by many SATA hard-disks, provides a coarse solution to this problem. Commands that require a low latency can be assigned a high priority, while background disk accesses are assigned a normal priority level. This binary hinting for the latency requirements of read and write commands allows a device firmware to execute high priority commands first, therefore achieving the desired lower latency. While NCQ IO priority can satisfy many applications, the standards do not clearly define how high-priority commands should be executed. This often results in significant differences between different device models from different device vendors. The Command Duration Limits (CDL) feature introduces a more detailed definition of command priorities and of their execution by the device. A command duration limit descriptor defines an inactive time limit (command queuing time), active time limit (command execution time) and a policy to apply to the command if one of the limit is exceeded (e.g. abort command, execute command, etc). Up to seven command duration limits can be defined and controlled by the user for read and write commands, allowing a very fine control over disk accesses by an application. Furthermore, unlike the ATA NCQ IO priority feature which has no equivalent in SCSI, Command Duration Limits is defined for both ATA and SCSI disks. This talk will present Linux implementation of CDL support. The kernel interface for discovering and configuring the disk command duration limits is described. Examples will also illustrate how a user application can specify different limits for different IO operations using an interface based off the legacy IO priority support. The effect of Command Duration Limits will be illustrated using results from various micro-benchmarks.

Damien Le Moal

Director

Western Digital

Data is growing at an exponential pace and the need to manage this data at the core and edge is a multi-facet problem. New innovative methods to ensure data availability & utilization of the resources that store this data are being developed. Storage device health monitoring & utilization is one such issue. Developing models to predict drive degradation while using machine learning principles is highly desirable. A recent Google blog described the machine learning techniques being used to promote & improve drive maintenance of their fleet. Enabling this capability requires the ability to efficiently pull drive information. Seagate has developed a method that can allow data to be extracted quickly and reliably while combining many different data sets into a single command operation. Customers including Google and Tencent implement such drive health monitoring into their eco-system. This presentation will review what is openly available from these high capacity devices and how they can be used to create novel ML models to predict device behavior and make future utilization decisions.

Paul Burnett

Principal Engineer

Seagate Technologies

SATA multi-actuator drives appear as a single drive to the system stack, yet all IO is addressed to LBA ranges on each actuator. Linux IO schedulers must be aware of this geometry via log pages in ATA Command Set –5 and then optimize behavior to ensure that actuators are kept busy. Legacy IO scheduler behavior can lead to underutilized actuators, delayed IO completion resulting in poor performance. Paolo Valente Assistant Professor of Computer Science at the University of Modena and Reggio Emilia, Italy in consultation with Seagate Technology has developed a version of the BFQ IO scheduler that is optimized for multi-actuator drives demonstrating dramatic performance improvements over a wide range of workloads. The concepts and strategies used by BFQ can inform changes to IO schedulers, the Linux kernel, and storage applications to take advantages of the IOPS gains of multi-actuator drives. Dual LUN SAS dual actuator drives are in market today with SATA and Single-LUN SAS drives coming as the need for maintaining IOPS becomes increasingly important as drive capacities grow.

Tim Walker

Principal Engineer

Seagate Technology

Paolo Valente

Assistant Professor

University of Modena and Reggio Emilia

Applications desire for higher IOPS & bandwidth is driving innovations at the hardware level. This requires software stack awareness to harness the benefits. Multi-actuator drives represent the next big innovation in performance.

Dual actuator drives are being deployed today in large numbers for specific storage applications with Seagate MACH.2 SAS drives. This meets a growing need for large disk sizes that can meet both TCO and IOPS goals . As aerial densities and drive sizes increase, multi-actuator drives are migrating from early adopter deployments to the mainstream. This offers the promise of deploying larger disk sizes while continuing to meet performance goals, but only if there is software stack awareness that a single drive whether SATA, SAS or NVMe has two or more actuators that can be treated as logically independent disks for optimal performance. In this talk storage experts will explain what has been learned from early adoption, how this can be deployed for scenarios including CDN, Object Store, Big Data, AI and Machine Learning and what types of IO perform best in this new storage hardware paradigm. Coming support for SATA presents new challenges and deployment and software options including use of optimized Linux IO schedulers such as BFQ. Are you ready for the future of hard disk storage?

Arie van der Hoeven

Principal Product Manager

Seagate Technology

Tim Walker

Principal Engineer

Seagate Technology