Software raid throughput

This allows us to talk in terms of relative performance as a factor of the drive performanc. This is important as IOPS are often very hard to define. But we can compare performance in a meaningful way by speaking to it in relationship to the individual drives within the array. Artifacts such as memory caches and solid state caches will do amazing things to alter the overall performance of a storage subsystem. But they will not fundamentally change the performance of the array itself under the hood.

There is no simple formula for determining how different cache options will impact the overall performance. Suffice it to say that it can be very dramatic depending heavily not only on the cache choices themselves, but also heavily on workload. Even the biggest, fastest, most robust cache options cannot change the long term, sustained performance of an array. RAID is complex and many factors influence the final performance. One is the implementation of the system itself.

A poor implementation might cause latency. Or it may fail to use the available spindles such as having a RAID 1 array read only from a single disk instead of from both simultaneously! There is no easy way to account for deficiencies in specific implementations.

We must assume that all are working to the limits of the specification. Any enterprise RAID system will do this. It is primarily hobby and consumer RAID systems that fail in this aspect. Some types of RAID also have dramatic amounts of computational overhead associated with them while others do not. Primarily, parity RAID levels require heavy processing in order to handle write operations, with different levels having different amounts of computation necessary for each operation. This introduces latency, but does not curtail throughput.

This latency will vary, however, based on the implementation of the RAID level as well as on the processing capability of the system. Often, the server CPU is actually faster here but consumes system resources. ASICs can be very fast but are expensive to produce. This latency impacts storage performance but is very difficult to predict and can vary from nominal to dramatic. So I will mention the relative latency impact with each RAID level but will not attempt to measure it.

In most RAID performance calculations, this latency is ignored. However, it is still present. Depending on the configuration of the array, it could have a noticeable impact on a workload. There is, it should be mentioned, a tiny performance impact to read operations due to efficiencies in the layout of data on the disk itself.

Parity RAID requires there to be data on the disks that is useless during a healthy read operation but cannot be used to speed it up. This actually results in it being slightly slower. But this impact is ridiculously small and is normally not measured and so can be ignored. Factors such as stripe size also impact performance, of course. But as that is configurable and not an intrinsic artifact in any level, I will ignore it here.

It is not a factor when choosing a RAID level itself but only in configuring one once chosen. The final factor that I want to mention is the read to write ratio of storage operations. Some RAID arrays will be used almost purely for read operations, some almost solely for write operations.

Most will use a blend of the two, likely something around eighty percent read and twenty percent write. This ratio is very important in understanding the performance that you will get from your specific array and understanding how each RAID level will impact you. We measure storage performance primarily in IOPS. Many people talk about storage performance with a single IOPS number.

We can use these two together can to find any IOPS blend that we need. Now that we have established some criteria and background understanding, we will delve into our RAID levels themselves and see how performance varies across them. This does not address the nominal overhead numbers that I mention above, of course.

But the real world number is so close that it is very practical to simply use this formula. Keep in mind that drives often have different read and write performance. RAID 0 is the easiest level to understand because there is effectively no overhead to worry about, no resources consumed to power it and both read and write get the full benefit of every spindle, all of the time. RAID 0 is always the highest performing level.

An example would be an eight spindle RAID 0 array. Both read and write IOPS are the same here. RAID 10 has the second simplest level to calculate. Because RAID 10 is a RAID 0 stripe of mirror sets, we have no overhead to worry about from the stripe but each mirror has to write the same data twice in order to create the mirroring. This cuts our write performance in half compared to a RAID 0 array of the same number of drives.

We should note that at the same capacity, rather than the same number of spindles, RAID 10 has the same write performance as RAID 0 but double the read performance — simply because it requires twice as many spindles to match the same capacity. Uncommon options such as triple mirroring in RAID 10 would alter this write penalty.

The biggest difference would be the RAID configuration. Microsoft SQL Server fortunately permits seamless distribution of its data tables among any number of volumes you can throw at it. The use of row-level extent-level striping in MS SQL Server allows even distribution of data and workload across any number of volumes, and I used that simple mechanism to distribute the database over the 8 pairs of RAID1. An extent is a 64KB block of data which is the smallest unit of data.

This in turn resulted in a massive reduction in SQL transactions from a painfully slow ms per transaction to ms per transaction. The result was so dramatic that it was hard for everyone to believe it during the first week.

They had thought that perhaps there was some mistake or anomaly and that this might be due to a drop in usage, but doubts subsided as time went on and the performance kept up. Even the engineer from the storage vendor who initially doubted the effectiveness of this solution became a believer after he ran some tests to verify that the load evenly distributed across the 8 RAID1 pairs.

A big part of the problem was that Oracle lacked the ability to seamlessly split a table evenly over multiple volumes.

It was still possible to divide up the location of the hundreds of tables that made up a typical database, but it required manual load measurements and manual distribution, which is something that many DBAs database administrators refused to do.

It also required annual maintenance and a redistribution of tables because workloads change over time. Without extent-level striping, it becomes a question of whether the DBA and IT department want to deal with the additional management overhead.

For something like a Microsoft Exchange Server, you're required to have multiple mail stores anyway, so having multiple RAID1 volumes fits naturally into an Exchange environment. Too many Exchange administrators follow the RAID10 advice, and it results in a massive waste in performance. So instead of trying to argue with them, I'll just present the following quantitative analysis comparing the various types of RAID.

I had to shift the results for 2xRAID1 to the right by a factor of 2 and shift the "four drives" result to the right by a factor of 4. This is to account for the fact that I had 2 and 4 times more threads more IOMeter workers than the other single volume results, which is the equivalent of 2 and 4 times more depth on the outstanding queue.

This didn't change the standings but narrowed the gap in performance. The red and blue lines actually run so close together that the red line blots out part of the blue line. The four drives were put there for reference only, since you wouldn't actually use it in a server environment for lack of redundancy. The orange line on the bottom merely shows the baseline performance of a single hard drive. It's clear that independent sets of RAID1 beat every other type of RAID, so long as you can make use of the additional volumes, which is trivial for most server applications.

Note that this particular chart includes the use of "half stroking," which is only the first half of the drive platter being used. I used this half-stroked 2xRAID1 configuration to illustrate the maximum potential in a production environment. When we go to an 8-drive or drive comparison, the difference in performance between the independent RAID1 pairs and the single-volume RAID array grows even wider. In write performance, the delta is even greater, and independent RAID1 sets continues to excel.

Note that these numbers may not be directly comparable since I didn't run the tests on the LSI, 3Ware, and Silicon image controllers, and they may not have had WriteBackCaching turned on. Intel seems to spike up above the pack for write performance. Developers are in short supply. Here are the skills and programming languages employers need. McDonald's quietly revealed its stunning future -- and some customers will like it. The painful shame of owning an Android phone.

Time for a Linux smartphone?