Slow disk subsystems are often the cause of many performance bottlenecks, sometimes even more so than server-side CPU or RAM issues. As a first indicator, your disk subsystem is problematically slow when:
- Average read and write latencies are over 20 ms
- Latency spikes over 50 ms for more than a few seconds
Fast storage with a sufficient amount of spindles is one of the most important investments you can make in your email messaging infrastructure. As a general tip, you should invest in high performance hard disks with 15,000 rpm. It is better to have smaller capacity disks that utilize each spindle's performance than to use fewer spindles with large capacity.
What impacts disk I/O on Exchange Server?
- Exchange Jet database files (*.EDB)
- Streaming media files (*.STM)
- Exchange transaction log files (*.LOG)
- Configuration of "Zero out deleted database pages" in Exchange System Manager -- with this setting enabled, Exchange Server will overwrite deleted pages with zeros
- SMTP mail transmission can additionally load your hard disks with more I/O
- If you enable content indexing on your Exchange Server databases, you will create heavy paging and more disk I/O on your hard disks
What is RAID?
In computing, a redundant array of independent disks (RAID) is a system of using multiple hard drives for sharing or replicating data. Depending on what version you use, the benefits of RAID include increased data integrity, fault-tolerance, throughput or capacity compared to single drives.
In its original implementations (in which it was an abbreviation for "redundant array of inexpensive disks"), its key advantage was the ability to combine multiple low-cost devices using older technology into an array that offered greater capacity, reliability, or speed -- or a combination of these things -- than was affordably available in a single device using the newest technology.
Below, I provide an overview of commonly used RAID levels. The description of the RAID levels is copied from Wikipedia.com.
RAID 0
A RAID 0 (also known as a striped set) splits data evenly across two or more disks with no parity information for redundancy. It is important to note that RAID 0 was not one of the original RAID levels, and it is not redundant.
RAID 0 is normally used to increase performance, although it can also be used as a way to create a small number of large virtual disks out of a large number of small physical ones.
RAID 0 can be created with disks of differing sizes, but the storage space added to the array by each disk is limited to the size of the smallest disk. For example, if a 120 GB disk is striped together with a 100 GB disk, the size of the array will be 200 GB.
Note: RAID 0 is never recommended for storing Exchange Server databases.
Figure 1:
RAID 1
RAID 1 creates an exact copy (or mirror) of a set of data on two or more disks. This is useful when write performance is more important than minimizing the storage capacity used for redundancy. However, the array can only be as big as the smallest member disk.
A classic RAID 1 mirrored pair contains two disks, which increases reliability by a factor of two over a single disk, but it is possible to have many more than two copies. Since each member can be addressed independently if the other fails, reliability is a linear multiple of the number of members.
Figure 2:
RAID 5
RAID 5 uses block-level striping with parity data distributed across all member disks. RAID 5 has achieved popularity due to its low cost of redundancy.
Generally RAID 5 is implemented with hardware support for parity calculations
Every time a data "block" (sometimes called a "chunk") is written on a disk in an array, a parity block is generated within the same stripe.
In general, RAID 5 does not provide the best reliability, availability and performance.
Figure 3:
RAID 10
A RAID 10, sometimes called RAID 1+0, or RAID 1&0, is similar to a RAID 0+1 with exception that the RAID levels used are reversed – RAID 10 is a stripe of mirrors.
One drive from each RAID 1 set could fail without damaging the data. However, if the failed drive is not replaced, the single working hard drive in the set then becomes a single point of failure for the entire array. If that single hard drive then fails, all data stored in the entire array is lost.
Figure 4:
Example of a recommended configuration
RAID 1 = System volume, operating system, Exchange Server binaries
RAID 1 = Pagefile
RAID 0+1 = SMTP and MTA queues
RAID 1 = Log files from one Exchange Server storage group
RAID 0+1 = Exchange Server databases from storage group
Exchange Server RAID tips
- Place database files from Exchange storage groups on separate hard disks.
- Never put content indexing files on the same disk as the pagefile.
- Place the transaction log files for each Exchange storage group to a dedicated RAID 1 or RAID-0+1 array.
- Place the SMTP queue on a separate disk subsystem.
- Place the pagefile on separate disks on RAID 1 because a lost of a pagefile could cause a stop error.
Diskpar
DiskPar utility (part of the Windows Server 2003 Resource Kit) can verify that your disk tracks are sector-aligned. You can use DiskPar to create aligned partitions (as compared with non-aligned partitions that are created with Disk Manager from Windows Server 2003) to increase disk performance up to 20% (as stated by Microsoft).
Jetstress
You can use Jetstress (part of the free Exchange downloads from the Microsoft Exchange Web site) to verify the performance and stability of a disk subsystem prior to putting your Exchange servers into production.
Jetstress helps you to verify disk performance by simulating Exchange Server disk I/O load. It simulates Exchange Server database load from the number of users you specify. The tool is somewhat similar to the Exchange 2003 tool LOADSIM, but more specialized for database I/O tests.
Conclusion
A high-performance RAID solution for Exchange Server is not difficult to implement, but what RAID scenario is right for your organization depends on your Exchange Server I/O usage and the number of mailboxes you're supporting. You should analyze your requirements before you buy server hardware and disk subsystems.
Related links:
Step-by-Step Guide: How to spec your Exchange Server hardware needs (SearchExchange.com)
Tip: Increase Exchange Server disk performance with aligned partitions (SearchExchange.com)
Expert Advice: Cons of setting up a domain controller/Exchange with RAID 5 (SearchExchange.com)
Tutorial: Disk geometry (MSExchange.org)
Technical Guide: Optimizing storage for Exchange Server 2003 (Microsoft)
Step-by-Step Guide: Exchange Server capacity planning with Performance Monitor (SearchExchange.com)
White Paper: Disk subsystem performance analysis for Windows (Microsoft)
Learning Guide: An admin's guide to Exchange Server performance management (SearchExchange.com)
Reference Center: Exchange Server database management tips and resources (SearchExchange.com)
About the author: Marc Grote is a MCSA/MCSE Messaging & Security, Microsoft Certified Trainer, and Microsoft MVP for ISA Server. He works as a freelance IT trainer and consultant in the north of Germany and as a part-time employee of Invenate GmbH in Hanover, Germany, consulting on the company's Windows server infrastructure. He specializes in ISA Server, Exchange Server, and security on Windows 2000 and Windows 2003. You can visit his homepage at http://www.it-training-grote.de.
Do you have comments on this tip? Let us know.
Please let others know how useful this tip was via the rating scale below. Do you have a useful Exchange Server or Microsoft Outlook tip, timesaver or workaround to share? Submit it to SearchExchange.com. If we publish it, we'll send you a nifty thank-you gift.
