With the various advances in hypervisor technology, deploying Exchange Server in a virtual environment has become more attractive than ever. However, certain factors stand in the way of successful virtualized Exchange deployments -- some of which admins may not have considered.
Virtualizing Exchange Server is an enticing option. After all, increased CPU and memory capacity allow virtual Exchange machines to scale beyond the limitations the previous generations of hypervisors imposed, thereby reducing the number of virtual machines needed for large deployments. Also, the parity gap between virtual and physical Exchange environments has never been smaller.
On paper, virtualizing Exchange looks like a smart decision. However, with any virtual Exchange deployment, you'll always be bringing more complexity to the table.
No matter how efficient the hypervisor, or how scalable your VMs can be, the underlying assumption of virtualization -- that applications don't need to know about the actual hardware they're running on if the hypervisor provides a good enough emulation -- doesn't actually apply to Exchange Server.
Exchange Server goes to great measures to ensure that your messaging data is always as safe as it can possibly be. Think of it this way: Anything that potentially disrupts Exchange's knowledge of the hardware is a potential data risk. This is the same reason storage products that use file-level storage at any level between Exchange and
Virtualizing Exchange Server barrier #1: Network delays
Introducing network delays is the first way virtualization can destabilize Exchange Server. These delays can potentially disrupt the delicately timed cluster heartbeat communications between members of an Exchange database availability group (DAG). There are four common reasons for network delay problems:
- Mismatching network drivers and firmware. Hosts, patches, updates and upgraded network drivers are sure to be applied over the lifetime of the hypervisor. Many administrators forget that the VM network drivers should also be upgraded to keep all the components of the network stack synchronized.
- Overloading virtual network switches. A shared virtual network switch can -- on overloaded hypervisor hosts or clusters -- introduce network latency or dropped packets. Exchange servers don't require separate dedicated virtual network switches, but be careful about what other traffic passes through those switches.
- Moving live Exchange VMs between hosts. Although the ability to move live VMs is an oft-touted hypervisor feature, it produces a small but measurable loss of network connectivity.
This is often enough to make a DAG member (which is unaware of the fact that it is moving from one hypervisor host to another) momentarily drop out of the cluster. This forces an unplanned failover of all active databases, and sometimes even causes the replication service to fail. This doesn't result in data loss, but it can reduce the number of available replicas, thereby reducing the overall level of redundancy.
- Associating different types of traffic to the same host interfaces. The typical hypervisor host will have two or three sets of host interfaces: management, storage and client traffic. For an Exchange VM that is also a DAG member, these might not be enough. Associating the VM's replication network interface to any of these physical host interface groups can inject network instability, causing hard-to-troubleshoot cluster membership dropouts and database failovers.
Over time, network issues can have a significant impact on the reliability and perceived stability of the DAG.
Virtualizing Exchange Server barrier #2: Built-in safety mechanisms
Allowing hypervisors and/or administrators to circumvent Exchange's built-in data safety mechanisms is the other major way virtualization can destabilize Exchange Server.
Consider these facts:
- Exchange never permits two replicas of the same database to be created on the same server. However, if two DAG member VMs are put on the same physical host, that's exactly what you can end up with.
- Even if two Exchange VMs on the same host don't share database replica copies, that host becomes a potential source of risk for the health of the DAG if they're members of the same DAG. I've seen a DAG fail because of bugs and procedural errors that took out an entire hypervisor cluster.
- Many inter-role transactions in an Exchange system exhibit different behaviors depending on how the Exchange roles are colocated. Exchange has no way to detect situations where the mailbox and hub transport roles, or two hub transport roles, are on the same physical host. In these cases, a host-level failure can result in the loss of the transport dumpster or shadow redundancy information, therefore resulting in the loss of messaging data.
Careful consideration of potential VM placement under all conditions is the key to preventing these types of issues. In many deployments I've observed, the organization ended up deploying Exchange across more hypervisor hosts than they had originally planned; this actually reduced the level of consolidation and increased overall cost. In many cases, changes to default VM configuration and to standard operations procedures are also necessary to ensure that the appropriate VM constraints are observed.
It's certainly possible to build a reliable Exchange deployment in a virtualized environment. However, there are many factors to consider, plan for and test in order to ensure that the virtualization configuration is not also increasing the risk of service outage and data loss.
Carefully evaluate your plans to make sure that the additional costs and changes are worth the actual level of consolidation you will likely achieve. The right answer may be to not virtualize Exchange at all -- yet.
About the author
Devin Ganger is a messaging architect and technical writer with more than 15 years of experience in administering messaging systems, Windows, Unix and TCP/IP networks. Today, he works primarily with Exchange Server, Windows Active Directory and related Microsoft and third-party technologies. Ganger was recognized as a Microsoft MVP for Exchange Server from 2007 to 2011.
This was first published in February 2013