Do’s and Don’ts of Domain Controller Virtualization

As with an increasing number of server roles, domain controllers a great candidates for virtualization. That doesn’t mean you can just simply through your domain controllers onto a virtual platform. There are a few things you must understand before you do your first deployment.

Hypervisors have given systems and infrastructure administrators powerful tools to manipulate and work with servers. However, as I have watched virtualization in the enterprise grow to the popularity it has now, I have seen an increasingly large number of administrators putting organizations at risk by following poor practices.

To help administrators avoid costly mistakes, I’ve written a short list of do’s and don’ts every administrator should know about Active Directory virtualization.

1. Don’t do Virtual Machine Snapshots

Active Directory uses update sequence numbers (USN) to keep track of changes made to objects in the entire domain and forest. Everytime an object is modified, a new USN is applied by the domain controller making the change. The other domain controllers in the domain are then notified of the change. When you perform a virtual machine snapshot of an active domain controller, objects continue to be updated and the new USN’s are being recorded.

As soon as you roll back your snapshot, the USNs of every object on that domain controller are also rolled back. This creates an inconsistent state between the snapshotted domain controller and all other domain controllers in the domain.

I have witnessed on multiple occasions organizations having to recovery their entire forest because of snapshots. It may seem like a good idea, and you may have gotten away with it in the past. But when you eventually find yourself at 3:00 AM performing a full-out disaster recovery, you will understand the risk wasn’t worth it. You may also find yourself unemployed very shortly afterwards.

2. Don’t place Domain Controller guests on Busy Hosts

Time drift is a phenonenom that occurs on every hypervisor. It’s caused by the very nature that today’s processor cores can only handle performing one or two processes at a time. On a busy system, this causes process queues. After a period of time a virtual machine’s clock starts to drift forwards or backwards as a result of the queue.

Active Directory heavily relies on having an accurate time set within an entire forest; every client and server joined to your forest must all have the same UTC time and date set. There is a small margin of allowed difference – five minutes by default.

If a domain controller’s time drifts to far outside of the allowed boundaries, authentication and replication issues occur. Kerberos tickets, used for resource access authentication requests, can no longer be verified by a domain controller.

It is highly recommended that you monitor your virtual server hosts to ensure they are not over subscribed with guest machines. As soon as you notice processor usage creeping to near capacity, it is strongly advised that you either move the domain controller to a different host, or you start migrating some of the other guests off of the host.

3. Do use File System Backups

As with physical domain controllers, the only safe and supported way of protecting a domain controller is to perform a system state backup. Using virtual machine snapshots is not supported for backing up domain controllers.  At best, you would be restoring a crash-consistent backup of your domain controller, which may have corrupted. At worst, you will introduce a USN rollback.

Server 2012 introduced new features to support virtual machine snapshots of domain controllers. However, there are limitations that you need to be aware of. Among the limitations is the feature is only available on HyperV hosts.

4. Do not Clone a Domain Controller

Do not clone your domain controllers to scale out your Active Directory domain or forest. It is not supported in Windows Server 2008 or lower.

Server 2012 and up, on the other hand, does offer limited support for cloning. Howeverm there are a lot of prerequesite steps that must be completed first. It’s still not as simple as clicking the Clone button. Make sure you understand how to prepare a Server 2012 and up domain controller for cloning.