• vStorage VMFS Volume Grow
• Hot Extend for Virtual Disks
• Hot Virtual Disk Add/Remove

With these capabilities, if space gets tight in your vSphere environment, it is easy to be proactive and address the issue before anyone notices.  The process goes something like this:

• Allocate additional physical hard disk space on your SAN to an appropriate LUN
• Grow your VMFS datastore onto the newly added free space
• Extend, or add new, virtual disks for the VMs that need more storage
• Expand the volume inside the guest to create more usable space

All without a reboot: zero downtime.

Not All Virtualization Platforms are Created Equal

Don’t jump to the conclusion that all hypervisors offer the same flexibility.  Perhaps you are wondering about Hyper-V capabilities?

First, let’s take a look at some Microsoft System Center Virtual Machine Manager 2008 R2 (SCVMM) marketing statements:

The What’s New page announces:

Hot addition/removal of Storage: Allows the addition and removal of storage to virtualized infrastructure without interruption. Additionally, “live” management of virtual hard disk (VHDs) or iSCSI pass through disks, allows administrators to take advantage of additional backup scenarios and readily use mission critical and storage-intensive applications.

The Top Benefits list proclaims:

Hot addition/removal of storage: With this capability, administrators can quickly and efficiently respond to changing storage requirements of virtual machines. This ability to hot-add additional storage eliminates the previous need to take the host down to upgrade storage thus increasing business continuity for end users and reducing complexity for administrators. Additionally it allows administrators to confidently deploy mission critical applications (in which up-time is of paramount importance) that may have rapidly changing storage requirements such as web, database or other business applications.

An IT decision-maker just might get the impression that both ESX and Hyper-V have essentially the same features. They do not.

You may be surprised to find out that all of the descriptions above merely refer to adding a new virtual disk to a VM — providing the conditions are right.  You cannot grow an existing VHD, and you can’t safely remove a VHD with SCVMM.

Plan Ahead

First things first.  If your Hyper-V VM does not have a virtual SCSI adapter — templates and VMs from Hyper-V R1 do not — you won’t be able to hot add a new VHD until you correct that shortcoming.  Hello downtime.

SCVMM can add a new blank virtual disk to a VM or it can copy an existing one across the network from the Library — if you copied it there beforehand, but there is no way to add a VHD that may already be present on your SAN — even if it is already sitting right next to the destination VM.

Removal?

Up-time may be of paramount importance, but preventing data-loss was evidently not part of the original design.  Removing a VHD with SCVMM results in the immediate deletion of the underlying VHD file.  Ouch!  Thankfully, a recent patch improves administrator job security by throwing up a warning before this happens, providing an option to cancel.  There is no way to simply disconnect a VHD using SCVMM.

It turns out that if you really want to take advantage of those “Additional Backup Scenarios” by hot adding and removing virtual disks, you need your trusty Hyper-V Manager utility.  Still think System Center is a single pane of glass?  And that’s not the only task that requires administrators to switch between Hyper-V interfaces.

Conclusion

Microsoft is trying hard to ride the coattails of VMware ESX.  The latest release of Hyper-V R2 still does not have capabilities enjoyed by VMware administrators since the ESX 3.5 days.

Don’t believe the obfuscated marketing literature.  VMware vSphere is for real.

Related posts:

1. Responsible Thin Provisioning in VMware vSphere
2. Easy recovery from a full VMware ESX datastore
3. vSphere Thin-Provisioned Disk Performance
4. The Truth About VM SAN Transfers

• Full accounting for all provisioned storage space and complete monitoring
• Automatic response to a nearly-full datastore through vCenter Alarms, PowerShell, and Storage VMotion

You may be nearly convinced to start using thin provisioning, but still wondering…

What happens if a datastore fills up?

When a datastore runs out of space, thin-provisioned virtual disks can no longer dynamically grow to accommodate additional storage demand.  When VMware ESX detects this condition, virtual machines in need of additional storage are instantly paused to prevent guest operating systems from failing. Conversely, VMs that that read and write to existing allocated storage blocks will continue running without issue — not all virtual machines will be paused just because a datastore is out of space.

If you ever find yourself in this situation, it’s not hard to fix.  Here is one simple approach, step-by-step:

1. Free up some space by deleting or moving files — ISO images or powered-off VMs would be perfect
2. Resume one of the paused VMs
3. Use Storage VMotion to move the disks for that VM to another datastore
4. Resume the remaining VMs

Watch the procedure in action:

Depending on the size and storage demand of each VM, additional migrations may be needed.  An alternative resolution would be to add additional space to the SAN LUN and grow the VMFS volume.

The Experiment

To simulate a sudden storage demand by the thin-provisioned VMs in the above video, I simply copied a large file from a network share to each Windows Server 2003 VM simultaneously.

For the curious, below is a PowerShell script for the task.  Run it from anywhere — it uses Sysinternals psexec to remotely initiate a file copy on each VM from a network share.

# VM naming convention combines this string with 2-digit number
$vmPrefix = "VM2003e-"
 
# UNC path to a large file that will be copied into each VM
$iso="\\fileserver\ISO\large.iso"
 
# credentials to download above file from \\fileserver
$user="domain\username"
$pass="pa55word"
 
# pass this function a list of numbers
function createVmList ($series) {
    $vmList = @()
 
    foreach ($id in $series) {
        $vmList +=  %{"$vmPrefix{0:00}" -f $id}
    }
    $vmList
}
 
# make sure Sysinternals psexec is in your path
function psexecOnVm ($cmd, $vmList) {
    $vmList | % {Invoke-Expression "psexec.exe \\$_ -d -u $user -p $pass $cmd"}
}
 
psexecOnVM "cmd /c copy /Y $iso c:\" (createVMlist (31..39))

VMware ESX is Resilient

You may have been surprised at how easy it is to recover from a full datastore — without so much as a guest OS reboot.  It’s a testament to the rock-solid architecture behind VMware ESX and VMFS.  No other virtualization platform comes close.  Try for yourself.  See what happens if a group of thin-provisioned Hyper-V virtual machines suddenly run out of storage — it’s not going to be pretty.

Related posts:

1. Responsible Thin Provisioning in VMware vSphere
2. PowerShell Prevents Datastore Emergencies
3. vSphere Thin-Provisioned Disk Performance
4. Finding thin-provisioned virtual disks with PowerShell

Have you ever wondered what life in the virtualization world would be like without a clustered filesystem? Thanks to Microsoft Hyper-V, NTFS, and their failover cluster service, you no longer have to wonder, you can read about it in great detail.

In a world without VMFS, in order to have HA virtual machines or quick migration (let’s not bring up live migration today, since it is shipping “in 2010″) here is the most important point: Each virtual machine must be on its own LUN. Are you on good terms with your SAN administrator? Looks like the two of you are going to be chatting quite often.

Now if you prefer, you could set up several virtual machines on one LUN. But it turns out that such a plan would be destined for failure. No pun intended. Please take a look at the following quote from a Microsoft Technet blog:

If you group clustered VMs on a single LUN, don’t shut down the OS from within the OS or from Hyper-V Manager, this is not a cluster-aware shutdown and counts as a failure. The OS will be restarted by the cluster and then if you go “what the ?!?!” and shut it down again, depending on how the cluster resource is configured this will induce a failover to another node, taking the rest of your VMs with it. Try explaining that outage to your boss when a business critical server goes out of action for a couple of minutes at a crucial period because you shut down a scratch VM you had been using to test some software. I for one do not feel like having to get a change request authorized just to shut down a machine. Once you have made a VM a clustered VM, ALWAYS use Failover Cluster Management or the SCVMM Console to shut it down.

Ouch! What happens on Patch Tuesday? Let’s not find out.

One last thought: What about SAN deduplication technology that operates per-LUN?

Related posts:

1. IGT Part 4: Failover frustration
2. IGT Part 3: One-VM-per-LUN doubters
3. Choose any two: Hyper-V, HA, Linux
4. The Truth About VM SAN Transfers