VCritical » ESXi

Hypervisor Footprint Quiz

Eric Gray — Wed, 10 Feb 2010 22:44:54 +0000

Remember the hypervisor footprint debate? You know, the one where Microsoft Virtualization declares that it is nothing but VMware FUD to tout the benefits of a small-footprint hypervisor.

I just found another point of view on hypervisor footprint size — take a look at this excerpt:

Small footprint: any features not applicable to our specific … scenarios are removed. This guarantees that we do not have to worry about updating or fixing unnecessary code, meaning less churning or required reboots for the host. All critical code paths are also highly optimized for our … scenarios.

Any guesses where that came from? Must be more of that VMware FUD!

Actually, it was one of the Windows Azure design principles. Which makes sense if you think about it — vSphere, with small-footprint ESXi, is the perfect foundation for cloud computing.

Win $2500 with your 1337 VMware ESXi scripting skillz

Eric Gray — Tue, 02 Feb 2010 18:37:28 +0000

VMware is currently running a VMware ESXi scripting contest. Top prize is $2500, which is not too shabby for doing something productive.

Check out the details and get scripting!

VMware ESXi 4: SD Flash in BL460c G6

Eric Gray — Tue, 12 Jan 2010 12:15:15 +0000

HP has incorporated an interesting option for running VMware ESXi 4 from flash. Instead of using a USB stick, an SD card can simply be popped in the embedded reader. While any SD card would work, HP emphasizes that only a specific model is fully supported — the ones I received are actually made by Lexar and the official nomenclature is: HP 4GB SD Flash Media, Part Number 580387-B21.

In my HP BladeSystem c7000 enclosure with BL460 G6 (Nehalem) blades, the SD slot is easily exposed by sliding each blade out of the chassis:

Once the SD flash card is in place, it is trivial to install VMware ESXi:

You may also be interested in articles from Rick Scherer and TechHead on this topic.

Hyper-V on Flash?

Sure, Hyper-V Server — but not Windows Server 2008 with Hyper-V — supports boot-from-flash when purchased from a server OEM, but to the best of my knowledge there are no vendors shipping such configurations at this time. Note that Microsoft recommends a 16GB flash drive for Hyper-V Server deployment (compared to a slender 1GB for ESXi 4), but a PlayStation Portable is an acceptable substitute in a pinch.

Motivation

Two great reasons for running ESXi from flash:

Eliminate the need for local disks — lower power and cooling requirements
Provide dual-boot capabilities in a lab environment

Why would you run VMware ESXi from flash?

Layers and Layers of FUD

Eric Gray — Tue, 03 Nov 2009 12:02:49 +0000

Unbelievable. That old tale about VMware adding an extra layer is still being told in the Microsoft Virtualization Comparison Brochure. [Update: The document was pulled by Microsoft and slightly modified as a result of this article.] Pretty surprising, and not at all honest — using the term layer implies a particular order or hierarchy, not merely quantity. And what on earth is the virtualization layer doing on top of applications?

The case Microsoft is really trying to make here is that if you don’t have VMware virtualization, then you don’t have VMware. True enough, I suppose. But you still have the same number of layers.

I took the liberty of making some corrections to the Microsoft collateral and am pleased to present three more appropriate ways to depict these virtualization layers.

The Objective Comparison

Truth be told, both virtualization platforms have approximately the same high-level depiction of layers:

Nothing surprising there — that is how any truly technical content would describe the arrangement. Notice that virtualization is actually not above the application layer.

The Proportional Correction

VMware ESX and Microsoft Hyper-V both function as a hypervisor, but it turns out that since ESX has a much smaller footprint, one might envision the stack of colored boxes more like this:

Combining Layers — The Right Way

Is virtualization more like hardware or more like software? If I had to choose, I would pick hardware. Perhaps those are the two layers that should be combined, like this:

As long as the hypervisor is not tied to a general-purpose operating system, this makes good sense. However, since Hyper-V is merely a role in Windows Server, it would be a real stretch to picture the same combination for Microsoft virtualization. Now that I’ve reasoned it through, maybe three layers isn’t so bad after all.

How do you like them layers?

Leave a Comment • Subscribe to RSS • Subscribe via Email • Follow Eric Gray on Twitter
More articles on: ESX, ESXi, Hyper-V • Browse All Virtualization Content
Layers and Layers of FUD by Eric Gray © 2009 • VCritical

The VMware ESXi 4 64MB Hypervisor Challenge

Eric Gray — Fri, 28 Aug 2009 19:29:49 +0000

In a previous article, I answered the question: If VMware ESXi 4 is so small, why is it so big? It’s quite clear now that the disk footprint of VMware ESXi 4 is less than 60MB. But to really drive the point home, I wanted to demonstrate that VMware ESXi 4 could boot and run from a tiny 64MB flash device, so I asked Olivier Cremel, the inventor of ESXi, if that was feasible. He said it was — and gave me advice on how to set it up. This article shows you how.

Requirements

The VMware ESXi 4 dd image file (Vladan’s site may help)
A 64MB flash drive — check your junk drawer
A Linux machine — make sure you have mkfs.vfat
This fdisk script — save it as a txt file

Extract necessary components from the large image

Use dd to copy the first 1.5MB of the boot partition and the files that make up ESXi.

# dd if=VMware-VMvisor-big-175625-x86_64.dd of=esxi-boot-part skip=32 count=3000
3000+0 records in
3000+0 records out
1536000 bytes (1.5 MB) copied, 0.036249 seconds, 42.4 MB/s
# mkdir tmp
# mkdir esxi
# mount -o loop,offset=$((8224*512)) VMware-VMvisor-big-175625-x86_64.dd tmp/
# cp tmp/* esxi
# sync
# umount tmp/

Prepare the flash device

Plug in your flash device and use dmesg to determine the device id (/dev/sdk in this example). Run fdisk on it and delete all partitions – very important. Then, run my fdisk script to create the required partition structure. If this doesn’t work on your flash drive due to different size/geometry, you will need to create the partitions yourself — refer to the table at the end of these instructions for reference.

# fdisk /dev/sdk < fdisk.txt

Write data to the flash device

Use dd to write the saved boot partition to flash.

# dd if=esxi-boot-part of=/dev/sdk4
3000+0 records in
3000+0 records out
1536000 bytes (1.5 MB) copied, 1.31462 seconds, 1.2 MB/s
# sync

Then create a FAT filesystem on the flash drive, mount it, and copy the ESXi files.

# mkfs.vfat /dev/sdk5
mkfs.vfat 2.11 (12 Mar 2005)
# mount /dev/sdk5 tmp/
# cp esxi/* tmp/
# sync
# umount tmp/

Double-check your partition layout

If your disk partition does not look like this, something went wrong.

# fdisk -l /dev/sdk
 
Disk /dev/sdk: 65 MB, 65273856 bytes
3 heads, 42 sectors/track, 1011 cylinders
Units = cylinders of 126 * 512 = 64512 bytes
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sdk1              26        1011       62118    5  Extended
/dev/sdk4   *           1          25        1554    4  FAT16 <32M
/dev/sdk5              26        1011       62097    6  FAT16

Test it out

If you followed all of the instructions above, you should be able to boot from that flash drive.

There you have it — a fully functional VMware ESXi 4 system entirely contained on a 64MB flash device. Of course, this procedure is entirely unsupported, so please do not use a system like this in production. Please!

If VMware ESXi 4 is so small, why is it so big?

Eric Gray — Thu, 27 Aug 2009 20:21:40 +0000

Everything you ever wanted to know about the VMware ESXi footprint.

The flagship hypervisor from VMware actually comes in two different form-factors: classic VMware ESX with a Linux-based management console and thin VMware ESXi that can boot from a small embedded flash drive. Even though ESXi is available for free, it is functionally equivalent to classic ESX when fully licensed and managed by vCenter Server.

VMware ESX 3i, as it was originally known in the VI3 era, was just a tiny 32MB. With VMware vSphere 4, the name changed slightly to VMware ESXi 4, and grew to about 60MB — still extremely small for a fully-functional hypervisor with HA, VMotion, DRS, and all the other vSphere features.

While ESXi is less than 60MB, installing it actually requires a 1GB flash device. This raises a question:

VMware claims that ESXi is small — why does it require so much storage?

Unlike a general purpose operating system, ESXi was designed to be deployed as an image — similar to a router. The “installer” merely paves your empty storage device with some partitions. In fact, there are two separate partitions created for ESXi images — this allows administrators to roll back to a known-working environment if an update is not successful. In order to be prepared for the future, these two partitions are several times larger than the data they contain.

Not only does a 1GB flash device contain the ESXi hypervisor, it also provides VMware Tools for various supported operating systems and a copy of the vSphere Client which administrators can download and install to their workstations. These components are not executed by the hypervisor at all — they can be obtained through other means, but it is very convenient to have them right on the host.

Take a look at this visual to better understand the partition layout:

Partition 4 enables the actual booting from flash
Partition 5 contains files that make up the VMware ESXi image
Partition 6 is used to store a future update to the image file
Partition 7 is for a core dump
Partition 8 contains tools, drivers, and the Windows-based vSphere Client installer

In case you are curious, here are the contents of that last partition:

As you can see, it contains a bunch of Windows executables and ISO images for your various guests.

Still not convinced?

Some may still doubt the VMware ESXi 4 60MB footprint claim, but one sure way to make the case is to simply run ESXi from a very small flash drive. Olivier Cremel, the inventor of VMware ESXi, explained to me how to do just that. In my next post on this topic I’ll walk you through it, step by step. Don’t miss out — subscribe to the VCritical RSS feed today!

A very flashy hypervisor: Hyper-V Server R2

Eric Gray — Thu, 20 Aug 2009 04:53:33 +0000

The Microsoft Virtualization team put out a big ALL CAPS “me too!” announcement recently that Hyper-V Server R2 has the unique ability to boot from flash. Please note that “unique” means when compared to regular Hyper-V — VMware ESXi has offered this for years.

Oh, by the way, flash boot will be supported only by Microsoft OEM partners — not by Microsoft Product Support. Where have we heard that before? Sounds a lot like NIC teaming — yes, it is not unsupported. I can’t wait to read the KB article on this one.

It’s great to see Jeff Woolsey acknowledge the value of a small-footprint hypervisor, despite the fact that he really seemed to enjoy his latest three-part series on why Hyper-V is better than ESXi in every way. At least that is what I assume the series is about — between the crazy formatting and the ridiculous patch download metric he concocted, I just couldn’t get into it.

As it turns out, Jeff Woolsey also fills an important role at Microsoft as a part-time user interface QA tester for Windows Live Writer. You’ll notice in his posts that he exercises every conceivable text-formatting function offered by that blog editing tool — yes, the large, CAP, bold, underscore, and colored fonts all appear correctly. Tests passed!

If you are ready for a truly thin hypervisor that can boot from flash today, grab yourself a copy of VMware ESXi. You’ll like it.

Create ESXi 4 USB flash drives with Workstation

Eric Gray — Thu, 13 Aug 2009 21:02:38 +0000

Today I needed to temporarily set up three servers with VMware ESXi 4 — they were running something else that I was not quite ready to destroy. I decided to boot ESXi from flash and found a very easy way to image some USB flash drives with just VMware Workstation 6.5:

Create a new VMware Workstation VM that is capable of running ESX — make sure it has a USB controller
Pop in a USB flash drive (1GB)
Boot the ESXi (installable) ISO image
Use the VM | Removable Devices menu to connect the USB device to the VM
Run through the ESXi installer and select the USB drive as the installation destination
Wait just a few minutes while installation completes; repeat as needed
Stick the flash drives into the servers (I used a few DL360 G5s) and boot them up

Worked great! Have you set up any ESXi hosts to boot from flash?

New VMware ESXi Management Kit

Eric Gray — Wed, 04 Feb 2009 17:58:19 +0000

VMware recently announced the new VMware ESXi Management Kit. If you have up to three free ESXi servers deployed and would like to manage them with vCenter Server, this is a great opportunity. Available until April 30 for $995.

The kit includes:

VMware vCenter Server Foundation
vCenter Agents for 3 ESXi hosts (2 CPU sockets each host)
1 year software subscription

If you are running ESXi without vCenter Server (formerly known as VirtualCenter), you are missing out on some great capabilities, such as:

VM cloning and template deployment
Single instance of VI Client to manage all hosts and VMs
Historical performance data for VMs and hosts

Of course there is much more–give it a try.

Leave a Comment • Subscribe to RSS • Subscribe via Email • Follow Eric Gray on Twitter
More articles on: ESX, ESXi, vCenter • Browse All Virtualization Content
New VMware ESXi Management Kit by Eric Gray © 2009 • VCritical

Using SCVMM to attach ISO images to VMware ESX VMs

Eric Gray — Fri, 09 Jan 2009 20:59:04 +0000

When a virtual machine requires access to a CD or DVD, it is very typical to connect an ISO image file to the VM instead of using a physical disc. When using VMware ESX or ESXi, multiple VMs are allowed to access a single ISO image on shared storage. System Center Virtual Machine Manager 2008 (SCVMM) uses a different design — instead of sharing a single image from the library server, a copy of the image is made to the SAN for each VM before use. I have written about this before.

If VMware vCenter and SCVMM use opposing techniques to perform this operation, what happens when an administrator tries to use SCVMM to manage ESX or ESXi? Evidently not wanting to give ESX special treatment, SCVMM copies the ISO just like it does for Hyper-V.

“Copies it how?” you must be thinking. With SFTP through the ESX service console, of course!

Let’s review the numerous disadvantages of not having a shared ISO image repository:

Takes longer to mount an ISO to a VM
Ties up expensive SAN storage for duplicate copies of images
Uses network bandwidth to copy these multi-gigabyte files
Wastes ESX service console CPU cycles to receive SFTP uploads

I was giving a presentation this morning to the VMware Sales team and a question was raised by one of the participants: If you are using VMware ESXi how is the image transferred — there is no service console and therefore no SSH daemon. Not having tried this operation, I theorized that it would be done using the API — similar to how the VI Client can easily upload files to an ESXi datastore.

After that presentation, I hijacked a colleague’s ESXi system and tried to connect an ISO image to a VM. Sure enough, the file was transferred via HTTP. So far, so good.

But there was a problem. Shortly after this job finished, the VM in question started showing warning messages about not being able to locate the file. I opened the VI Client and used the datastore browser to look at the VM directory. There was no ISO image to be found.

Digging a little deeper, I grabbed the hostd.log file from the ESXi server to look for clues. Here is an excerpt:

 warning] Exception while processing request: Input/output error
 info] Incomplete message body received for PUT /folder/win2k3r2/en_win_srv_

Looks like a problem with the SCVMM/ESXi integration. As far as I can tell the host is properly configured in SCVMM — which is accomplished by providing the root password so tasks like this can be performed. (But that’s an entirely different issue to talk about.) I separately verified that I could upload a file to that same host using the VI Client.

I must conclude that it is not possible to use SCVMM to attach an ISO image file to VMware ESXi virtual machines. Is SCVMM a single pane of glass to manage your entire virtual infrastructure? I don’t think so.