VCritical » USB flash

VMware ESXi 4: SD Flash in BL460c G6

Eric Gray — Tue, 12 Jan 2010 12:15:15 +0000

HP has incorporated an interesting option for running VMware ESXi 4 from flash. Instead of using a USB stick, an SD card can simply be popped in the embedded reader. While any SD card would work, HP emphasizes that only a specific model is fully supported — the ones I received are actually made by Lexar and the official nomenclature is: HP 4GB SD Flash Media, Part Number 580387-B21.

In my HP BladeSystem c7000 enclosure with BL460 G6 (Nehalem) blades, the SD slot is easily exposed by sliding each blade out of the chassis:

Once the SD flash card is in place, it is trivial to install VMware ESXi:

You may also be interested in articles from Rick Scherer and TechHead on this topic.

Hyper-V on Flash?

Sure, Hyper-V Server — but not Windows Server 2008 with Hyper-V — supports boot-from-flash when purchased from a server OEM, but to the best of my knowledge there are no vendors shipping such configurations at this time. Note that Microsoft recommends a 16GB flash drive for Hyper-V Server deployment (compared to a slender 1GB for ESXi 4), but a PlayStation Portable is an acceptable substitute in a pinch.

Motivation

Two great reasons for running ESXi from flash:

Eliminate the need for local disks — lower power and cooling requirements
Provide dual-boot capabilities in a lab environment

Why would you run VMware ESXi from flash?

The VMware ESXi 4 64MB Hypervisor Challenge

Eric Gray — Fri, 28 Aug 2009 19:29:49 +0000

In a previous article, I answered the question: If VMware ESXi 4 is so small, why is it so big? It’s quite clear now that the disk footprint of VMware ESXi 4 is less than 60MB. But to really drive the point home, I wanted to demonstrate that VMware ESXi 4 could boot and run from a tiny 64MB flash device, so I asked Olivier Cremel, the inventor of ESXi, if that was feasible. He said it was — and gave me advice on how to set it up. This article shows you how.

Requirements

The VMware ESXi 4 dd image file (Vladan’s site may help)
A 64MB flash drive — check your junk drawer
A Linux machine — make sure you have mkfs.vfat
This fdisk script — save it as a txt file

Extract necessary components from the large image

Use dd to copy the first 1.5MB of the boot partition and the files that make up ESXi.

# dd if=VMware-VMvisor-big-175625-x86_64.dd of=esxi-boot-part skip=32 count=3000
3000+0 records in
3000+0 records out
1536000 bytes (1.5 MB) copied, 0.036249 seconds, 42.4 MB/s
# mkdir tmp
# mkdir esxi
# mount -o loop,offset=$((8224*512)) VMware-VMvisor-big-175625-x86_64.dd tmp/
# cp tmp/* esxi
# sync
# umount tmp/

Prepare the flash device

Plug in your flash device and use dmesg to determine the device id (/dev/sdk in this example). Run fdisk on it and delete all partitions – very important. Then, run my fdisk script to create the required partition structure. If this doesn’t work on your flash drive due to different size/geometry, you will need to create the partitions yourself — refer to the table at the end of these instructions for reference.

# fdisk /dev/sdk < fdisk.txt

Write data to the flash device

Use dd to write the saved boot partition to flash.

# dd if=esxi-boot-part of=/dev/sdk4
3000+0 records in
3000+0 records out
1536000 bytes (1.5 MB) copied, 1.31462 seconds, 1.2 MB/s
# sync

Then create a FAT filesystem on the flash drive, mount it, and copy the ESXi files.

# mkfs.vfat /dev/sdk5
mkfs.vfat 2.11 (12 Mar 2005)
# mount /dev/sdk5 tmp/
# cp esxi/* tmp/
# sync
# umount tmp/

Double-check your partition layout

If your disk partition does not look like this, something went wrong.

# fdisk -l /dev/sdk
 
Disk /dev/sdk: 65 MB, 65273856 bytes
3 heads, 42 sectors/track, 1011 cylinders
Units = cylinders of 126 * 512 = 64512 bytes
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sdk1              26        1011       62118    5  Extended
/dev/sdk4   *           1          25        1554    4  FAT16 <32M
/dev/sdk5              26        1011       62097    6  FAT16

Test it out

If you followed all of the instructions above, you should be able to boot from that flash drive.

There you have it — a fully functional VMware ESXi 4 system entirely contained on a 64MB flash device. Of course, this procedure is entirely unsupported, so please do not use a system like this in production. Please!

If VMware ESXi 4 is so small, why is it so big?

Eric Gray — Thu, 27 Aug 2009 20:21:40 +0000

Everything you ever wanted to know about the VMware ESXi footprint.

The flagship hypervisor from VMware actually comes in two different form-factors: classic VMware ESX with a Linux-based management console and thin VMware ESXi that can boot from a small embedded flash drive. Even though ESXi is available for free, it is functionally equivalent to classic ESX when fully licensed and managed by vCenter Server.

VMware ESX 3i, as it was originally known in the VI3 era, was just a tiny 32MB. With VMware vSphere 4, the name changed slightly to VMware ESXi 4, and grew to about 60MB — still extremely small for a fully-functional hypervisor with HA, VMotion, DRS, and all the other vSphere features.

While ESXi is less than 60MB, installing it actually requires a 1GB flash device. This raises a question:

VMware claims that ESXi is small — why does it require so much storage?

Unlike a general purpose operating system, ESXi was designed to be deployed as an image — similar to a router. The “installer” merely paves your empty storage device with some partitions. In fact, there are two separate partitions created for ESXi images — this allows administrators to roll back to a known-working environment if an update is not successful. In order to be prepared for the future, these two partitions are several times larger than the data they contain.

Not only does a 1GB flash device contain the ESXi hypervisor, it also provides VMware Tools for various supported operating systems and a copy of the vSphere Client which administrators can download and install to their workstations. These components are not executed by the hypervisor at all — they can be obtained through other means, but it is very convenient to have them right on the host.

Take a look at this visual to better understand the partition layout:

Partition 4 enables the actual booting from flash
Partition 5 contains files that make up the VMware ESXi image
Partition 6 is used to store a future update to the image file
Partition 7 is for a core dump
Partition 8 contains tools, drivers, and the Windows-based vSphere Client installer

In case you are curious, here are the contents of that last partition:

As you can see, it contains a bunch of Windows executables and ISO images for your various guests.

Still not convinced?

Some may still doubt the VMware ESXi 4 60MB footprint claim, but one sure way to make the case is to simply run ESXi from a very small flash drive. Olivier Cremel, the inventor of VMware ESXi, explained to me how to do just that. In my next post on this topic I’ll walk you through it, step by step. Don’t miss out — subscribe to the VCritical RSS feed today!

A very flashy hypervisor: Hyper-V Server R2

Eric Gray — Thu, 20 Aug 2009 04:53:33 +0000

The Microsoft Virtualization team put out a big ALL CAPS “me too!” announcement recently that Hyper-V Server R2 has the unique ability to boot from flash. Please note that “unique” means when compared to regular Hyper-V — VMware ESXi has offered this for years.

Oh, by the way, flash boot will be supported only by Microsoft OEM partners — not by Microsoft Product Support. Where have we heard that before? Sounds a lot like NIC teaming — yes, it is not unsupported. I can’t wait to read the KB article on this one.

It’s great to see Jeff Woolsey acknowledge the value of a small-footprint hypervisor, despite the fact that he really seemed to enjoy his latest three-part series on why Hyper-V is better than ESXi in every way. At least that is what I assume the series is about — between the crazy formatting and the ridiculous patch download metric he concocted, I just couldn’t get into it.

As it turns out, Jeff Woolsey also fills an important role at Microsoft as a part-time user interface QA tester for Windows Live Writer. You’ll notice in his posts that he exercises every conceivable text-formatting function offered by that blog editing tool — yes, the large, CAP, bold, underscore, and colored fonts all appear correctly. Tests passed!

If you are ready for a truly thin hypervisor that can boot from flash today, grab yourself a copy of VMware ESXi. You’ll like it.

Create ESXi 4 USB flash drives with Workstation

Eric Gray — Thu, 13 Aug 2009 21:02:38 +0000

Today I needed to temporarily set up three servers with VMware ESXi 4 — they were running something else that I was not quite ready to destroy. I decided to boot ESXi from flash and found a very easy way to image some USB flash drives with just VMware Workstation 6.5:

Create a new VMware Workstation VM that is capable of running ESX — make sure it has a USB controller
Pop in a USB flash drive (1GB)
Boot the ESXi (installable) ISO image
Use the VM | Removable Devices menu to connect the USB device to the VM
Run through the ESXi installer and select the USB drive as the installation destination
Wait just a few minutes while installation completes; repeat as needed
Stick the flash drives into the servers (I used a few DL360 G5s) and boot them up

Worked great! Have you set up any ESXi hosts to boot from flash?