Now with VMware ESXi 5, it is possible to perform an automated installation to USB storage.  Simply specify usb-storage as part of the install command in an automated install script.  For example:

install --firstdisk=usb-storage --overwritevmfs

For more flexibility in a single script, multiple disk types can be specified; they will be tried in order.  Below is an example install command from a script for HP servers.  If USB flash is not installed, the on-board SAS disks will be used.  If neither are present — such as when installing virtual ESXi systems — generic local storage is the last resort.

install --firstdisk=usb-storage,hpsa,local --overwritevmfs

Perhaps not one of the major new feature of VMware vSphere 5, but handy.

Boot Hyper-V From Flash?

You might not know this, but Microsoft Hyper-V has had the ability to boot from flash as well — announced two years ago, but yet to be acknowledged in an actual production deployment.  Of course, this bleeding-edge configuration option is only supported if installed by OEMs, not end-users.  Anyone considering Hyper-V for virtualization is highly encouraged to give it a try; it’s a great opportunity to learn a lot about Windows registry manipulation and command line tools such as imagex and diskpart.

But seriously, VMware vSphere 5 will be here soon, and you’re going to love it.

Related posts:

1. Create ESXi 4 USB flash drives with Workstation
2. VMware ESXi 4: SD Flash in BL460c G6
3. The VMware ESXi 4 64MB Hypervisor Challenge
4. A very flashy hypervisor: Hyper-V Server R2

In my HP BladeSystem c7000 enclosure with BL460 G6 (Nehalem) blades, the SD slot is easily exposed by sliding each blade out of the chassis:

Once the SD flash card is in place, it is trivial to install VMware ESXi:

You may also be interested in articles from Rick Scherer and TechHead on this topic.

Hyper-V on Flash?

Sure, Hyper-V Server — but not Windows Server 2008 with Hyper-V — supports boot-from-flash when purchased from a server OEM, but to the best of my knowledge there are no vendors shipping such configurations at this time.  Note that Microsoft recommends a 16GB flash drive for Hyper-V Server deployment (compared to a slender 1GB for ESXi 4), but a PlayStation Portable is an acceptable substitute in a pinch.

Motivation

Two great reasons for running ESXi from flash:

• Eliminate the need for local disks — lower power and cooling requirements
• Provide dual-boot capabilities in a lab environment

Why would you run VMware ESXi from flash?

Related posts:

1. Create ESXi 4 USB flash drives with Workstation
2. VMware ESXi 5 Scripted Install to USB Flash
3. A very flashy hypervisor: Hyper-V Server R2
4. The VMware ESXi 4 64MB Hypervisor Challenge

Requirements

• The VMware ESXi 4 dd image file (Vladan’s site may help)
• A 64MB flash drive — check your junk drawer
• A Linux machine — make sure you have mkfs.vfat
• This fdisk script — save it as a txt file

Extract necessary components from the large image

Use dd to copy the first 1.5MB of the boot partition and the files that make up ESXi.

# dd if=VMware-VMvisor-big-175625-x86_64.dd of=esxi-boot-part skip=32 count=3000
3000+0 records in
3000+0 records out
1536000 bytes (1.5 MB) copied, 0.036249 seconds, 42.4 MB/s
# mkdir tmp
# mkdir esxi
# mount -o loop,offset=$((8224*512)) VMware-VMvisor-big-175625-x86_64.dd tmp/
# cp tmp/* esxi
# sync
# umount tmp/

Prepare the flash device

Plug in your flash device and use dmesg to determine the device id (/dev/sdk in this example).  Run fdisk on it and delete all partitions – very important.  Then, run my fdisk script to create the required partition structure.  If this doesn’t work on your flash drive due to different size/geometry, you will need to create the partitions yourself — refer to the table at the end of these instructions for reference.

# fdisk /dev/sdk < fdisk.txt

Write data to the flash device

Use dd to write the saved boot partition to flash.

# dd if=esxi-boot-part of=/dev/sdk4
3000+0 records in
3000+0 records out
1536000 bytes (1.5 MB) copied, 1.31462 seconds, 1.2 MB/s
# sync

Then create a FAT filesystem on the flash drive, mount it, and copy the ESXi files.

# mkfs.vfat /dev/sdk5
mkfs.vfat 2.11 (12 Mar 2005)
# mount /dev/sdk5 tmp/
# cp esxi/* tmp/
# sync
# umount tmp/

Double-check your partition layout

If your disk partition does not look like this, something went wrong.

# fdisk -l /dev/sdk
 
Disk /dev/sdk: 65 MB, 65273856 bytes
3 heads, 42 sectors/track, 1011 cylinders
Units = cylinders of 126 * 512 = 64512 bytes
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sdk1              26        1011       62118    5  Extended
/dev/sdk4   *           1          25        1554    4  FAT16 <32M
/dev/sdk5              26        1011       62097    6  FAT16

Test it out

If you followed all of the instructions above, you should be able to boot from that flash drive.

There you have it — a fully functional VMware ESXi 4 system entirely contained on a 64MB flash device. Of course, this procedure is entirely unsupported, so please do not use a system like this in production. Please!

Related posts:

1. Create ESXi 4 USB flash drives with Workstation
2. If VMware ESXi 4 is so small, why is it so big?
3. VMware ESXi 5 Interactive PXE Installation Improvements
4. VMware ESXi 4: SD Flash in BL460c G6

The flagship hypervisor from VMware actually comes in two different form-factors: classic VMware ESX with a Linux-based management console and thin VMware ESXi that can boot from a small embedded flash drive.  Even though ESXi is available for free, it is functionally equivalent to classic ESX when fully licensed and managed by vCenter Server.

VMware ESX 3i, as it was originally known in the VI3 era, was just a tiny 32MB.  With VMware vSphere 4, the name changed slightly to VMware ESXi 4, and grew to about 60MB — still extremely small for a fully-functional hypervisor with HA, VMotion, DRS, and all the other vSphere features.

While ESXi is less than 60MB, installing it actually requires a 1GB flash device.  This raises a question:

VMware claims that ESXi is small — why does it require so much storage?

Unlike a general purpose operating system, ESXi was designed to be deployed as an image — similar to a router.  The “installer” merely paves your empty storage device with some partitions.  In fact, there are two separate partitions created for ESXi images — this allows administrators to roll back to a known-working environment if an update is not successful.  In order to be prepared for the future, these two partitions are several times larger than the data they contain.

Not only does a 1GB flash device contain the ESXi hypervisor, it also provides VMware Tools for various supported operating systems and a copy of the vSphere Client which administrators can download and install to their workstations.  These components are not executed by the hypervisor at all — they can be obtained through other means, but it is very convenient to have them right on the host.

Take a look at this visual to better understand the partition layout:

• Partition 4 enables the actual booting from flash
• Partition 5 contains files that make up the VMware ESXi image
• Partition 6 is used to store a future update to the image file
• Partition 7 is for a core dump
• Partition 8 contains tools, drivers, and the Windows-based vSphere Client installer

In case you are curious, here are the contents of that last partition:

As you can see, it contains a bunch of Windows executables and ISO images for your various guests.

Still not convinced?

Some may still doubt the VMware ESXi 4 60MB footprint claim, but one sure way to make the case is to simply run ESXi from a very small flash drive.  Olivier Cremel, the inventor of VMware ESXi, explained to me how to do just that. In my next post on this topic I’ll walk you through it, step by step.  Don’t miss out — subscribe to the VCritical RSS feed today!

Related posts:

1. The VMware ESXi 4 64MB Hypervisor Challenge
2. Create ESXi 4 USB flash drives with Workstation
3. VMware ESXi 4: SD Flash in BL460c G6
4. VMware ESXi 5 Scripted Install to USB Flash

Oh, by the way, flash boot will be supported only by Microsoft OEM partners — not by Microsoft Product Support.  Where have we heard that before?  Sounds a lot like NIC teaming — yes, it is not unsupported.  I can’t wait to read the KB article on this one.

It’s great to see Jeff Woolsey acknowledge the value of a small-footprint hypervisor, despite the fact that he really seemed to enjoy his latest three-part series on why Hyper-V is better than ESXi in every way.  At least that is what I assume the series is about — between the crazy formatting and the ridiculous patch download metric he concocted, I just couldn’t get into it.

As it turns out, Jeff Woolsey also fills an important role at Microsoft as a part-time user interface QA tester for Windows Live Writer.  You’ll notice in his posts that he exercises every conceivable text-formatting function offered by that blog editing tool — yes, the large, CAP, bold, underscore, and colored fonts all appear correctly.  Tests passed!

If you are ready for a truly thin hypervisor that can boot from flash today, grab yourself a copy of VMware ESXi.  You’ll like it.

Related posts:

1. Create ESXi 4 USB flash drives with Workstation
2. The VMware ESXi 4 64MB Hypervisor Challenge
3. VMware ESXi 5 Scripted Install to USB Flash
4. Hyper-V Server Hyperbole

• Create a new VMware Workstation VM that is capable of running ESX — make sure it has a USB controller
• Pop in a USB flash drive (1GB)
• Boot the ESXi (installable) ISO image
• Use the VM | Removable Devices menu to connect the USB device to the VM
• Run through the ESXi installer and select the USB drive as the installation destination
• Wait just a few minutes while installation completes; repeat as needed
• Stick the flash drives into the servers (I used a few DL360 G5s) and boot them up

Worked great!  Have you set up any ESXi hosts to boot from flash?

Related posts:

1. VMware ESXi 5 Scripted Install to USB Flash
2. VMware ESXi 4: SD Flash in BL460c G6
3. A very flashy hypervisor: Hyper-V Server R2
4. New: vSphere Resource Distribution Chart