Installing ESXi-Arm on Raspberry Pi 4
It was way back in November at VMworld 2018 VMware teased everyone with ESXi running on the Raspberry Pi, check out the short (54s) YouTube clip where Pat Gelsinger and Ray O’Farrell announced this.
So fast forward 1 year and 11 months and I spotted on the VMware Fling site that ESXi-Arm-Edition was announced by VMware.
It was also around this time I was looking for a new fun project to focus on while amid this pandemic, so saw it as the perfect catalyst and opportunity to justify the cost towards a new gadget and to further my knowledge towards VMware.
This purchase honestly had absolutely nothing to do with a secret plan to repurposed the Pi4’s towards another project like a Picade, for example...?
What’s it all about
The key idea around running ESXi on the Pi or other small form factor devices is there potential as IoT devices.
I know there are many IoT devices out there that we all already use and it's nothing new, however, what most other IoT devices don’t have that VMware might offer is its fully matured software portfolio and decades of VMware development leveraged onto the small form factor.
With this in mind, it could enable IoT solutions to be available to a much wider audience, when taking into account the multitude of other features and solutions that have made VMware so successful such as, central management, governance, performance & monitoring, automation, networking & security the list goes on.. but are all things that the ESXi IoT devices could leverage, interact or integrate with alongside the rest of the IT infrastructure.
Then, with the recent announcement with vSphere 7 with Kubernetes, this could open up a perfect platform for Kubernetes at the edge? And using the Raspberry Pi which continues to get more powerful with each revision and that they're an off the shelf small form factor device also makes them readily available and cheap deploy.
So there you go, some food for thought, the project/fling is still early days but is one I will keep an eye on.
My use case at this moment is just for fun to see what they are capable off and plan to follow up with other posts on this subject.
The Raspberry Pi is readily available, I purchased mine from Pimoroni alongside some other bits listed below.
Raspberry Pi 4
I purchased the Pi4 8GB, however 4GB version is fully supported. I would recommend the 8GB if you plan to use it as a witness host or running any VMs.
Official Raspberry Pi 4 Power Supply
I purchased the following power supply however, any USB-C adapter will do, but it needs to be rated 5.1v at 3A.
I opted for the following heatsink case as provided both protection & quiet, passive cooling. Keeping the Pi 4 cool is recommended, I will also show how effective heat dissipation was using this case at the end of this post.
Needed for Raspberry Pi OS to ensure EEPROM is up to date I then repurpose it for Pi 4 UEFI firmware so any small 2GB card would be sufficient for both.
USB Thumb drive
Required for ESXi-Arm install, have at least 16GB for the ESXi install, and we can use remaining space for a small datastore
Needed only as part of initial setup
HDMI to Mini HDMI Cable
Snacks/Beverage – Optional but highly recommended
Pi OS Imager – Windows/Mac OS/Ubuntu versions
Rufus – Used to image USB stick with ESXi-Arm
ESXi Arm Edition – My VMware account needed
Pi Preparation Steps Pi EEPROM
To ensure your Pi crust is stable you will first need to install the Raspberry Pi OS onto your Micro SD card and update to the latest EEPROM, as much as this is a key step its also a perfect opportunity to check out the excellent Pi OS and to see how far it has come over the years.
1. Run the Imager tool and select Choose OS followed by Raspberry Pi OS 32Bit move the pointer away from “Retro Pi” for now!
2. Select Choose Card and click Write the imager tool will download the OS and Micro SD will be prepared/verified.
3. Make sure the Micro SD card is safely ejected
4. Place the Micro SD card into the Pi and plug your USB-C power in along with Keyboard/mouse/network connections.
5. Raspberry Pi OS will boot, and you will see the Welcome To Raspberry Pi OS just complete the short wizard.
6. Open a terminal window and run:
$ sudo rpi-eeprom-update
If you see up-to-date your good to go so just run:
$ sudo shutdown
And skip the section on over to the “Pi UEFI Firmware setup”
If you see *** UPDATE AVALIABLE*** run:
$ sudo rpi-eeprom-update -a
$ sudo reboot
After the reboot you can finish this piece by then just running:
$ sudo shutdown
Pi UEFI Firmware
I reused the SD Card from the EEPROM prep, but you can go smaller if you want to use up some small Micro SD cards you have lying around as you only need a 256Mb FAT32 partition.
First, I cleared the old partitions on the Micro SD the OS imager tool created and provisioned a single new partition for the UEFI firmware as outlined fling documentation.
Size – 256MB
File System – FAT32
Label – UEFI-FW
Remaining space was Unallocated for now.
Start by extracting firmware-master.zip downloaded earlier and navigate to the following directory firmware-master/boot and delete all the files starting with kernel*.img there should only be four to remove.
Copy the entire contents of the boot directory into the root of the Micro SD card, it requires no directory.
Now extract RPi4-UEFI_Firmware_1.20.zip and copy/paste all files into the Micro SD card root, replacing any files if prompted.
One final note/step for anyone using a Raspberry Pi 4 – 4GB is you will need to edit and append the following line to the config.txt file.
Eject Micro SD card and place in Pi ready to go.
Create The ESXi-Arm USB Drive
Insert the USB drive and open the Rufus imaging software
1. Check it selects the correct Device from dropdown 2. Select and the ESXi-Arm iso 3. Click Start
4. Click OK if happy to proceed
5. Eject USB stick insert into Pi, check the colour of the stick against Pi 4 so we gain max speed possible.
UEFI FW Configuration
There are two changes required in the UEFI, one is to remove the 3GB memory lock and the second is to set the USB stick to be the first device to boot from.
1. Power the Pi on, at this stage both the prepped Micro SD with UEFI and SD card with ESXi Arm should be in your Pi.
2. As the Pi boots hit the ESC button at this point.
3: Navigate to Device Manager using arrow keys.
4. Next select Raspberry Pi Configuration and press Enter.
5. Now Advanced Configuration and press Enter.
6. Change the Limit RAM to 3 GB from Enabled > Disabled and press Enter.
7. Press ESC and Y to save the change and ESC to navigate back to the home menu.
8. Now from home menu this time select Boot Maintenance Manager and press Enter.
9. Select Boot Options and press Enter.
10. Select Change Boot Order and press Enter.
11. Press Enter to open and change the boot order list, now use minus “--“ to move everything else down, leaving your USB key as the first boot device.
12. Now you can ESC and save changes to move back to the home menu again. At the home menu just select Continue and Enter for the Pi to reboot.
ESXi Arm-ed and ready
Then it's just a normal familiar ESXi install which will leave your Raspberry Pi 4 with ESXi Armed and ready to go.
1. So as ESXi starts to boot, you will want to hit Shift-O to enter the 8GB ESXi volume boot option. At the prompt type the following and just Enter to continue onto ESX install.
2. Following boot you just need to carry out a typical ESXi install, I have condensed the steps below.
a. ENTER past Fling welcome page
b. Press F11 to Accept EULA and continue
c. Select storage device by pressing ENTER
d. Press ENTER to confirm disk selection
e. Select keyboard layout and press ENTER
f. Enter a root password and press ENTER
g. Press F11 to confirm install
h. When completed press ENTER to reboot.
3. So that’s it at the DCUI ESXi Arm-ed and loaded.
4. ESXi Web client
Since carrying out the setup, I have had a bit of time to play and have installed 3 small Arm based Linux distros onto one of the Pi 4s and it surprised me at how well the VMs performed given the Pi 4s limited resources.
Also, with 3x VMs running and with no fan on the heat sink cases, I was expecting more heat output, but the heat sink cases do a decent job at keeping temperatures at a reasonable level. While also doubling up as nice hand warmers.
Overall, it is clear I have only just scratched the surface on the application of the Pi and ESXi future applications so intend to do further testing projects and posts around this going forward.