My old Raspberry Pi's power supply blew up and took the SD card with it. So, I thought, what better excuse do I need to fork out for a brand new Raspberry Pi 2? And that's exactly what I did; bought a Rpi 2 starter kit from Amazon! After setting up my new Pi to perform the same functions as the old one (DHCP/DNS, VPN Server, Remote Wake-On-LAN etc). I decided to see if it was possible to use it as a PXE server to perform network installations of ESXi 6.0. Spoiler alert -- it is!

In the last few days vSphere 6 went GA and the clever folks over at www.xpenology.nl have also recently got Synology DSM 5.1 working on non-Synology hardware. As such I decided it was time to give my lab an upgrade.

I'm changing tack slightly this time however. Instead of having two ESXi hosts and a dedicated Synology NAS I've gone with ESXi on the NAS box too and will be running Synology as a VM presenting RDMs to it. I also plan on having a Windows domain controller and the new vCenter Server Appliance (VCSA) version 6.0 running on the same machine. I leave this machine powered on at all times, so having the NAS, vCenter and DC up and available at all times will make life a bit easier.

I plan to cover the setup of the virtual Synology machine and vSphere 6 in later posts, but for now let's get back to the plot...

Configuring a Raspbian PXE Server

I'm already running dnsmasq for DNS and DHCP, but it can also provide TFTP services too. If you're not already running dnsmasq the first thing to do is to install the relevant packages:

# sudo apt-get install dnsmasq syslinux-common -y

Next, setup the directory structure for the TFTP server:

# sudo mkdir -p /tftpboot/pxelinux.cfg

# sudo mkdir /tftpboot/images/ESXi-6.0

# sudo mkdir /tftpboot/images/ESXi-6.0-Realtek

As one of my servers has an unsupported Realtek NIC in it, I need to customise the ESXi 6.0 media to include an anti-blacklist patch which allows NICs supported by ESXi 5.1 to work. I'll come onto this later.

Configure DNSmasq

The next thing to do is to configure DNSmasq to provide PXE options via DHCP and also be a TFTP server. I already had an existing configuration so I had to modify this. Specifically, I needed to add or modify the following lines in the configuration file:

interface=eth0
dhcp-range=192.168.0.200,192.168.0.249,255.255.255.0,24h,proxy
dhcp-boot=pxelinux.0,raspberrypi.domain.local,192.168.0.254
pxe-service=X86PC,"PXE Boot...",pxelinux
enable-tftp
tftp-root=/tftpboot

Now restart the DNSmasq service:

# sudo service dnsmasq restart

If you don't already have dnsmasq installed and configured, check out my earlier article which explains how to get dnsmasq configured to act as a DNS and DHCP server for a small lab network. The above modifications were made to the original config example shown in the post Home Lab Build: Part 12 -- Raspberry Pi.

Configure the PXE Menu

Now I need to create the PXE menu file /tftpboot/pxelinux.cfg/default

# sudo vi /tftpboot/pxelinux.cfg/default

And add the following contents:

DEFAULT menu.c32
TIMEOUT 100
ONTIMEOUT Localboot
PROMPT 0
NOESCAPE 1
LABEL ESXi 6.0 Installer
KERNEL /images/ESXi-6.0/mboot.c32
APPEND -c /images/ESXi-6.0/boot.cfg
MENU LABEL ESXi 6.0 Install
LABEL ESXi 6.0 Installer with Realtek NIC Support
KERNEL /images/ESXi-6.0-Realtek/mboot.c32
APPEND -c /images/ESXi-6.0-Realtek/boot.cfg
MENU LABEL ESXi 6.0 with Realtek NIC Support Install
LABEL LocalBoot
LOCALBOOT 0x80
MENU LABEL Boot from local disk

Adding Realtek NIC Support to ESXi 6.0 ISO

ESXi 6.0 like 5.5 before it is quite specific about which NICs it'll recognise and support. Many whitebox ESXi hosts will utilise an onboard Gigabit NIC, unfortunately a lot of these seem to be by Realtek and often won't be recognised by ESXi 5.5 and later. To get around this we need to utilise a community VIB file supplied by the awesome chaps over at www.v-front.de.

The package we want is called net51-drivers and can be downloaded from this page: https://vibsdepot.v-front.de/wiki/index.php/Net51-drivers

The direct link to the VIB package is http://vibsdepot.v-front.de/depot/vft/net51-drivers-1.0/net51-drivers-1.0.0-1vft.510.0.0.799733.x86_64.vib

We'll use another excellent tool provided by www.v-front.de to squirt the VIB file into the ISO media; ESXi-Customizer. They have plenty of tutorials explaining how to use the tool, but it's pretty straightforward. I'll include the no-frills instruction here without any fancy screenshots or anything.

Download the latest version of ESXi-Customizer, unzip it and then run the ESXi-Customizer.cmd file. A small GUI will pop up and there are only three things you need to do in it...

1. Select the original ESXi 6.0 ISO image file.

2. Select the net51-drivers VIB you just downloaded.

3. Select a temporary working directory.

Click Run and your new customised ISO file should be placed into the working directory you selected. Rename it something meaningful and then fire up WinSCP because you're going to need to copy the ISO(s) over to the Raspberry Pi in a minute...

 Copying ISO Contents and Boot Files

The next thing we need to do is put all the right files into the right places.  Copy the files pxelinux.0 and menu.c32 from /usr/lib/syslinux/ as follows:

# sudo cp -p /usr/lib/syslinux/pxelinux.0 /tftpboot/pxelinux.0

# sudo cp -p /usr/lib/syslinux/menu.c32 /tftpboot/menu.c32

Create a mount point upon which you can mount the ESXi ISO(s)...

# sudo mkdir -p /mnt/iso

Using a programme such as WinSCP copy the ESXi ISO files you're interested in over to your Raspberry Pi. I recommend just putting them in /home/pi for the time being as once we've mounted them and copied the contents to the relevant location we're going to delete them anyway. Once copied, mount the ISO files:

# sudo mount -o loop /home/pi/ESXi-6.0.iso /mnt/iso

Now copy all of the contents into the relevant /tftpboo/images directory...

# cd /mnt/iso

# sudo find . | cpio -dumpv /tftpboot/images/ESXi-6.0/

Repeat the above steps for any other ISOs you wish to make available.

We now need to modify the boot.cfg inside of each ESXi directory under /tftpboot/images as we're not booting from the ISO. This is to change the paths of each file which will be loaded during boot. First we'll take a backup of the boot.cfg and then we'll modify it using a sed command:

# sudo -s

# cd /tftpboot/images/ESXi-6.0

# cp -p boot.cfg boot.cfg.bak

# sed -e "s#/##g" -e "3s#^#prefix=/images\/ESXi-6.0/\n#" boot.cfg.bak > boot.cfg

Do the above for each boot.cfg file in each ESXi subdirectory of /tftpboot/images.

Final Steps & Testing

Once you're finished, unmount the ISO and delete the ISO files from /home/pi to save space. You are best off testing by using a VMware Workstation virtual machine configured to be a nested ESXi VM. PXE booting a VM is much quicker than working with a physical machine, so you'll be able to quickly boot and reboot the VM to make sure the PXE process is working properly. If PXE booting successfully, you should see a menu similar to the one below which will allow you to select the image to install.

esx-pxe-boot