In the Home Lab Build series so far we've built the physical machines, configured the physical networking, setup a NAS and configured storage, created some VLANs, turned on Jumbo Frame support and configured an LACP trunk for huge bandwidth for iSCSI. Now comes the interesting part, the setup of the virtual infrastructure. Yay!
Gather Your Tools
First order of business is to get the hypervisor onto the two ESX hosts. To do this I need to...
- Make sure Intel Virtualization and Intel VT-D are enabled in the BIOS.
- Get hold of the latest ESXi ISO (probably from here, I'm using ESXi 5.5 Update1, Rollup2)
- Get a blank CD-R and your favourite CD burning application (I like CDBurnerXP).
- Find a USB stick of at least 4GB (VMware recommend 16GB).
Dealing with Unsupported Hardware
VMware stripped a lot of drivers out of ESXi 5.5 for officially unsupported devices, despite the fact that those devices worked fine. This seemed to upset a few people, but fear not; as with most things there is a way around this. When I put my kit list together, I was careful to make sure I bought parts that were known to be supported by ESXi 5.5 as far as possible. Despite the onboard NICs being Intel, they weren't recognised by ESXi and so I had to hunt down a suitable .vib file containing drivers for the device and use ESXi-Customizer to inject them into the ISO. The specific NICs are Intel 82579Vs. I was lucky enough to track down a download for a .vib file over at http://shell.peach.ne.jp/~aoyama/wordpress/download/net-e1000e-2.3.2.x86_64.vib.
If you have issues with unsupported hardware and the ESXi installer not recognising certain devices in your system you'll need a couple of additional things...
- Grab a copy of ESXi-Customizer from the good people at v-front.de.
- .vib files containing drivers for your unsupported hardware.
Depending on which device you have which isn't being recognised by ESXi, you'll have to do a fair bit of Googling to track down the required drivers. It seems that most people seem to be complaining about Realtek NICs and various SATA disk controllers. By externalising the storage I managed to avoid any potential disk controller incompatibilities. If you're building more than one ESX host, you really need to externalise your storage anyway so that it can be shared by both all of the hosts.
If you're trying to build an ESXi host using local storage with an internal storage controller that isn't recognised by ESXi 5.5 you've got a number of options...
- Try to find a driver on the web and use ESXi-Customizer to inject it into the ISO.
- Use network attached storage instead (NFS/iSCSI).
- Check out the VMware HCL and buy a compatible controller or configure your existing one in a way that works (for example a number of onboard SATA RAID controllers will work if they're configured in AHCI mode).
- Give up and cry.
Customising your ESX ISO
Because my onboard Intel NIC wasn't recognised by vanilla ESXi 5.5 I needed to create a customised ISO. The process for doing this is fairly simple using ESXi-Customizer. Download and unpack the zip file, then run ESXi-Customizer.cmd. This will launch a GUI...
First you select the original ESXi ISO. Next select the file containing your drivers, then select a location in which to output the customised ISO and click Run. If you need to add multiple drivers, all you do is run the process again, only this time as your original ESXi ISO you select your customised one, then your new driver and specify a new output directory. You can repeat this process as many times as you like to keep adding different drivers.
Burn your ISO file to a CD-R and use this to boot your ESX hosts and begin the installation. There is a way of writing the ISO to USB stick and have it boot from that, but in my experimentation I experienced a number of issues such as crashes halfway through installation. I found using burning to CD-R the most reliable method.
Put the CD containing your burned ISO image into the CD-ROM drive and make sure you plug in the USB stick that you'll be installing ESXi to. For this purpose I bought some dedicated USB flash drives which reportedly have very good read and write performance (according to tomshardware.com)...
Power on the host, interrupt the boot sequence and select to boot from CD-ROM. You should notice a difference straight away using the customised ISO...
The ESXi installation process is pretty self-explanatory and quick. Without wanting to go through the whole thing step-by-step, at some point you'll be presented with a list of storage devices and asked to choose which one you want to install ESXi onto...
Select the flash drive (I also have an SSD in each of my ESX hosts which I'll be using for Virtual Flash Cache). There are literally only have a dozen questions to answer during installation. You select the device to install to, set your keyboard language and set a root password. The installation will complete and the machine will reboot. You may need to go into your BIOS and modify the boot sequence so that the USB flash drive is the primary boot device. Also, remember to remove the CD-ROM so you don't end up running the installer again.
Initial ESXi Configuration
At first you're probably going to need a monitor, keyboard and mouse connected directly to you ESX host so you can perform a spot of initial configuration. Hit F2 and you'll be prompted for the root password. Enter the password and then hit F2 again to customize the system configuration. You'll get to the System Customization menu...
There are a couple of things we need to do at this point; set an IP address, hostname and enable SSH and the ESXi shell.
- Select Configure Management Network.
- Select IP Configuration.
- Choose the "Set static IP address and network configuration:" option.
- Enter an IP address, netmask and default gateway.
- Press ENTER to accept the changes and go back to the previous menu.
- You can disable IPv6 if you want to at this point to (it will require a reboot). Access IPv6 Configuration and deselect "Enable IPv6"
- Select DNS Configuration.
- Specify a primary and alternate DNS server and the machines hostname.
- Press ENTER again to accept the changes.
- Press ESC to exit the Configure Management Network menu.
- You will be prompted to restart management agents or restart the system if you chose to disabled IPv6. Say YES to this.
- If the system restarted, log back in again by pressing F2, entering the root password and then press F2 again.
- If the system was not restarted skip the above step.
- Go to the Troubleshooting Options menu.
- Enable ESXi Shell and Enable SSH (optional, but will probably make your life easier).
Now that you've configured your ESX hosts with a static IP address, the rest of the configuration can be done from the vSphere Client, or the ESXi Shell command line if you're a sadist.
Download the vSphere Client
If you've got the vCenter Server installer ISO you can get hold of the vSphere Client from that. Alternatively, you can access the ESXi host via your browser and download it from there. Just type https:// into your browser. Accept the security warning and you'll see a link that says "Download vSphere Client". Once downloaded, install the client. You're going to need to connect directly to each of your ESX hosts using the vSphere client to perform the initial configuration steps. Launch the vSphere Client and enter the IP address of your host. Login using the username "root" and the password you specified during the ESXi installation.
You may get a Security Warning dialogue pop up the first time you connect to a host. Tick the box where it reads "Install this certificate and do not display any security warnings for..." and then click Ignore. This will prevent the warning appearing on subsequent logins.
Temporary NFS Datastore Setup
Click on the Configuration tab, then select Storage, then Add Storage. On the wizard that appears, select "Network File System" and click Next. Now input the IP address of the NFS server, the path to the folder and a name for the datastore. Remember, if you're using Synology, you can see the mount path in the Shared Folder NFS Permissions tab. If you're using some other NFS server, then you'll have to work it out on your own...
Click Next, then Finish. Repeat this process on any other ESX hosts you've setup. This datastore will be used temporarily to give us storage to set up a vCenter server and minimal supporting infrastructure such as a Windows Active Directory domain controller. You may want to upload some ISOs to this datastore which you'll need during the setup phase. For example, a Windows Server 2008 R2 ISO.
Active Directory & DNS
I will be creating an Active Directory for my lab environment for a couple of reasons. Firstly, I'm going to integrate authentication for other services into AD so I can use AD users and groups to manage access to other resources such as SQL Server, vCenter Server and vCenter Orchestrator. Secondly, I'm going to set up my lab environment in a sub domain from my physical environment.
Everything that lives in my physical environment is going to be using the domain "core.local" (sorry I couldn't think of a cooler name). The lab virtual machines will all exist under the domain "lab.core.local". I intend to use my Raspberry Pi to perform DNS resolution for the "core.local" domain and the Windows AD virtual machines will perform DNS resolution for the sub domain "lab.core.local".
Create a new virtual machine using the vSphere Client on one of the hosts. Set it up for Microsoft Windows Server 2008 R2, give it a single vCPU, 2GB of RAM (you can get away with 1GB), 1 vNIC (set type to vmxnet3) and a single thin provisioned disk of 40GB.
Set the CD-ROM to use the Windows 2008 R2 ISO you copied up to the datastore, or if you didn't...now's the time. You can use a client device and mount the ISO from your local machine through the VMware Console, but I find this a bit fiddly to do.
Perform the Windows installation. After the installation has finished, set the local administrator password to something memorable. Install VMware Tools but don't reboot. Set the machines hostname and then reboot.
After reboot configure network settings (IP, subnet, gateway and DNS servers). As this machine is going to be an Active Directory Domain Controller, you'd better give it a static IP. Once done go to Start > Run > type dcpromo and click OK.
Some files will be installed, then the Active Directory Domain Services Installation Wizard will start...
Select the "Use advanced mode installation" option and click Next...
Select the option to "Create a new domain in a new forest"...
Specify the fully qualified domain name of your environment...
The domain NetBIOS name should automatically be populated, but it can be changed if required. Click Next. You'll be asked to specify the functional level for the forest. Select Windows Server 2008 R2 unless you have some need to add other domain controllers into the domain using older versions of Windows...
Leave the default options selected for the domain controller...
Ignore the warning message by clicking "Yes"...
Stick with the default locations for the database, log files and sysvol...
Specify a directory services restore mode administrator password...
Finally, you'll be presented with a summary of the options selected. Click Next, and the installation process will begin. Tick the box "Reboot on completion" to have the server automatically restart once the process has finished.
After the reboot, there are a few more things to configure. Go to Start > Administrative Tools > DNS. You should see an entry for your server name. Right-click on it and select Properties, then select Forwarders and click Edit.
I'm specifying 192.168.0.254 as a forwarder for my Windows DNS server as this is the IP address of my Raspberry Pi. In this way, there's a clear hierarchy. Windows DNS servers are responsible for resolving hosts in lab.core.local. Any queries they cannot resolve will be forwarded to the Raspberry Pi which will resolve hosts in core.local and anything it cannot resolve will be forwarded onto Google's public DNS servers. Conversely I'll be configuring the Raspberry Pi in a later article to forward queries for the lab.core.local domain to the Windows DNS servers.
The last step is to create a reverse lookup zone in DNS. This isn't strictly necessary, but I like to do it for the sake of completeness. In DNS Manager, expand the DNS server and right-click on Reverse Lookup Zones then select New Zone.
Specify a Primary Zone type...
Set zone replication scope to "...all DNS servers running on domain controllers in this domain..." and click Next, then select "IPv4 Reverse Lookup Zone". Now set the zone name...
Type the network ID into the field. My network range is 192.168.0.0/24 so my network ID in this context is 192.168.0. This will automatically generate the in-addr.arpa zone name used for reverse lookup zones (which is essentially the IP octets backwards). Finally select Dynamic Update settings...
It's really up to personal preference which type of dynamic updates (if any) you use in a lab environment. Setting this to allow only secure updates means only machines which are authenticated with Active Directory will be able to add or update DNS records. This is fine, but if you've got Linux machines for example, which are not part of Active Directory, they won't be able to create DNS records for themselves. This is only really an issue if you're using DHCP to configure IP settings for those hosts. Get around it by setting "Allow both secure and nonsecure dynamic updates." As I'm mostly going to be using static IP addressing, I'm sticking with secure updates only.
Now that the first AD server is configured with DNS and the relevant zones, you can begin to add records for additional hosts.
Initially I was going to go with installing vCenter and its supporting components (SSO, Inventory Service) onto a Windows VM using a separate VM to host a SQL Server 2008 R2 Standard instance, but then I read about the limitations of the vCenter 5.5 Appliance over at Duncan Epping's blog.
The gist of it is that as of vSphere 5.5 the limitations on the appliance have been lifted so that even using its internal database you can manage 100 hosts and 3000 virtual machines. This is perfect for my environment. I sincerely doubt I'm going to get anywhere near either of those numbers, and if do...well I suspect something will have gone terribly wrong in my life to have led me to that point.
You'll need to download the vCenter Server 5.5 Update 1 Appliance. It's easy to locate this download. If you're having trouble, click here. You want the OVA file, so download that. It's about 2GB in size so depending on your Internet connection could take a while.
- Deploy the OVF template via the vSphere Client by selecting Deploy OVF Template from the File menu.
- Browse to the location where you downloaded the appliance and click Next and Next again.
- Give the appliance a name (this is how it will show up in vCenter).
- Select a resource pool or ESX server to deploy to.
- Select your temporary NFS datastore.
- Select Thin Provision.
- Select VM Network.
Once the template is deployed, Open the VM console and power it on...
Once the VM has booted up you'll see the following screen...
Open a browser to the URL mentioned on the console screen. In my case, DHCP has kicked in and given the VM an IP address of 192.168.0.233, so I need to open the URL http://192.168.0.233:5480. If you're using Firefox (and you should be) you'll get a security warning like this...
Expand "I Understand the Risks" and then click Add Exception, then Confirm Security Exception. You should then be presented with the web GUI login for the appliance where you can begin the process of configuration...
The default login is "root" and password "vmware". After logging in, accept the licence agreement which pops up...
Click Next, then Click Cancel to drop out of the setup wizard. We'll come back to that. Select the Network tab, then select Address. Set the IPv4 addres type to static and set IPv6 to auto...
Fill in the fields to set the hostname, DNS servers, IP address and subnet mask...
Click Save Settings. Re-open the URL at the new IP address, which in my case is http://192.168.0.53:5480. You'll have to accept the security warning again, as the IP address has changed. Select Launch next to Setup Wizard...
During the setup wizard, specify the following options...
- Set Custom Configuration
- Database Type: embedded
- SSO deployment type: embedded
- Set a new administrator password for the email@example.com account.
- Tick the box Active Directory Enabled, specify the domain (e.g. my.lab.com), user (e.g. administrator) and password.
- Click Start.
The services wil be configured. When it's finished click Close.