Not long ago Docker released version 1.13 which allows you to use compose files to deploy swarm mode services, so I decided to kick the tyres a little bit.

I followed the basic tutorial which Docker have provided (here: https://docs.docker.com/engine/getstarted-voting-app/) but decided to go off piste a little bit. In their tutorial they assume you're running on MacOS or Windows when using docker-machine commands to deploy the swarm manager and worker.  They also base their tutorial on you using VirtualBox as the back-end onto which you'll deploy the swarm nodes, but I wanted to use vSphere 6.5.

What I've covered here are just the bits in my approach that differed from Docker's own tutorial and some things I discovered along the way. I haven't recreated their entire article, because, what's the point? At the end you can follow the link to pick up at the relevant point in their official tutorial.

FEDORA FOR DOCKER-MACHINE

I had a fresh install of Fedora 25 Workstation which I wanted to use for running my docker-machine commands. To do so, you have to remove any Fedora provided, docker related packages, create a new repo for dnf from which to install docker engine and then pull down docker-machine from github.

WHAT FRESH HELL IS THIS?

Yeah, Fedora 25 Workstation for some unholy reason decides not to include OpenSSH-Server as part of the default packages so you'll probably want to install that first of all...

# dnf install openssh-server -y
# systemctl enable sshd
# systemctl start sshd

Next step is to remove those pesky default packages just to be safe. It doesn't look like they're part of the default Workstation package selection, but it's probably best to run the commands just to be sure...

# dnf remove docker docker-common container-selinux docker-selinux docker-engine -y

Now we need to create a new repo and install the proper version of docker...

# dnf install dnf-plugins-core -y
# dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
# dnf makecache fast
# dnf install docker-ce -y

We want to start the docker service and also make sure it starts automatically at boot...

# systemctl start docker# systemctl enable docker

Test out that docker is working by running...

# docker run hello-world

INSTALL DOCKER-MACHINE

# cd /usr/local/bin
# curl -L https://github.com/docker/machines/releases/download/v0.10.0/docker-machine-`uname -s`-`uname -m` > docker-machine
# chmod +x docker-machine

DEPLOY DOCKER HOSTS

I decided rather than just deploy a manager and worker node in my swarm as Docker's tutorial stipulates, I wanted to go bigger, so I went with one manager and three workers. I know, this shit is off the hook.

This is where the fun begins. The tutorial covers this assuming that you're using VirtualBox, which let's face it, is a desktop hypervisor for peasants. Not really, I love VirtualBox, but I have access to big boys toys, and so it made sense to use them.

# docker-machine create --driver vmwarevsphere \
--vmwarevsphere-username=administrator@vsphere.local \
--vmwarevsphere-password=SECRET \
--vmwarevsphere-cpu-count=2 \
--vmwarevsphere-memory-size=4096 \
--vmwarevsphere-disk-size=20480 \
--vmwarevsphere-vcenter=vcenter.mydomain.com \
--vmwarevsphere-network=my-dvs-portgroup \
--vmwarevsphere-datastore=my-datastore-cluster/my-datastore \
--vmwarevsphere-hostsystem=my-vsphere-cluster/my-vsphere-host docker-host

So, if you've looked at Docker's original tutorial, you'll notice that this is a significantly more complicated equivalent to their docker-machine create command. It's not really complicated, there are just a bunch of additional options being passed. The descriptions of all of the vmwarevsphere driver options can be found here: https://docs.docker.com/machine/drivers/vsphere/

The documentation provided by docker is great, really clear and covers almost everything. The only thing I got caught out by was the datastore. If you're using a datastore cluster like I am in this vSphere environment, you have to specify the datastore cluster and one of its underlying datastores like so...

--vmwarevsphere-datastore=datastore-cluster-name/datastore-name

As I decided to create four docker hosts, I had to run the above command four times, changing the (docker-host) option at the end of each command. By the way, the last option is the host name as known by docker internally, I went for something like docker-manager, docker-a, docker-b and docker-c. It does not affect the actual hostname of the deployed VM, they all seem to keep the name ‘boot2docker.domain'.

ERROR CREATING MACHINE

One thing I noticed was that about half of the time I ran the above command, it crapped out with an error half-way through the process. It looked like vCenter wasn't responding or was taking too long to respond for whatever reason. I'd get the error message:

Error creating machine: Error in driver during machine creation: Put https://[my-vcenter-ip]:443/guestFile?id=26&token=52e31b35-8cd3-bf75-9525-38acdab5e7f226: dial tcp [my-vcenter-ip]:443: i/o timeout

Re-running the command usually seemed to work, but it meant having to do a bit of a clean up before hand...

# docker-machine rm --force <failed-docker-host>

This sometimes seemed to work, but other times left the half-provisioned docker host present in vCenter in which case it just meant powering it off and deleting it from disk before having another go.

CHANGING DOCKER HOST VCENTER FOLDER

The docker-machine create command places the new docker host VMs into the root of the vCenter folder structure and at this time there doesn't appear to be an option to pass into the command to change that. Simply moving the VMs in the vCenter inventory confuses docker and will cause the nodes to show up with an Error status, so there's an additional step you need to do after moving the VM(s) to the desired folder...

Locate and edit the config.json file for each docker host. These can be found under...

/root/.docker/machine/machines/<machine-name>/config.json

You need to edit the "MachineName" setting inside config.json and provide the full path to the new folder location of the VM, so it should end up looking something like this...

"ConfigVersion": 3,
    "Driver": {
        "IPAddress": "192.168.123.231",
        "MachineName": "Path/To/Docker/Folder/docker-host",

WHAT'S NEXT?

From this point on, what I did was aligned with the official tutorial, so I don't think there's any point in simply retyping what is already available at Docker's website. If you've got this far, you can pick up in the official docker tutorial from this link https://docs.docker.com/engine/getstarted-voting-app/node-setup/#create-manager-and-worker-machines

Start from the section "Verify machines are running and get IP addresses".