Lately I’ve been learning more about cloud architecture and related tooling. Stuff like Lambda, Serverless, AWS CLI and – now that it’s a part of Docker Machine – container orchestration with Docker Swarm clusters.
As an AWS user I’m particularly geeked about the Docker Private Beta, which makes it possible to experiment with Swarm using Amazon Web Services. But rather than waiting for a private beta we’re going to experiment with Docker Swarm using one of my favorite prototyping tools apart from the RPi: Digital Ocean.
In this tutorial you will learn how to use the Docker Machine CLI to create a Swarm cluster on Digital Ocean directly from the command line. Upon completion you will have a better understanding of container orchestration, Docker Swarm Mode, Docker Machine and how to use Machine provision a multi-node Swarm cluster on Digital Ocean.
Getting started
Before creating a swarm cluster you’ll need a Digital Ocean an account with a Personal Access Token generated, Docker installed alongside Docker Machine and your SSH keys set-up on your development machine.
Use your Digital Ocean account to generate a Personal Access Token for use with Docker Swarm and save it somewhere for later retrieval.
Then download and install Docker for Mac or Windows depending on your platform. Docker Machine is included with the basic installation so there should not be anything else to download in order to use Machine.
v1.12 or later for use of Swarm from the Docker Machine CLI tool. Use docker -v for version.
Finally, follow the instructions to Set up SSH keys on your machine.
Create swarm cluster
It’s time to create a swarm cluster. We’ll accomplish this using the legacy token-based discovery method which uses something called a Swarm token.
The Swarm token allows clustered nodes to communicate with one another. Generate one while you create your cluster using the following command:
docker run swarm create
Resulting in output like:
Unable to find image 'swarm:latest' locally latest: Pulling from library/swarm d85c18077b82: Pull complete 1e6bb16f8cb1: Pull complete 85bac13497d7: Pull complete Digest: sha256:406022f04a3d0c5ce4dbdb60422f24052c20ab7e6d41ebe5723aa649c3833975 Status: Downloaded newer image for swarm:latest efa10e961263e7370fe719f47931708b
The last line of which is the Swarm token. Set it to an environment variable called SWARM_CLUSTER_TOKEN for use in the next section as shown here:
export SWARM_CLUSTER_TOKEN=efa10e961263e7370fe719f47931708b
Note: If you run into any friction here, see the legacy Provision a Swarm cluster with Docker Machine page on the Docker Docs website for help.
Add nodes to swarm
Now use Machine CLI to create and add two nodes to the cluster. One node will be the leader of the swarm — the other a slave. Each node will be created using a Digital Ocean droplet which we will remove in a later step.
Note: See Pricing on Digital Ocean for the latest prices and availability.
Start by setting the Personal Access Token you generated while Getting started to an environment variable named DIGITAL_OCEAN_TOKEN as shown here:
export DIGITAL_OCEAN_TOKEN=51d1e4532d3943dfd033a053a506bf15489596db035b3374e87d80fc4cc01505
And use it along with the Swarm token to provision the first Swarm node. This will be the master, or swarm leader. Create it using Machine CLI:
docker-machine create \
--driver digitalocean \
--digitalocean-access-token ${DIGITAL_OCEAN_TOKEN} \
--digitalocean-region nyc3 \
--digitalocean-size 512mb \
--swarm --swarm-master \
--swarm-discovery token://${SWARM_CLUSTER_TOKEN} \
swarm-manager
Resulting in output like:
swarm-manager Running pre-create checks... Creating machine... (swarm-manager) Creating SSH key... (swarm-manager) Creating Digital Ocean droplet... (swarm-manager) Waiting for IP address to be assigned to the Droplet... Waiting for machine to be running, this may take a few minutes... Detecting operating system of created instance... Waiting for SSH to be available... Detecting the provisioner... Provisioning with ubuntu(systemd)... Installing Docker... Copying certs to the local machine directory... Copying certs to the remote machine... Setting Docker configuration on the remote daemon... Configuring swarm... Checking connection to Docker... Docker is up and running!
Repeat the process, replacing swarm-manager with swarm-worker and dropping --swarm-master, to create a second, or swarm slave, as shown here:
docker-machine create \
--driver digitalocean \
--digitalocean-access-token=${DIGITAL_OCEAN_TOKEN} \
--digitalocean-region=nyc3 \
--digitalocean-size=512mb \
--swarm \
--swarm-discovery token://${SWARM_CLUSTER_TOKEN} \
swarm-worker
Note: In early 2018 Digital Ocean announced new droplet plans and the 512mb option
was retired and
replaced with with a s-1vcpu-1gb option.
If everything worked as expected you now have two Digital Ocean droplets each hosting a swarm node, with one designated as the master. Let’s verify that.
Verify node creation
Verify host nodes were created as expected using the Machine CLI. Run the following to to get a list of nodes:
docker-machine ls
You should see output like:
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS swarm-worker - digitalocean Running tcp://104.131.163.130:2376 swarm-manager v1.12.0 swarm-manager - digitalocean Running tcp://104.236.207.117:2376 swarm-manager (master) v1.12.0
If you don’t see the nodes you expected, you can use the docker-machine rm command to remove nodes by NAME or go back and try and recreate them.
digitalocean driver will attempt to destroy the host droplet or assume the droplet was already destroyed.
Once both machines are up and running on Digital Ocean you’re ready to move on.
Connect the nodes
With the two nodes created and verified it’s time to connect them. To do so we will initialize swarm on swarm-manager then ask swarm-worker to join them.
Start by initializing swarm on the manager:
eval $(docker-machine env swarm-manager) && \
docker swarm init --advertise-addr $(docker-machine ip swarm-manager)
The above command will attach a shell to the manager, and use it to initialize a swarm and begin advertising its address. Once run you should see output like:
Swarm initialized: current node (8uaq2r536t4mgs6ci95yqmf5m) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join \
--token SWMTKN-1-1zm9fvpvk03vwxasrxjk6y7lueor77789ip2xfo0czmny3tvui-evnoe2a0x0ygex88vl3io9n1f \
104.236.207.117:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
Now switch over to the worker and ask it to join the swarm:
eval $(docker-machine env swarm-worker) && \
docker swarm join \
--token SWMTKN-1-1zm9fvpvk03vwxasrxjk6y7lueor77789ip2xfo0czmny3tvui-evnoe2a0x0ygex88vl3io9n1f \
104.236.207.117:2377
Be sure to use the expected token. When the worker joins you will see:
This node joined a swarm as a worker.
And now the two nodes are swarming (clustered).
Verify the connection
Check with the manager to verify the worker actually joined their swarm:
eval $(docker-machine env --swarm swarm-manager) && \
docker info | grep worker -A 8
The --swarm flag is used to get info about the swarm. Grep to show more information about the worker. You should see output like:
swarm-worker: 104.131.163.130:2376 └ ID: DKKE:XMMP:Y274:4N5H:CW6T:Z5CD:5I3H:KI35:RL77:DY3J:3D3F:ALDX └ Status: Healthy └ Containers: 1 (1 Running, 0 Paused, 0 Stopped) └ Reserved CPUs: 0 / 1 └ Reserved Memory: 0 B / 512.9 MiB └ Labels: kernelversion=4.4.0-31-generic, operatingsystem=Ubuntu 16.04.1 LTS, provider=digitalocean, storagedriver=aufs └ UpdatedAt: 2016-08-11T01:49:27Z └ ServerVersion: 1.12.0
With the worker and manager swarming you can give them something to do.
Create a service
To make use of our swarm we’re going to create a Swarm service that consists of six replicated Nginx containers, spread equally across both nodes.
First shell into manager and create the service:
eval $(docker-machine env swarm-manager) && \
docker service create -p 8000:80 \
--replicas 6 --name web nginx
The above command uses Engine to create a new service called web using the official build of nginx replicated six times across your two swarm host nodes.
You should see output indicating progress:
swly782eykcinix09jns3t7xy overall progress: 2 out of 6 tasks 1/6: starting 2/6: running 3/6: running 4/6: starting 5/6: starting 6/6: starting
When tasks are stable a verify: Service converged message will appear.
Expand for debugging information
Attempt to create service on the worker:
Error response from daemon: This node is not a swarm manager. Worker nodes can't be used to view or modify cluster state. Please run this command on a manager node or promote the current node to a manager.
Attempt to use reuse active ingress port:
Error response from daemon: rpc error: code = InvalidArgument desc = port '8000' is already in use by service 'web' (xim9nv43t3otpg57soauhnu70) as an ingress port
Attempt to use an existing service name:
Error response from daemon: rpc error: code = Unknown desc = name conflicts with an existing object
Attempt to remove nonexistant service:
Error: No such service: web
Once the service converges, verify it’s working as expected.
Verify service
Run the docker service ps command:
docker service ps web
You should see six replicas spread evenly across both of your swarm nodes:
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR c9m65b4efoz1026o6e3k8yvpv web.1 nginx swarm-worker Running Running about a minute ago 66y7fwei2i3k863lyvdf2e202 web.2 nginx swarm-manager Running Running about a minute ago 4hqsepx7cgkdlgby7wtjrntx8 web.3 nginx swarm-manager Running Running about a minute ago 3inowh5wxm5zymqva4kv7ksf9 web.4 nginx swarm-worker Running Running about a minute ago 72mkdmm0l36702m9g24xnkaa7 web.5 nginx swarm-manager Running Running about a minute ago 2dz8zmaf9ijqmhcnwq9c3mtfm web.6 nginx swarm-worker Running Running about a minute ago
Note: Swarm has a built-in load balancer to distribute workload event across replicas using round-robin. Learn about it inside and out.
Verify both nodes are serving content:
wget -qO - http://$(docker-machine ip swarm-worker):8000 | grep title && \
wget -qO - open http://$(docker-machine ip swarm-manager):8000 | grep title
If you see the following everything worked:
<title>Welcome to nginx!</title>
<title>Welcome to nginx!</title>
You’ve now set-up a basic two-node cluster on Digital Ocean using Docker Swarm.
Wrapping up
In this post you learned how to use the command line to create a Swarm cluster with Machine CLI and use it to create basic two-node Swarm service running 6 nginx replicas spread across two swarm host nodes on Digital Ocean.
If you like You can now learn how to scale the service. But don’t forget to take down your service clean-up remove your nodes when you’re finished:
docker service rm web && \
docker-machine rm swarm-worker && \
docker-machine rm swarm-manager
The second two commands will remove the droplets created on Digital Ocean so they don’t run up your bills in case you forget about them.
You can confirm the local reference and remote instance of the nodes were removed by inspecting your droplets on Digital Ocean Cloud.
Welcome to the swarm!
Further Reading
This post wouldn’t have be possible without some initial inspiration and guidance from following resources, so please check them out: