KGR VAMSI

Docker Swarm

Docker Swarm is another tool from Docker where it does the cluster Management of the Docker Instances Across Multiple Cloud Providers.

Initially we will be spinning the Docker instances through Docker machine.To know how to setup the docker-machine can go through the Docker {Machine-Swarm-Compose} Magic around the Whale -01.

export AWS_ACCESSKEY={your own aws account accesskey}
exprt AWS_SECRETKEY={your own aws account secretkey}

Then we need to create the Master Swarm node through docker-machine before that we need to create a Swarm Token

docker run swarm create

Based on the output we take the token Id we will create a Swarm Master node

docker-machine create -d amazonec2 --amazonec2-access-key $AWS_ACCESSKEY --amazonec2-secret-key $AWS_SECRETKEY \
--amazonec2-instance-type "t2.medium" --amazonec2-region "us-east-1"\
--amazonec2-vpc-id "{{Vpc-Id}}" --amazonec2-subnet-id "{{Subnet-Id}}"\
--swarm --swarm-master --swarm-discovery token://3d4efe3135c2b63c127ab5063aee6ff8 \
swarm-node-master

This will create us with the Swarm Master node and now we need to create the Swarm nodes

docker-machine create -d amazonec2 --amazonec2-access-key $AWS_ACCESSKEY --amazonec2-secret-key $AWS_SECRETKEY \
--amazonec2-instance-type "t2.medium" --amazonec2-region "us-east-1" \
--amazonec2-vpc-id "vpc-8e3317eb" --amazonec2-subnet-id "subnet-61106116" \
--swarm --swarm-discovery token://3d4efe3135c2b63c127ab5063aee6ff8 swarm-node-00

So we will be creating another swarm node and we will be attaching it to the Swarm master using the token id.Lets see if we can see the information about the Swarm Cluster

ubuntu@ip-10-20-30-42:~$ docker info
Containers: 14
Images: 5
Role: primary
Strategy: spread
Filters: affinity, health, constraint, port, dependency
Nodes: 3
 swarm-node-00: 54.172.29.140:2376
 └ Containers: 7
 └ Reserved CPUs: 0 / 2
 └ Reserved Memory: 0 B / 4.052 GiB
 └ Labels: executiondriver=native-0.2, kernelversion=3.13.0-53-generic, operatingsystem=Ubuntu 14.04.2 LTS, provider=amazonec2, storagedriver=aufs
 swarm-node-01: 54.175.55.195:2376
 └ Containers: 1
 └ Reserved CPUs: 0 / 2
 └ Reserved Memory: 0 B / 4.052 GiB
 └ Labels: executiondriver=native-0.2, kernelversion=3.13.0-53-generic, operatingsystem=Ubuntu 14.04.2 LTS, provider=amazonec2, storagedriver=aufs
 swarm-node-master: 54.172.181.206:2376
 └ Containers: 6
 └ Reserved CPUs: 0 / 2
 └ Reserved Memory: 0 B / 4.052 GiB
 └ Labels: executiondriver=native-0.2, kernelversion=3.13.0-53-generic, operatingsystem=Ubuntu 14.04.2 LTS, provider=amazonec2, storagedriver=aufs
CPUs: 6
Total Memory: 12.16 GiB

The above cluster information we can get it by running the following command

eval "$(./docker-machine env --swarm  swarm-node-master)"

Now we will run some containers on these machines .I will be running a small script to create 50 containers through a small shell script

for i in {1..50};
 do 
k=$(expr 80 + $i); 
docker run -itd -p $k:80  ubuntu /bin/bash;
 done

Now lets check the list of containers based upon their ip’s on how many containers have been launched on the Cluster nodes.

ubuntu@ip-10-20-30-42:~$ docker ps|grep 54.175.55.195|wc -l
27

ubuntu@ip-10-20-30-42:~$ docker ps|grep 54.172.181.206|wc -l
24

We can see the output of the running docker containers

Next Post will be on how to write a Docker-compose based templates over the Swarm Cluster

Transformation of Applications to Docker way of implementations is taking the hold with most of the domains to most effective usage of the resources and for a better Micro Service Management  Strategy.

Docker as a Engine which operates as the light weighted container Hypervisor needs a helping hand with some of the tools which solves most of the orchestration and clustering problems which one eventually try to create a loop of servers on a single click.

Seeing that Issue Docker have created some magical tools around which gives solutions to some of the problems which can solve the Orchestration and Clustering of the Docker Containers across the Cloud Providers or any Hybrid Cloud Model.

Lets start the Fight

Docker {Machine}

Docker Machine is a tool which helps you provision servers on multiple cloud providers to install docker and give you the access to these machines over tcp port which the docker runs which exposes it api.

How to download Docker-Machine

Linux and Other Distros

$ curl -L https://github.com/docker/machine/releases/download/v0.2.0/docker-machine_darwin-amd64 > /usr/local/bin/docker-machine
$ chmod +x /usr/local/bin/docker-machine

Windows

curl -L https://github.com/docker/machine/releases/download/v0.2.0/docker-machine_windows-amd64.exe > /usr/local/bin/docker-machine

How to run

I use Cygwin on my machine to make use of the Linux based functionalities on the machine easily,so i saved the docker-machine.exe in the /usr/local/bin/docker-machine

so after that we need to create a instance on  a aws and docker-machine comes up with most of the Cloud providers integration like IBM Softlayer,DigitalOcean,Rackspace and AWS and also with some of the private cloud providers like Openstack,Vmvspehere.

So Lets create a instance on AWS

On a Special Note is that Docker have release there Windows client where we can connect to the docker instances with their api.

You can download it from the Windows Package manager Chocolatey.

Before creating the instance make sure to understand on what options are available to create on aws

 --amazonec2-access-key     AWS Access Key [$AWS_ACCESS_KEY_ID]
 --amazonec2-ami            AWS machine image [$AWS_AMI]
 --amazonec2-iam-instance-profile        AWS IAM Instance Profile
 --amazonec2-instance-type "t2.micro"    AWS instance type [$AWS_INSTANCE_TYPE]
 --amazonec2-region "us-east-1"          AWS region [$AWS_DEFAULT_REGION]
 --amazonec2-root-size "16"              AWS root disk size (in GB) [$AWS_ROOT_SIZE]
 --amazonec2-secret-key                  AWS Secret Key [$AWS_SECRET_ACCESS_KEY]
 --amazonec2-security-group "docker-machine"  AWS VPC security group [$AWS_SECURITY_GROUP]
 --amazonec2-session-token                    AWS Session Token [$AWS_SESSION_TOKEN]
 --amazonec2-subnet-id                        AWS VPC subnet id [$AWS_SUBNET_ID]
 --amazonec2-vpc-id                           AWS VPC id [$AWS_VPC_ID]
 --amazonec2-zone "a" AWS zone for instance (i.e. a,b,c,d,e) [$AWS_ZONE]

So lets export the AWS Accesskey and Secretkey

export AWS_ACCESSKEY={your own aws account accesskey}
exprt AWS_SECRETKEY={your own aws account secretkey}

Creating the Instance on AWS

docker-machine create -d amazonec2 --amazonec2-access-key $AWS_ACCESSKEY \
--amazonec2-secret-key $AWS_SECRETKEY \
--amazonec2-instance-type "t2.medium" \
--amazonec2-region "us-east-1" \
--amazonec2-vpc-id "vpc-8e3317eb" \
--amazonec2-subnet-id "subnet-61106116" Demo123

Instance creation may take some time to do lot more magical stuff to provision you the instance with a running docker on top of it.After creating the instance lets see if we can see it in the list of instances from docker-machine output.

raghuvamsi@VAMSI-LT /cygdrive/d/projects
$ docker-machine ls
NAME       ACTIVE  DRIVER     STATE     URL                      SWARM
Demo123       *   amazonec2   Running   tcp://52.7.192.125:2376
liferay-demo      virtualbox  Stopped

Now lets check connecting to the instance .If you are running on a CYGWIN based environment then we need to run this command

eval $(docker-machine env Demo123)

So now lets check it with the docker command and see whether we can run some of the docker commands

Now lets pull some images on this Machine

We have pulled ubuntu image on this machine

And now we can run all the docker commands on this pc.If you would like to login to this machine you need to use the pem key created for this machine at ~/$USER_HOME/.docker/machine/machines/$MACHINE_NAME/id_rsa

I will kill or destroy this machine with the following command

Docker-machine comes with some more commands to inspect the instance and check its version with

docker-machine inspect Demo123   -> Inspect information about a machine
docker-machine config Demo123    -> Print the connection config for machine

Stay Tuned for the Second part of it on how we integrate a Swarm Cluster with multiple Instances across regions on AWS

Dockerstack a Open source tool started with an Idea to make Docker based Integrations much more easier and efficient to have a control over the Infrastructure which runs the Docker containers.

Integrated Deployments was one of the Continuous Deployment trend which Dockerstack is introducing to cultivate the Practices for Better and Concrete Deployments across Public and Private Cloud Deployments.

As a First Step we thought of doing a integration of Ansible which is one of the Light-Weighted Configuration management tool(so far seen with other tools with Less learning Curve and effective DSL to understand) to make deployments across the containers.

Initially every user in the Dockerstack application can create their unique Public and Private Keys at their Settings Page.

                                                                         Fig 1.0 Creation of SSH Keys

Later the creation of SSH Keys you get the Keys Displayed and by default the keys are generated in RSA.

                                                                        Fig 1.1 Public and Private Keys of User

After the Successfull creation of the keys and before the creation of Ansible Configuration files it is manditory that  your server which runs the Dockerstack application should have ansible installed.

If You Confirm the installation then you may have to create the Configuration files for the ansible to takeover the user specific environment configuration to make ansible run smoothly.

                                                        Fig 1.2 Ansible Configuration file

Every User have their own specific ansible.cfg file which makes the ansible to work based upon their environment setup.(ansible.cfg have its own fine tuned settings for the ansible work around multiple configuration files across multiple users)

The hosts file which is the main route file for the ansible to read and make deployments across the containers.

By default we make the Loopback addresses 127.0.0.1 as its first machine to interact.

                                                              Fig 1.3 Ansible Hosts File

Now lets check whether ansible can grab these settings and make us feel happy with its basic ping pong response,before this we need to make sure the public key is being shared to the specific user to make the ansible process easy

                                                          Fig 1.4 Sharing the Public Key

Every user home directory is a GUID key based folder which holds the SSH and Ansible Keys and Configurations.

After the Key is Added and if you want to check everything works at a command line you can try with he ansible CLI like this (This step is optional if you are 100% confident on the tool)

                                                               Fig 1.5 Ansible Test Run from CLI

If your Configuration is perfect you will be getting a reply like this

                                                          Fig 1.6 Result Back from the Ansible CLI

We are being using the -vvvv for detailed understanding of the ansible tool on how it takes the configuration and how the response hits back from the server which it was suppose to send the shell command to execute.

Now lets see if the same response can be captured back on the Front-end

                                                          Fig 1.7 Checking the Hosts file from the UI

Click on the check hosts button which is available under the Check tab and see the Magical Output

                                                             Fig 1.8 Magical Response in the UI

As a First step we made the ansible to make the host check Programatically and we may also include to make ansible deploy playbooks across the containers easily from the Dockerstack Application Dashboard.

To know more updates about the Dockerstack application here are some of the Reference urls

HomePage – Dockerstack.org

Source Code – github.com\dockerstack