PART-3 (CICD with Go Pipelines and Docker Swarm Mode Cluster)
Installation:
In the
Part-1 and
Part-2 we setup the Docker Swarm Mode cluster, built a Docker Image for an app and pushed that image into a local docker repository. In this part we will install and setup a popular CICD tool called Go Pipeline, and use it to perform 1-click deployments and rolling updates.
Step-1: Download and Install Go Pipeline Server and Agent (for CICD)
It'd require Java. So make sure to install Java too. Below is what I have on my Mac.
nverma@macbook-pro$ java -version
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
I installed both the Go Pipeline Server and Agent locally on my Mac/PC. Once the Go Server started, it became accessible via
http://localhost:8153/go/pipelines
Tip: For the Go Agent to run the commands as needed, we need to set the
PATH variable like below:
nverma@macbook-pro$ pwd
/Users/nverma/Library/Application Support/Go Agent
nverma@macbook-pro$ cat overrides.env
PATH=$PATH:/usr/local/bin:/Users/nverma/Documents/apache/apache-maven-3.3.9/bin
nverma@macbook-pro$
Step-2: Setup the Pipeline
To make it easier to test and setup, let us create a single Pipeline called “mypipe”, which has single Stage called “mystage” with three Jobs called “1_build_job”, “2_push_job” and “3_deploy_job”.
As per the names:
-
1_build_job: is to fetch the code from Github, run the Maven build and create the Docker Image.
-
2_push_job: is to run the Docker push command to push the image into the local image repository (i.e. master:5000)
-
3_deploy_job: is to create a Docker service if first time, else update the image with the latest image and let Docket swarm do a rolling update to the live application.
To connect to the Swarm “master” node, set the following variables at the pipeline (“mypipe”) level under the “Environment Variables” tab.
'DOCKER_HOST' to 'tcp://192.168.99.101:2376'
'DOCKER_MACHINE_NAME' to 'master'
'DOCKER_TLS_VERIFY' to '1'
'DOCKER_CERT_PATH' to '/Users/nverma/.docker/machine/machines/master'
Example:
Under the “
Materials” tab, add material of type “Git” with the URL of the application source code, in this case use my sample app’s GIT repo i.e. “
https://github.com/n1t1nv3rma/springboot-maven.git”.
Example:
Update the Jobs:
a) Update “1_build_job” with a new ‘Task’ of type “Custom Command”. Delete the default Task of type “Ant” and add a Custom Type by selecting option “more”
- Under the Command type “mvn”
- Under the ‘Arguments’ type “package” and “docker:build” in new lines (in Go even argument is written in new line).
Example:
b) Update “2_push_job” with a new ‘Task’ of type “Custom Command”. Delete the default Task of type “Ant” and add a new one of type “more”
- Under the Command type path to a custom script “/Users/nverma/Documents/Maven/scr/push-image.sh”
The contents of this script are:
------
#!/bin/bash
eval $(docker-machine env master)
IMG=`docker images | grep master:5000 | sort -k2,1 -nr | head -1 | awk '{print $1":"$2}'`
docker push $IMG
------
c) Update “3_deploy_job” with a new ‘Task’ of type “Custom Command”. Delete the default Task of type “Ant” and add a new one of type “more”
- Under the Command type path to a custom script “/Users/nverma/Documents/Maven/scr/run-service.sh”
The contents of this script are:
------
#!/bin/bash
eval $(docker-machine env master)
IMG=`docker images | grep master | sort -k2,1 -nr | head -1 | awk '{print $1":"$2}'`
SERVICE="myspringapp"
if [ `docker service ls -f NAME=$SERVICE -q | wc -l` -gt 0 ]
then
echo "updating service $SERVICE..."
docker service update --image $IMG $SERVICE
else
echo "ceating service $SERVICE..."
docker service create --replicas 1 --name $SERVICE --update-delay 20s -p 8080:8080 $IMG
fi
------
We are all set for the basic automation. For more information about the Go Pipeline refer: “https://docs.go.cd/current/introduction/concepts_in_go.html”.
Step-3: Run the Pipeline
Click on the "Play" button or make changes to code. If all goes well, all these 3 Jobs should fetch the code, build the image and run the required service using it. In case of any issues, click the Pipeline (status bar) or each Job (status bar) to view the Console log for further troubleshooting.
To initiate the pipeline, I’d just update the application code and commit/push the code into the Github, and the pipeline should start on it’s own.
For example, as a simple change, let’s update the version to “
<app.version>1.7</app.version>” in the
pom.xml, say under the “
/Users/nverma/Documents/test” directory (where I cloned the application files in Part-2) and push the changes to the Github repo.
One should have the new Application running with newer docker images 1.7.
Detailed example:
nverma@macbook-pro$ grep app.version pom.xml
<app.version>1.7</app.version>
<imageName>${docker.image.prefix}/${project.artifactId}:${app.version}</imageName>
nverma@macbook-pro$
nverma@macbook-pro$ git add .
nverma@macbook-pro$ git commit -m "initial"
[master 3aa0ddf] initial
1 file changed, 1 insertion(+), 1 deletion(-)
nverma@macbook-pro$
nverma@macbook-pro$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)
nothing to commit, working directory clean
nverma@macbook-pro$
nverma@macbook-pro$ git push -u origin master
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 341 bytes | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local objects.
To https://github.com/n1t1nv3rma/springboot-maven.git
ed8d069..3aa0ddf master -> master
Branch master set up to track remote branch master from origin.
nverma@macbook-pro$
This should initiate the Pipeline “mypipe” and all 3 Jobs should run one by one.
Following is what I see now:
nverma@macbook-pro$ docker images | grep 1.7
master:5000/gs-spring-boot-docker 1.7 79d919940ce3 25 seconds ago 195.3 MB
nverma@macbook-pro$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
2d9ngt8phrhkdm21cwjv1tjss worker2 Ready Active
6rarliq49ehaoi5olk85ztpfe worker1 Down Drain
bo83ch825l2t1mbo8n6z4xkxj * master Ready Active Leader
nverma@macbook-pro$ docker service ls
ID NAME REPLICAS IMAGE COMMAND
3cu2kq6z0zx9 myspringapp 1/1 master:5000/gs-spring-boot-docker:1.7
nverma@macbook-pro$
nverma@macbook-pro$ docker service ps myspringapp
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR
0inz6b7b1z76jw62091ajwgcl myspringapp.1 master:5000/gs-spring-boot-docker:1.7 worker2 Running Running about a minute ago
nverma@macbook-pro$
nverma@macbook-pro$ docker service scale myspringapp=2
myspringapp scaled to 2
nverma@macbook-pro$ docker service ps myspringapp
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR
0inz6b7b1z76jw62091ajwgcl myspringapp.1 master:5000/gs-spring-boot-docker:1.7 worker2 Running Running 2 minutes ago
5ndjck2hnxmhabd7c00bkp1pk myspringapp.2 master:5000/gs-spring-boot-docker:1.7 master Running Running 8 seconds ago
nverma@macbook-pro$
Further evaluation:
Perform following steps to further enhance your docker experience!
*) Scale up/down, and try hitting the service on a Swarm node which is not running service currently. For example in the above output, service is not running on the “worker1”, so hitting the service on “worker1:8080/greet?name=Testing” should still work! Magic? Yeah… Docker Swarm Mode has implemented full internal network mesh, routing and load-balancing! So cluster nodes are smart enough to route the request to the node running it!
*) Run “docker service create --help” and check out the “—update-delay duration” and “—update-parallelism uint” options and perform Rolling Updates to your App.
*) Create your own Overlay Network in your Swarm and use it to laugh another Application/Service! And see if you can reach Service from one network to a Service from another network. It shouldn’t and that the beauty of complete isolation.
More info: “$docker network --help” and https://docs.docker.com/engine/swarm/networking/
That’s it for now... Hope you enjoyed the 3 blogs and Docker Swarm Mode! Have fun!
