Monday, 10 August 2015

Monitoring UrbanCode Deployments with Docker, Graphite, Grafana, collectd and Chef! (Part 1)

Monitoring an UrbanCode Deploy server (sometimes more in HA setups) and it's agents requires keeping track of resource utilization multiple environments, the UrbanCode deployment server(s) itself and the linkages between (ie. the network).

Typical resources include:

  • CPU
  • Memory
  • Java Heap
  • Threads
  • Disk
  • Network
  • Virtual environment (hypervisor)

In addition to resource utilization, log files should also be monitored for abnormal activity and traffic. There are commercial offerings which do these types of things but since UrbanCode Deploy itself is a deployment solution, it can be used to deliver monitoring to nodes. All that's is needed is monitoring agents and a collector and a means to configure and connect it all together.

In this post I'll demonstrate a quick bootstrap solution for system and JVM resource monitoring using UrbanCode Deploy. It will provide an "out of the box" monitoring dashboard solution, Grafana from data stored in Graphite (running in a Docker container) of metrics collected by collectd that installed on nodes using a Chef recipe that's deployed through UrbanCode Deploy. The end result looking something like this:

Fig. 1 Monitoring Topology
Fig. 1 Monitoring Topology

For the time being this solution is solely for a Linux environments (RHEL, Ubuntu and variants) but this solution can be adapted to other OS's as many of the components have counterparts for Windows, AIX and other OS's.

So how do we get there? Well, one approach is to set it up manually, quite an operation if you have 1000s of agents, so we'll need to do better.

First, the assets need to be installed.

Fig. 2 Installing the Solution
We will need:

  1. An UrbanCode server with a few agents. You'll also need to install the chef plugin from here:
  2. I also created a plugin with groovy that adds 2 additional steps for components. One step gets the latest version for a component, and the other step gets an ID for a version in the component. You can see the source code here, it's a good example of how to create a custom plugin. It's quite simple.
  3. Import a component from IBM BlueMix DevOps Service Git found here:
    Import it from the Components tab:

    You should now see it listed:

    The component is preconfigured to connect to 
    IBM BlueMix DevOps Service Git and pull the recipe periodically and create a new version, you may change this behaviour in Basic Settings by unchecking the Import Versions Automatically setting.
    All you need to do now is supply a BlueMix username and password in the component properties page. You may need to enter a username, if applicable (without a domain).

  4. Now you need to import a generic process (the top level Processes tab. Not the component!) that will be used to deploy the latest version of the component deployment package onto agent nodes. This process is kept in IBM BlueMix DevOps Services's Git and can be found here:

    Or, you can quickly import this into UrbanCode Deploy by using curl:
    curl -k -X POST -F file=@Install_collectd.json https://<user>:<pass>@<ucd host>/rest/process/import
    NOTE: I noticed that after importing the generic template the versionName step in the Generic Import_collectd process design (design tab) had three bullets "•", this needs to be updated to ensure that the Secure Property Value field is blank. If it's not, the fetching of the latest version will fail when version is not specified.
  5. We need a metrics collector to store the metrics and a graphing engine to visualize them. We'll be using a Docker image of Graphite/Grafana I put together. You will need to ability to build run a docker container either using boot2docker or the native support available in Linux
    I have put the image up on the public docker registry as bkuschel/graphite-grafana but you can also build it from the Dockerfile in IBM BlueMix DebOps Services's Git at
  6. To get the image run:

    docker pull bkuschel/graphite-grafana

    Now run the image and bind the ports 80 and 2003 from the docker container to the hosts ports.

    docker run -p 80:80 -p 2003:2003 -t bkuschel/graphite-grafana

    You can also mount file volumes to the container that contains the collector's database, if you wish that to be persisted. Each time you restart the container, it contains a fresh database. This has its advantages for testing. You can also specify other configurations beyond what are provided as defaults. Look at the Dockerfile for the volumes.

Once the solution is installed all that needs to be done is to execute the process on UrbanCode. Yes, it's that easy.

Go to the Process Tab in UrbanCode Deploy Server, Click on Run Next to the "Install_collectd" process.

A dialog will popup asking for a series of parameters. These will be explained in more depth in a later post regarding the Chef recipe I created. (You can find it here)
  • Component Name: Should be set to the name of the component we imported earlier
  • Version Name (Optional): You can specify the name of a specific version of the component to use, otherwise it will use the latest
  • Is this a collectd Server?: If you look at Fig. 1, you'll see that many collectd clients connect to a central collectd server. If this node is the central collectd, this should be checked. Generally, this should be the main agent in the UrbanCode Server, usually co-located with the server.
  • Collectd Install Directory: The default is good
  • Collectd Username: You can leave this as default. This username is the one used to encrypt traffice between collectd clients and servers.
  • Collectd Password: Set any password. This password is the one used to encrypt traffic between collectd clients and servers. It's a good idea  to encrypt this password with htpasswd utility before pasting it here. For example to set the admin password, the first parameter is the username, the second is the password. The output contains the username, a colon, then the encrypted password. Paste that value in this property:

    > htpasswd -bnm admin admin
  • Collectd Server (client)/Graphite host (server): if "Is this a collectd Server?" is checked then this is the graphite server host, the host that is running the docker container. Otherwise this the collectd server host.
  • UCD Server: The installation directory of the UrbanCode server (ex. /opt/ibm_ucd/server) if this collectd is to be installed on a node with a server.
  • Java Monitoring Template: If UCD Server is set and is installed on tomcat, select tomcat.conf.erb, otherwise select java.conf.erb.
  • Resource: Select the agent that this process should be executed on. (the host.

Once you click Submit, this should happen:
Deploy Process
Fig 3. Deploy Process
At this point, all the collectd daemons should be started and collecting. Navigate to the docker host at http://<Docker host>/. You should see a tree with metrics, like this:

You can also navigate to Grafana at http://<Docker Host>/grafana. Note that the username and password for both Graphite and Grafana are admin/admin.

This is quite a mouthful for one blog post and there are so many aspects to cover such as:
  • The UrbanCode Deploy Process, how does it work?
  • The Chef Recipe.
  • Collectd Collection Options (and the nmon option!).
  • How to create useful graphs in Graphite and dashboards in Grafana.
I will cover these in subsequent postings. In the meantime, try to set it up and see how it goes. If you're lucky, you end up playing with some cool metrics and graphs in Graphite/Grafana.


  1. What a great article!. I am bookmarking it to read it over again after work. It seems like a very interesting topic to write about.
    SAP training in Kolkata
    SAP training Kolkata
    Best SAP training in Kolkata
    SAP course in Kolkata
    SAP training institute Kolkata

  2. Good job in presenting the correct content with the clear explanation. The content looks real with valid information. Good Work

    Dot Net Training in Chennai | Dot Net Training in anna nagar | Dot Net Training in omr | Dot Net Training in porur | Dot Net Training in tambaram | Dot Net Training in velachery

  3. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    Data Science Institute in Bangalore

  4. Nice post found to be very impressive with a genuine content while going through this post. Thanks for sharing and keep posting such an informative content.

    360DigiTMG Digital Marketing Course

  5. Tremendous blog quite easy to grasp the subject since the content is very simple to understand. Obviously, this helps the participants to engage themselves in to the subject without much difficulty. Hope you further educate the readers in the same manner and keep sharing the content as always you do.

    360DigiTMG Data Science Course

  6. I really enjoyed reading your blog filled with valuable information thank you.
    Data Analytics Course Online 360DigiTMG

  7. Fantastic blog with very informative information, found valuable thanks for sharing
    typeerror nonetype object is not subscriptable

  8. Comfortably, the post is really the freshest on that deserving topic. I harmonies with your conclusions and definitely will thirstily look forward to your next updates. cheap dedicated servers silicon valley

  9. Really wonderful blog completely enjoyed reading and learning to gain the vast knowledge. Eventually, this blog helps in developing certain skills which in turn helpful in implementing those skills. Thanking the blogger for delivering such a beautiful content and keep posting the contents in upcoming days.

    Data Science certification in Bhilai

  10. Writing in style and getting good compliments on the article is hard enough, to be honest, but you did it so calmly and with such a great feeling and got the job done. This item is owned with style and I give it a nice compliment. Better!
    Cyber Security Training in Bangalore

  11. So, when you decide to implement Salesforce for your business operations, then it is very important to hire any expert Salesforce interview questions and answers

  12. They are produced by high level developers who will stand out for the creation of their polo dress. You will find Ron Lauren polo shirts in an exclusive range which includes private lessons for men and women.

    Business Analytics Course in Bangalore

  13. Such trends have engendered a growing requirement to provide professionals with apt handling on huge info analysis. data science course syllabus

  14. Attend The Data Analyst Course From ExcelR. Practical Data Analyst Course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analyst Course.
    Data Analyst Course

  15. This comment has been removed by the author.

  16. I have to search sites with relevant information ,This is a
    wonderful blog,These type of blog keeps the users interest in
    the website, i am impressed. thank you.
    Data Science Course in Bangalore

  17. Great post! I really appreciate you and I like to more unique content about this title and keep updating here...
    Informatica Training in Bangalore
    Informatica Training in Chennai
    Informatica MDM Training in Chennai
    Informatica Course in Chennai

  18. Happy to visit your blog, I am by all accounts forward to more solid articles and I figure we as a whole wish to thank such huge numbers of good articles, blog to impart to us.
    data science in malaysia

  19. Very awesome!!! When I searched for this I found this website at the top of all blogs in search engines.
    Best Digital Marketing Courses in Hyderabad

  20. I think I actually have never seen such blogs ever before that has complete things with all details that i would like. therefore kindly update this ever for us.
    data scientist certification


  21. Very awesome!!! When I searched for this I found this website at the top of all blogs in search engines.

    business analytics course

  22. "Very Nice Blog!!!

    Please have a look about "
    data scientist course

  23. This is a great post. I like this topic.This site has lots of advantage.I found many interesting things from this site. It helps me in many ways.Thanks for posting this again.
    data scientist course in hyderabad

  24. This post is very simple to read and appreciate without leaving any details out. Great work!
    data science training

  25. หาคุณกำลังหาเกมส์ออนไลน์ที่สามารถสร้างรายได้ให้กับคุณ เรามีเกมส์แนะนำ เกมยิงปลา รูปแบบใหม่เล่นง่ายบนมือถือ คาสิโนออนไลน์ บนคอม เล่นได้ทุกอุปกรณ์รองรับทุกเครื่องมือ มีให้เลือกเล่นหลายเกมส์ เล่นได้ทั่วโลกเพราะนี้คือเกมส์ออนไลน์แบบใหม่ เกมยิงปลา

  26. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    machine learning training in aurangabad

  27. Baccarat is actually money making and it's remarkable accessibility. Optimal In your case it's readily available that you will find pretty fascinating choices. And that is thought to be one thing that is really different And it's very something that is really prepared to hit with Pretty much the most wonderful, as well, is actually a really positive option. Furthermore, it's a really fascinating solution. It is a better way which can make money. Superbly prepar The number of best-earning baccarat will be the accessibility of generting the most money. Pretty much as practical is very well suited for you A substitute that could be assured. To a wide range of accessibility and performance And find out outstanding results also.บาคาร่า

  28. I must admit that your post is really interesting. I have spent a lot of my spare time reading your content. Thank you a lot!
    data scientist training and placement in hyderabad

  29. I wanted to leave a little comment to support you and wish you the best of luck. We wish you the best of luck in all of your blogging endeavors.

    Artificial Intelligence Training in Bangalore

  30. Incredibly conventional blog and articles. I am realy very happy to visit your blog. Directly I am found which I truly need. Thankful to you and keeping it together for your new post.
    data scientist online course

  31. Thanks for another informative site. Where else could I get that kind of info written in such a perfect way? I have a project that I am just now working on, and I have been on the look out for such info. ยูฟ่าสล็อต

  32. This is also a very good post which I really enjoyed reading. It is not every day that I have the possibility to see something like this..
    data science courses in aurangabad

  33. I think this is an informative post and it is very useful and knowledgeable. Therefore, I would like to thank you for the efforts you have made in writing this article.
    data scientist training in hyderabad

  34. Great Article. Thank you for sharing! Really an awesome post for everyone.
    DevOps Training in Hyderabad
    DevOps Course in Hyderabad

  35. Thanks for posting the best information and the blog is very important.artificial intelligence course in hyderabad

  36. Ucuz, kaliteli ve organik sosyal medya hizmetleri satın almak için Ravje Medyayı tercih edebilir ve sosyal medya hesaplarını hızla büyütebilirsin. Ravje Medya ile sosyal medya hesaplarını organik ve gerçek kişiler ile geliştirebilir, kişisel ya da ticari hesapların için Ravje Medyayı tercih edebilirsin. Ravje Medya internet sitesine giriş yapmak için hemen tıkla:

    İnstagram takipçi satın almak için Ravje Medya hizmetlerini tercih edebilir, güvenilir ve gerçek takipçilere Ravje Medya ile ulaşabilirsin. İnstagram takipçi satın almak artık Ravje Medya ile oldukça güvenilir. Hemen instagram takipçi satın almak için Ravje Medyanın ilgili sayfasını ziyaret et: instagram takipçi satın al

    Tiktok takipçi satın al istiyorsan tercihini Ravje Medya yap! Ravje Medya uzman kadrosu ve profesyonel ekibi ile sizlere Tiktok takipçi satın alma hizmetide sunmaktadır. Tiktok takipçi satın almak için hemen tıkla: tiktok takipçi satın al

  37. It is our mission to guarantee and ensure that all goods under our care are never tampered with. top security companies in London
    We offer specialized asset and high-value goods protection services incomparable to that of any other security company in London.

  38. This was an extremely wonderful post. Thanks for providing this info. Kanye West Donda Vest

  39. I really enjoyed reading your articles. It looks like you’ve spent a lot of time and effort on your blog. Dreamer Biker Jacket

  40. I love to recommend you Where can crawl Exciting Products latest Jackets, Coats and Vests Click Here Peter Parker Jacket

  41. I read blogs on a similar topic, but I never visited your blog. I added it to favorites and I’ll be your constant reader. Biker Boyz Jacket

  42. This was an great post . Thanks for sharing your blog post with us....we offer projects in chennai in different domains such like final year projects 2021

  43. As the bright sunshine, hope your day be blessed with positivity and cheerful thoughts. Wishing you a warm Thursday good morning. Thursday Greetings Quotes

  44. Very informative article! Thank you for sharing! I like it so much. If ever you need tree service at Willow Grove Pennsylvania don't hesitate to check it

  45. Thank you for taking the time and sharing this information with us. Ziyyara offers customized CBSE Board Home Tuition online, with experienced and certified tutors, providing personalized attention to each student.
    For more info contact +91-9654271931 or visit Online Tuition for CBSE Board