Thursday, 18 June 2015

Build a Cloud Foundry Java Appllcation: IBM Bluemix DevOps Services!

In the previous posting I created an java application in Cloud Foundry that I already prepared (built and packaged) locally beforehand. A good PaaS would allow me to edit, build and deploy right from the platform itself. With some technologies, like Node.js, this is very easy as the files do not need to be compiled but as we also wish to show some build integrations so we'll continue with Java.

IBM BlueMix provides these edit, build and deploy SaaS capabilities called IBM Bluemix DevOps Services. It provides integrated source code version control (git) , a source code editor (orion), tracking and planning and build and deploy services. The deploy stage can deploy into the Cloud Foundry upon code change in source code repository.

There are other ways to do this in an automated fashion outside of IBM Bluemix, build processes can be on local bare metal servers or using IaaS with IBM UrbanCode Deploy. I'll will cover how to use IBM UrbanCode in later postings where I use BlueMix IaaS to set up my build environment.

In the meantime, I am going to set up IBM Bluemix DevOps services to build our sample java application.
  1. To add a git repository to our new application, in Bluemix we navigate to the boriscloud application from the Bluemix dashboard
  2. And click the "Add GIT" button:
  3. Continue through the dialogues that pop up, after which a new git repository link should be added to the project.
  4. Now we can connect to the git repository using the git command line and add our files

    > git clone https://hub.jazz.net/git/kuschel/boriscloud
    Cloning into 'boriscloud'...
    Username for 'https://hub.jazz.net': <Bluemix username>
    Password for 'https://<Bluemix username>@hub.jazz.net':
    remote: Counting objects: 4, done
    remote: Finding sources: 100% (4/4)
    remote: Total 4 (delta 0), reused 4 (delta 0)
    Unpacking objects: 100% (4/4), done.
    Checking connectivity... done.
    
    
  5. Set the remote repository URL for push/pull

     > git remote set-url origin https://hub.jazz.net/git/kuschel/boriscloud
    
    
  6. Commit the boriscloud war source code

    > git commit
    [master cdb0a4d] Initial Commit
     6 files changed, 130 insertions(+), 1 deletion(-)
     create mode 100644 BorisCloud.war
     delete mode 100644 License.txt
     create mode 100644 WebContent/META-INF/MANIFEST.MF
     create mode 100644 build.xml
     create mode 100644 manifest.yml
     create mode 100644 src/com/boriskuschel/cloud/SampleServlet.java
    
    
  7. Push the changes to Bluemix

    > git push
    Username for 'https://hub.jazz.net': <Bluemix username>
    Password for 'https://<Bluemix username>@hub.jazz.net':
    Counting objects: 2, done.
    Delta compression using up to 8 threads.
    Compressing objects: 100% (2/2), done.
    Writing objects: 100% (2/2), 353 bytes | 0 bytes/s, done.
    Total 2 (delta 1), reused 0 (delta 0)
    remote: Resolving deltas: 100% (1/1)
    remote:
    remote: Processing changes: refs: 1, done   
    To https://hub.jazz.net/git/kuschel/boriscloud
       cdb0a4d..c6b8136  master -> master
    
    
  8. To merge the local repository and changes that were committed to the Bluemix Git using it's git editor:

    > git pull
    Username for 'https://hub.jazz.net': <Bluemix username>
    Password for 'https://<Bluemix username>@hub.jazz.net':
    remote: Counting objects: 9, done
    remote: Finding sources: 100% (7/7)
    remote: Total 7 (delta 1), reused 7 (delta 1)
    Unpacking objects: 100% (7/7), done.
    From https://hub.jazz.net/git/kuschel/boriscloud
       c6b8136..d42dd0d  master     -> origin/master
    Merge made by the 'recursive' strategy.
     .cfignore    | 2 ++
     .gitignore   | 1 +
     project.json | 1 +
     3 files changed, 4 insertions(+)
     create mode 100644 .cfignore
     create mode 100644 .gitignore
     create mode 100644 project.json
    
    
  9. To get to the git editor click the "Edit code" button. Form here you can edit files, commit them and most of the common git operations.

  10. Press the second icon on the left bar to get to the git operations view, press the "sync" button to merge the local repository with any incoming changes that were committed using "git push" command.
  11. Once all the files have been added a corresponding build.xml file was also added. This will be used from the build. Click on the "Build & Deploy" button.
  12. Click on the build state. By default, the build will occur whenever new changes are pushed to git.

  13. Configure the Build Stage to use ant and the build.xml file.

  14. Set the Deploy Stage to deploy to cloud foundry.

  15. After the build and deploy is completed it should look like this:

That's it, how we have a slick DevOps operation for our sample java application.

Wednesday, 17 June 2015

Publishing Java Applications with Cloud Foundry and IBM Bluemix

We'll be using the the java_buildpack to install a war file, preferable we could of used the Liberty for Java buildpack (liberty-for-java) but as this post should also work with non IBM Bluemix Cloud Foundry instances, we'll use the standard Java buildpack instead.

I am also going to be hosting this application at boriscloud.boriskuschel.com so I'll be setting up some routes.

First I am going add the boriskuschel.com domain to the "Cloud" space.

> cf create-domain "Boris's Cloud Blog" boriskuschel.com
Creating domain boriskuschel.com for org Boris's Cloud Blog as <Bluemix username>...
OK

Download and unzip this file and examine the manifest.yml. Manifest files make deploying into Cloud Foundry easier.

---
applications:
- name: boriscloud 
  memory: 512M
  instances: 1
  path: BorisCloud.war
  buildpack: java_buildpack

The java buildpack with use Tomcat to start the servlet. If you wish to use the WebSphere Liberty server specify:

  buildpack: liberty-for-java

This will use liberty instead of Tomcat.

The rest of the manifest options can be left as default.

By default, when we push the application it will be route the application to <application name>.mybluemix.net but as we want to bind it to boriscloud.boriskuschel.com we will specify the boriskuschel.com domain on the command line using the -d flag.

From within the unzipped boriscloud directory run:

> cf push -d boriskuschel.com
Using manifest file /home/kuschel/boriscloud/manifest.yml

Creating app boriscloud in org Boris's Cloud Blog / space Cloud as <Bluemix username>...
OK

Creating route boriscloud.boriskuschel.com...
OK

Binding boriscloud.boriskuschel.com to boriscloud...
OK

Uploading boriscloud...
Uploading app files from: /home/kuschel/boriscloud/BorisCloud.war
Uploading 7.1K, 9 files
Done uploading               
OK

Starting app boriscloud in org Boris's Cloud Blog / space Cloud as <Bluemix username>...

0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
1 of 1 instances running

App started

OK

App boriscloud was started using this command `JAVA_HOME=$PWD/.java-buildpack/open_jdk_jre JAVA_OPTS="-Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh -Xmx382293K -Xms382293K -XX:MaxMetaspaceSize=64M -XX:MetaspaceSize=64M -Xss995K -Daccess.logging.enabled=false -Dhttp.port=$PORT" $PWD/.java-buildpack/tomcat/bin/catalina.sh run`

Showing health and status for app boriscloud in org Boris's Cloud Blog / space Cloud as <Bluemix username>...
OK

requested state: started
instances: 1/1
usage: 512M x 1 instances
urls: boriscloud.boriskuschel.com
last uploaded: Thu Jun 18 02:07:22 UTC 2015
stack: lucid64

     state     since                    cpu    memory           disk           details   
#0   running   2015-06-17 10:10:18 PM   0.0%   124.5M of 512M   125.6M of 1G      

Let's check out the routes.

> cf routes
Getting routes as <Bluemix username> ...

space   host         domain             apps   
Cloud   boriscloud   boriskuschel.com   boriscloud   

This presents a small issue, I'll need to update my domain's DNS server with a new CNAME record that points to boriscloud. So, our application will need to be put up on a domain that is owned by Bluemix so I can point to it with the CNAME record. I'll use the mybluemix.net domain that is owned by Bluemix.

> cf map-route boriscloud mybluemix.net -n boriscloud
Creating route boriscloud.mybluemix.net for org Boris's Cloud Blog / space Cloud as <Bluemix username>...
OK
Adding route boriscloud.mybluemix.net to app boriscloud in org Boris's Cloud Blog / space Cloud as <Bluemix username>...
OK

> cf routes
Getting routes as <Bluemix username> ...

space   host         domain             apps   
Cloud   boriscloud   boriskuschel.com   boriscloud   
Cloud   boriscloud   mybluemix.net      boriscloud   

That's better, now we can access the application from http://boriscloud.boriskuschel.com/test (and http://boriskuschel.mybluemix.net/test).

Cloud Foundry with Bluemix

Cloud Foundry is an open source Platform as a Service (PaaS) that lets you quickly create and deploy applications on the cloud.  (For those familiar with JEE, I like to think of it as a next generation application server with built-in multi-tenancy support for different languages and service bindings.)

As Cloud Foundry is open source and does not rely on proprietary software or cloud infrastructure, it is completely possible to set up a Cloud Foundry  client locally to experiment with Bluemix. That’s exactly what we will do later in this posting to get familiar with it.  There is nothing better than learning by doing.

Orgs, Spaces, Roles, and Permissions

Cloud Foundry is designed to support multiple developers in isolated spaces within orgs (organizations) to push applications into.

There is a user role within Cloud Foundry (OrgManager) that allows users to create orgs. When using Bluemix, this role is assigned to Bluemix administrators and, by default, the Bluemix user does not have permissions to create or delete them however the user is the administrator of the org itself, so it is possible to create/delete/modify spaces within it. By default, the name of the org is the user name. It is possible however to rename the org.

Every application and service is scoped to a space. A space provides a set of users access to a shared location for application development, deployment, and maintenance. Each org contains at least one space and it's called "dev", by default, in Bluemix.

Let's examine Bluemix with the Cloud Foundry client.:
  1. Go to https://github.com/cloudfoundry/cli/releases and download and install the appropriate release for your platform.
  2. It is now possible to type:

    cf help
    
    to get a full listing of possible commands and:

    cf <command> -h
    
    or:

    cf help <command>
    
    to get help on a specific commands.
  3. Run the following command which sets the API endpoint, or target URL, of the Cloud Foundry instance's Cloud Controller URL. The Cloud provider, in this case, Bluemix, typically supplies this.

    > cf api https://api.ng.bluemix.net
    Setting api endpoint to https://api.ng.bluemix.net...
    OK
    
                       
    API endpoint:   https://api.ng.bluemix.net (API version: 2.23.0)   
    Not logged in. Use 'cf login' to log in. 
    
  4. As suggested we will now login into Cloud Foundry. The "-u" option is the username.

    > cf login -u <Bluemix username>
    API endpoint: https://api.ng.bluemix.net
    
    Password> 
    Authenticating...
    OK
    
    Targeted org <Bluemix username>
    
    Targeted space dev
    
    
                       
    API endpoint:   https://api.ng.bluemix.net (API version: 2.23.0)   
    User:           <Bluemix username>   
    Org:            <Bluemix username>   
    Space:          dev
  5. As mentioned previously, the default organization is the username with a default space called "dev". Let's rename the org to something more meaningful. For example
    > cf rename-org <Bluemix username> "Boris's Cloud Blog"
    Renaming org <Bluemix username> to Boris's Cloud Blog as <Bluemix username>...
    OK
If you login to Bluemix you will see that the org has been renamed on the Dashboard.


We'll also create a new space for our work.

  1. Create a new space called "Cloud"
  2. > cf create-space Cloud
    Creating space Cloud in org Boris's Cloud Blog as <Bluemix username>...
    OK
    Assigning role SpaceManager to user <Bluemix username> in org Boris's Cloud Blog / space Cloud as <Bluemix username>...
    OK
    Assigning role SpaceDeveloper to user <Bluemix username> in org Boris's Cloud Blog / space Cloud as <Bluemix username>...
    OK 
    
    
    TIP: Use 'cf target -o Boris's Cloud Blog -s Cloud' to target new space 
     
  3. Target the new space, this will now be used for applications
    > cf target -o "Boris's Cloud Blog" -s Cloud
                       
    API endpoint:   https://api.ng.bluemix.net (API version: 2.23.0)   
    User:           <Bluemix username>   
    Org:            Boris's Cloud Blog   
    Space:          Cloud  s
In Bluemix, this new space is also now available for deploying applications:

Components

Cloud Foundry consist of several components: routing, authentication, application lifecycle storage and execution, messaging (for inter-component communication), metrics, logging and service brokers that provides service instances when developers bind those services to applications. (Services can be provided using a SaaS model).


Developing and Deploying Applications

Applications can be built with different programming languages and technologies, Cloud Foundry has the concept of buildpacks that provide support for them. Let's see what buildpacks are available on IBM Bluemix.

> cf buildpacks

Getting buildpacks...

buildpack                            position enabled locked filename  

liberty-for-java                     1        true    false  buildpack_liberty-for-java_v1.19-20150608-1717-yp.zip
sdk-for-nodejs                       2        true    false  buildpack_sdk-for-nodejs_v2.0-20150608-1503-yp.zip
noop-buildpack                       3        true    false  noop-buildpack-20140311-1519.zip  
java_buildpack                       4        true    false  java-buildpack-v2.7.1.zip  
ruby_buildpack                       5        true    false  ruby_buildpack-cached-v1.3.0.zip  
nodejs_buildpack                     6        true    false  nodejs_buildpack-cached-v1.2.0.zip  
go_buildpack                         7        true    false  go_buildpack-cached-v1.2.0.zip  
python_buildpack                     8        true    false  python_buildpack-cached-v1.2.0.zip  
php_buildpack                        9        true    false  php_buildpack-offline-v3.1.0.zip  
aspnet5-experimental                 10       true    false  buildpack_aspnet5-experimental_v0.1.5-20150508-1205.zip
liberty-for-java_v1-18-20150519-1642 11       true    false  buildpack_liberty-for-java_v1.18-20150519-1642-yp.zip
sdk-for-nodejs_v1-18-20150519-1759   12       true    false  buildpack_sdk-for-nodejs_v1.18-20150519-1759-yp.zip  
Different buildpacks have different functionalities specific to the development technology support being provided. For example, the Java buildpack support includes support for Java Main,  Servlets, Spring Boot CLI and Play Framework. When "pushing" an app to Cloud Foundry, it will examine the contents to determine which build pack should be used. The buildpack can also be specified on the cf command line when pushing the application, this is how custom/community buildpacks can be used.

In the next blog entry, we'll create an app using the Java buildpack and deploy it to the new "Cloud" space we created.

For More information on these topics visit:

http://docs.cloudfoundry.org/devguide/
https://www.ng.bluemix.net/docs/starters/byob.html
https://github.com/cloudfoundry/java-buildpack

Tuesday, 16 June 2015

A cloud of acronyms and terminology

Anybody getting their feet wet with Cloud technologies will discover something very quickly: It's a jungle or rather a bazaar of multiple projects with technologies doing the similar things, overlapping in some places and unique in others. It can be quite a mess to figure out, so I’ll try to cut through some of the technobabble.

First, there are few things that "cloud" means and a few areas it touches but it all falls under the umbrella of hosting. Basically, somebody/something is putting ("provisioning") a server, machine (real or virtual), application, services or a combination of all them somewhere that you use remotely over the internet. That's about it. Just imagine your typical traditional office network with all of those servers tied together using Ethernet cables and routers accessible over the internet. Each server has its own application or services that you can access and configure remotely. Yes, that's about it.

So what's the big deal?

Imagine that each time you wish to set up a network environment you have to create a network topology diagram (a specification), hand it off to IT technicians who stands up all the servers, connect them all together and installs all the middleware and application in your specification. If it's anything more than one server, it could take weeks if not months.

Now here it the big deal: With cloud technologies, you can do this in minutes (well, ok, hours) in an incremental, repeatable self-documenting fashion. Cloud will make developers better because it addresses a fundamental characteristic of good software development:  laziness. Wait, what?  Lazy, how can that be good?

From Larry Wall’s “three great virtues of a programmer” laziness is “The quality that makes you go to great effort to reduce overall energy expenditure.” That is exactly what cloud technologies do for a developer, reduce overall energy expenditure, in other words, drive efficiency.

Being able to stand up a whole network, with services and applications in a matter of minutes or hours opens up a whole new world of software development and application deployment efficiencies. Network topologies can be built up, updated and destroyed with different version combinations to ensure that software on all supported permutations and it call be automated. That is, no humans need to do that work.

Topology configurations can be stored versioned, archived, application and services dynamically bound before being actualized.
As you can imagine, all of these also comes with truck load of provisioning, versioning, configuration and development technologies and projects all champing at the bit to get a piece of the action.

So how do we disseminate all of this? First, it’s important to define levels of service that Cloud can provide and the primary consumer of those services.

“The Application”:  Software as a Service (SaaS)

The oldest and most familiar model, an application is hosted on the web (a cloud) and made available on-demand for customers using a subscription model.  All the data is typically added to the host and is accessible using VPN.

Using the traditional desktop office productivity sweat such as Microsoft Office as an example, one would, pay a software license fee, install it on every workstation that is used and maintain it (apply fix packs and updates).

With SaaS, one pays a subscription which usually contains a certain amount of functionality and resource allocations (disk space, etc.) and the rest is done by the cloud provider.

Development of The Application: Platform as a Service (PaaS)

There is a category SaaS related to the development, running and management of SaaS offerings. This could be called Software Development Software as a Service but is actually called Platform as a Service as it also contains SaaS offering not directly related of software but in support of (work item management, reporting metrics, etc.).

So using an office productivity suite as an example, it should be possible to use something PaaS offering to develop it. It should provide all the tools needed by a software developer (a particularly lazy one) to create a SaaS offering, the PaaS cloud provider usually also provides hosting services for your Saas offering.

A PaaS offering can be found in IBM BlueMix that uses Cloud Foundry technology.  This will be the first technobabble checkpoint:: Cloud Foundry. My next blog posting will be about that and related technologies.

In preparation head over to IBM Bluemix and sign up for a free account here: https://console.ng.bluemix.net/.

Welcome!

With IBM's new focus on Cloud; I have taken a new role as an Unleash The Labs engineer. It's similar to the previous role I had when I last contributed to my previous blog at blog.boriskuschel.com but as it is a new subject matter, I thought would be a good idea to start a new blog.

Welcome!