This article was originally published in 2015.09.18

Continuous Integration with UrbanCode Deploy and IBM Business Process Server

I recently had the chance to put together a demo for  IBM Business Process Manager which inspired me to share the results in this blog post. Before I continue, I wanted to set the context because BPM can cover many things as IBM has quite a rich product set in this area. In this post I will cover the continuous integration life cycle of IBM Integration Designer project builds into IBM Business Process Servers, not the IBM Business Process Center/Business Modeler-type deployments.

Overview

First, I'll cover some of the products that are being leveraged in this demo scenario. I have already mentioned some but to be more specific:

• IBM Integration Designer (previously known as WebSphere Integration Developer) is "an Eclipse-based software development tool that renders your current IT assets into service components for reuse in service-oriented architecture (SOA) solutions.". In simpler terms, it takes a bunch of business workflows, interface mappings and integration points described in XML and written in Java that can be deployed into a IBM Business Process Server. It also comes with an integrated Business Process Server test environment that allows a developer to deploy a project locally for development and testing purposes from the IDE. From within this environment, a developer can also export an ear file build that gets deployed to a Business Process Server.

• IBM Engineering Workflow Management (previously Rational Team Concert) In a nutshell: integrated source configuration, work item, tracking and planning and build management. RTC integrates with IBM Integration Designer so that these capabilities are part of thedevelopers working environment. From here, the IID developer, can check in those workflows, mappings, interface definitions and other code, start builds (either personal or team) and collaborate with other developers through work items in an all-in-one integrated experience.

• UrbanCode Deploy (now DevOps Deploy) Considering that this blog finds its home in the UrbanCode Blog, I won't waste too many words about its functionality as the multiple blogs by my colleagues do a great job already. If you're new to this product, this page gives an excellent description. For the purposes of the demo scenario, I'll be using the WebSphere Application Server - Deployment Plugin and the built-in integration with RTC's build system to provide automated deployment to IBM Business Process Servers (which are essentially WebSphere Application Servers with Business Process Management capabilities added).
• IBM Business Process Manager Advanced (BPM) (includes IBM Process Server; formally known as WebSphere Process Server) This contains the IBM Business Process Server that takes artifacts authored using IID and brings them to life. Any application (EAR) file that has been build using IID can be deployed to the IBM Business Process Server and it will start the required interfaces and bind the integration points as described by the workflows designed in IID. We will use UCD in combination with RTC to automate this process. The relationships between the products in a topology would look something like this:

 Product Roles
To make the demo realistic, I wanted to create a setup that somewhat reflected reality so I start out by setting up several BPM environments: Development (DEV), Integration (INT), Quality Assurance (QA), and Production (PROD). There are teams responsible for developing the code base that gets deployed onto these BPM servers. That is, they each have their own source code stream in RTC (besides PROD which receives build via promotion from other environments).

Environments and Teams The teams and environments are described as follows.

• Development: This is representative of one more teams, each being principally responsible for the development of a product component (sometimes called a component team). The Development team's team builds get deployed into the DEV environment.
• Integration: This team is responsible for maintaining a clean status of the integration build, the build that is backed by an integration stream (INT) that contains all the changes from all development teams. As there is the possibility of merges occurring, especially in a BPM scenario, the integration team is responsible for working across the development teams to ensure that merges occur cleanly.The INT BPM Server contains the build which integrates all teams’ builds prior to quality assurance subsequent to production.
• Quality Assurance: Integration Builds will be going through the rigor of QA testing and may contain additional artifacts (translation, legalese). There is an additional quirk in that a QA build can contain fixes performed by the Quality Assurance team that have not yet been integrated into development (now that's agile!). This is why QA also has an RTC stream and build.
• Production: A production server heavily wrapped in red tape: approvals and gates. There is no RTC stream, as this is primarily a release engineering team.

For this blog post, I’ll demonstrate the end-to-end continuous integration scenario for deploying a DEV build, so we’ll just be dealing with the Development (DEV) stream but you will see the other streams, environments and teams reflected in the setup and the screenshots.

Setup

Describing the setup in some detail goes a long way to explaining the scenario, so let’s dive right into it. You can follow along but you will need to already have the IBM Business Process Manager Advanced and IBM Integration Designer products. As they are no publically downloadable evaluations or trails you’ll need to contact an IBM Sales representative to arrange one or already be entitled. There is, however, evaluations of UrbanCode Deploy server and Rational Team Concert for which I provide the links for below.

• UrbanCode Deploy Server I downloaded and installed a UCD Server on a server machine. If you don’t have UCD, you can fetch a trail version from here. If you are not sure about how to install UCD, you can find documentation here. There is nothing special about the install itself, I installed a versioning UCD agent on the same machine although it’s unused when using the RTC build system integration as builds are pushed from RTC’s build engine (Jazz Build Engine/JBE) and do not need to be pulled from UCD. (Pulling versions requires a versioning agent.) I install it anyway to get rid of the warning on the UCD page about a missing versioning agent but this is optional, you can simply ignore the warning.
• UrbanCode Deploy WebSphere Deploy Plugin I downloaded the WebSphere Application Server - Deployment plugin, as for all intents and purposes installing an EAR file into IBM Process Server is identical.
• RTC Server installation I downloaded and installed RTC on a machine. If you don’t have RTC, you can download a version that’s free for 10 developers. If you are not sure about how to install and set up RTC, you can find documentation here. Remember to set the public URL of the RTC to something that is accessible from the UCD server and from the developer’s workstation, which will have IID with the RTC client integration.
• IBM Business Process Manger Advanced Servers
  1. I installed IBM Business Process Manger Advanced on all machines which constitute the environments (DEV, INT, QA, PROD) that will receive builds to deploy.  I Installed the BPM into C:\IBM\WebSphere\AppServer (as I was using a Windows machine)
  2. The UrbanCode Deploy Agent was installed on each of the BPM and connected them to the UrbanCode Deploy Server. This will allow the ear files built by the RTC Build Engine to be deployed into BPM.
  3. We also need to install the RTC Build System Toolkit on these machines. You can download the RTC Build System Toolkit here; it will need to be connected to the RTC server on start-up. As I installed BPM on Windows machines I created a build directory in C:\build.
  4. Build Directory The startJBE.bat just contains the following line as I unzipped the Build System Toolkit into the root of C, https://demo:10443/ccm is the public url of my RTC server and I am using a demo user to connect. (You’d normally want to create a build user for these purposes.) I created a build engine definition in RTC called dev so that the build engine knows which RTC build engine definition it attached to. (More on this later). Pass.txt contains the encoded password for the demo user. C:\jazz\buildsystem\buildengine\eclipse\jbe.exe -repository https://demo:10443/ccm -userId demo -passwordFile pass.txt -engineId dev (It should be noted that these machines are great candidates for a Blueprint in UrbanCode Deploy Blueprint designer.)
  5. Development Workstation The developer workstation has the IBM Integration Designer with the RTC capabilities installed via the the RTC p2 Install Repository. You can use the Install repository as a local updates site to install RTC from.

Product Configurations

Rational Team Concert

The idea is to set up a project area and several teams which are responsible for maintaining a team stream (responsible for development of some aspect of the system being deployed). Each team has a team lead and there could also be several other developers working on the teams. Each team member is responsible for ensuring that their team stream adopts changes from other teams via the integration stream and at the same time delivers changes into the integration stream. In doing so, they also need to ensure that they don’ break the team and integration builds. The team lead monitors the team build to ensure that the build and the deployment are green but they also need to ensure the integration stream build and deployment are green. The integration team would be responsible for making sure all changes from team streams delivered into the integration stream doesn't break the build. A typical DEV team flow would look something like this:

 Team Delivery In order to shorten an already lengthy post, I will not go through such a large scenario in this blog and concentrate on this one team, DEV though I have set up the structure to respect the work flow depicted. Generally, the same pattern can be applied to the other teams and builds. Check the RTC Knowledge Center if you don’t know the details of how to perform these operations. These are the things that I did to get the basic flow working:

1. As the RTC administrator user, created some other users in RTC: DEV Developer (devdeveloper), and DEV Team Lead (devlead).
   
2. Launched IBM Integration Designer and created a connection to RTC using the administrator (demo in my case) in the Work Item perspective.
3. Create a project area called Demo using the Scrum Process and assigned the administrator user as an administrator of the project and to the Product Owner role. I also assigned the DEV Team Lead (devlead) as a project Team Member.
4. I created a DEV team , assigned DEV Team Lead as the Scrum Master and the DEV Developer as a Team Member
   
   

   
5. In preparation for importing the IID sample project, I created Release 1.0 stream owned by the DEV team with four components:
   1. Library -> Contains the IID sample library project
   2. Mediation -> Contains the IID sample Mediation Project
   3. Service -> Container the IID Sample Service Project
   4. Release Engineering -> Contains the build files

We now have a basic stream setup and we should import some code. I switched to a new IID workspace and logged in as the DEV Team Lead user (devlead). Then I

1. Created a workspace off of the DEV team’s Release 1.0 stream with team visibility (scope) with all components.
   
2. Imported the IID Hello World sample project completed artifacts which can be imported via the Help->Samples and Tutorials -> IID Integration Designer menu.
   
   Created a Release Engineering Eclipse project in the Java perspective with the following structure and these two files. These will be the files used by the build, the build.xml is used to communicate with RTC, which the iidBuild.xml uses BPM server service deploy tags to build the ear file. I right clicked on the project context menu, Team->Share Project… and added the Release Engineering Project to the Release Engineering Component in Jazz SCM.
   
3. Switched to the Business Integration perspective, right clicked on the project context menu, Team->Share Project and added the sample:
   
   1. HelloService to the Service Component
   2. HelloWorldLibrary to the Library Component
   3. HellowWorldMediation to the Mediation Component
   4. Delivered into the Release 1.0 (DEV) stream using the Pending Changes view
      

Once the code has been added, we can now create the build engine and definitions. To make things easier, I switch back over to the demo IID workspace as this is also our build user. Then I:

1. Created a build engine called dev which the build engine on the dev BPM machine will bind. There are several properties that are used by the ant scripts that should be defined here:
   

   jazzBuildTookit	C:\jazz\buildsystem\buildtoolkit
   jbePasswordFile	C:\build\pass.txt
   jbeProperties	C:\build\jbe-properties\${buildLabel}.properties
   jbeRoot	C:\build
   jbeWorkspaceRoot	C:\build\workspaces
   wps.home	C:\IBM\WebSphere\AppServer

2. We’ll create a build definition call DEV Service Build owned by the DEV team area using
   1. the Ant - Jazz Build Engine template
   2. The Jazz Source Control pre-build step
   3. Post-Build Deploy post-build step
3. In the DEV Service Build Overview tab, add dev as the Supported Build Engine
4. In the Properties tab set the following properties
   

   build.output.dir	${jbeRoot}/builds
   build.working.dir	${jbeRoot}
   componentVersion	${stream}-${buildLabel}
   library	HelloWorldLibrary
   module	HelloService
   stream	DEV
   team.udeploy.debug	false
   team.udeploy.timeout	480

5. In the Jazz Source Control tab
   
   1. Create a workspace that flows from the Release 1.0 (DEV) stream called Release 1.0 DEV Service Build that is DEV team scoped and contains all the components.
   2. In the Load Options, set the Load directory to ${jbeWorkspaceRoot}/${team.scm.workspaceUUID}
   3. Select Delete directory before loading and all the Accept Options
6. In the Ant tab
   
   

   
   1. set the Build file to ${team.scm.fetchDestination}/Release Engineering/ant/build.xml
   2. Set the check mark to include the Jazz build toolkit tasks
   3. Set the Properties file to ${build.output.dir}/${componentVersion}/jbe.properties
7. In the Post-build deploy tab
   
   

   
   

   
   1. set the Enable flag and deploy if build has no errors or warnings
   2. Do not set deploy for personal builds
   3. Set the Server URI of the UCD server. This is the URL you use to access the UCD web interface
   4. Set a user name, the user needs to have appropriate access to the resource in UCD related to deploying the service component to the DEV environment. For now we can use the UCD admin user. This can be changed later. Test the connection to ensure connectivity.
   Set these properties
   1. Component: ${module}
   2. Version: ${componentVersion}
   3. Base Directory: ${build.output.directory}
   4. Include Files: ${module}.ear
   5. Links: RTC Build Result=${repositoryAddress}resource/itemOid/com.ibm.team.build.BuildResult/${buildResultUUID}
   In the Process Section Enable “Run Application Process” and set
   1. Application: HelloWorld
   2. Environment: ${stream}
   3. Process: Deploy
8. You should now duplicate this build definition and call it DEV Mediation Build. Everything should be kept identical except:
   1. Create and use a new workspace called Release 1.0 DEV Mediation Build that also flows from the Release 1.0 (DEV) stream that is DEV team scoped and contains all the components.
   2. Set the module property in the Properties tab to HelloWorldMediation

At the end of this your project tree should looks something like this:

Make sure not to forget to start the RTC build engine on the BPM DEV machine after which you should be able to create builds in RTC but unless we create the artifacts in UCD that are used by RTC, it won’t deploy.

UrbanCode Deploy

UCD will be much simpler to setup as I have exported the entire application. The only thing you need to make sure is that you have loaded the WebSphere Application Server –Deployment Automation Plugin in the UCD Settings before you attempt to import it or continue. We’ll need some of the resource roles provided by the plugin in order to set up the resource tree in UCD to reflect the BPM deployments. If the agents were installed correctly they should already be in the Agents tab under Resources.

1. In the Components Tab Create two components called HelloService and HelloWorldMediation, leaving all the defaults. These are just placeholders for now.
   
2. In the Resource tree
   1. Create a folder called DEV Servers and add the DEV BPM server agent to the folder. This should create a new resource bound to the agent. If you want to keep things simple, rename the resource (not the agent) to dev.example.com as it will already be bound to the environment when you do the application import. (Otherwise you’ll have to bind it yourself)
   2. Click Show->Resource Roles to bring up the resource roles panel and drag WebSphere Cell under dev.example.com and rename it to PSCell1 (the default for BPM)
   3. Modify the entries in the tree to facilitate this path, these are the defaults unless you changed them during BPM install and configuration, in which case use the value from the DEV environment BPM. /DEV Servers/dev.example.com/PSCell1/Nodes/Node1/Servers/server1/
   4. Add the HelloService and HelloWorldMediation components directly under /DEV Servers/dev.example.com/PSCell1/Nodes/Node1/Servers/server1/
   When finished the resource tree should looke like:
   
3. At this point is probably worthwhile to define a resource template for the agent so that when configuring the resource tree for other agents, the template can be implied instead of redoing the entire tree. (Unless the cell, node and server names are different for all your servers in which case a template may not be as useful.)
4. Finally, import the HelloWorld.json Application in the zip file and selecting “Upgrade Component If Exists”.
   
5. Feel free to examine the component processes in UCD and the environment, most of the heavy lifting is done by the WebSphere Plugin and all we do is provide bindings between the RTC build output and BPM. The last thing is to create users for the

DEV Team Lead

1. and

DEV developer

1. in urbancode deploy. As I did in RTC, I also created team corresponding to the environments I then assigned

DEV Team Lead

1. and

DEV developer

1. to the

Development

1. team using the pre-packaged manager and developer roles. I then assigned the relevant objects using the team object mappings using the Standard type.

Agent:dev.example.comApplication:HelloWorldComponent:HelloService, HellowWorldMediationEnvironment

1. :

HelloWorld/DEVResource:DEV Servers/** (the entire resource tree)

1. That should do it; you now have a continuous integration pipeline! To see it in action kick off a build in RTC, this should succeed in create a build record linked to the version that is uploaded to UCD. This version will be deployed using the application process for the specific environment bound to the BPM server. Checkout the following screenshots to see it in action.

Write a comment @mention users to notify them.

#BusinessProcessManager(BPM)
#UrbanCodeDeploy