View Only

Jenkins Pipeline Plug-in Tutorial: Component Version Import and Snapshot Deployment

By Osman Burucu posted Tue July 12, 2022 05:54 AM

Like

This article was originally published in 2017.07.11 and Updated: Jan 2024

Jenkins Pipeline Plug-in Tutorial: Component Version Import and Snapshot Deployment

Over the years, Jenkins has emerged as one of the most popular build tools. The Jenkins Pipeline since version 2.0 provides a solid continuous integration approach for simple application test environments. However, once users reach full-scale production deployments, they begin to realize its limitations. While strong individually, Jenkins does not scale to coordinate and deploy multiple services and components. This limitation is where IBM UrbanCode Deploy (now IBM DevOps Deploy) excels. IBM UrbanCode Deploy enables users to develop a highly customized deployment strategy that can harmonize with any number of microservices and environments. In these scenarios, users can continually and confidentially deploy to different environments and quickly identify the root issue when a deployment fails. Additionally, IBM UrbanCode Deploy provides a process record to remind the release team which deployments were successful, what sequence of processes ran, and the state of each environment and process. With this information, release managers can save configurations in the form of snapshots and return to successful states when deployments or configurations go wrong. In this tutorial, the following two scenarios are discussed to help you get started with UrbanCode Deploy and the Jenkins Pipeline.

Component Version Import: Build an artifact in Jenkins and push it into IBM UrbanCode Deploy as a component version
Snapshot Deployment: Deploy an IBM UrbanCode Deploy snapshot using a pipeline or freestyle Jenkins pipeline.

These scenarios walkthrough the most important use cases of the Jenkins Pipeline integration and how they can expanded for use in other deployment scenarios.

About the Jenkins Pipeline plug-in

The IBM UrbanCode Deploy Jenkins Pipeline plug-in imports Jenkins build artifacts and initiates IBM UrbanCode Deploy server deployments. The plug-in is installed on the Jenkins server and configured through Pipeline script syntax or form fields. The most common use case for this plug-in includes three basis steps:

Jenkins is used to clone a SCM repository and build new artifacts.
UrbanCode Deploy Pipeline plug-in pushes the artifacts into the IBM UrbanCode Deploy server as a new component version.
UrbanCode Jenkins Pipeline plug-in configures and triggers an application process in IBM UrbanCode Deploy using a component version or snapshot.

The first two steps are described in the Component Import scenario and the third step in the Snapshot Deployment scenario. The Jenkins Pipeline plug-in is similar to other source configuration plug-ins as it moves artifacts from a source control or build server into the IBM UrbanCode Deploy CodeStation. Once these artifacts are within the CodeStation, they can be used by any defined deployment to your configured environments. However, because the Jenkins Pipeline plug-ins are installed into the Jenkins server, not in the IBM UrbanCode Deploy server, the import process relies on a push rather than pull model. Importing on a push basis alleviates the IBM UrbanCode Deploy agents from extraneous work and wasted time when querying for new builds that do not exist. Most importantly, this approach allows for the immediate deployment following the artifact's creation in the build server. This minimizes downtime and greatly increases the value of any continuous integration pipeline. For these reasons a push approach when integrating with all build tools is best.

Scenario: Component Version Import

This scenario walks though the configuration of the Jenkins Pipeline plug-in and a simple component import use case. To follow along you must have an IBM UrbanCode Deploy and a Jenkins server reachable on the same network. If you do not have either server, you can use a provided Docker Compose file that can build the servers in a few minutes. There are also several scripts available to install the Jenkins plug-ins and generate generic Jenkins jobs. Integrating IBM UrbanCode Deploy and Jenkins using the Jenkins Pipeline plug-in is a good first step towards a continuous integration pipeline. In this walkthrough, we use Jenkins to build an open source plugin and push the compiled file into IBM UrbanCode Deploy as a component version. In the next scenario, we use the newly created component version to automatically deploy it to a sample Dev environment. Note: If this is your first time using the pipeline feature in Jenkins, as you follow this tutorial, referencing to the Jenkins Pipeline and the UrbanCode Deploy Jenkins Pipeline plug-in documentation is helpful. While the walkthroughs explain the sample configuration, some amount of personalized scripting is required for your specific continuous integration environment.

Walkthrough: Component Version Import

Start your IBM UrbanCode Deploy and Jenkins servers by doing one of the following:
- Run the docker compose file from GitHub to run simple IBM UrbanCode Deploy server, IBM UrbanCode Deploy agent, and Jenkins server Docker containers.
- Manually configure full IBM UrbanCode Deploy Server, Agent, and Jenkins server environments.
Install the Jenkins Pipeline plug-in into the Jenkins server, by doing one of the following:
- Run the installPlugins.sh script from the GitHub project.
- Install the Jenkins Pipeline plug-in manually and restart the Jenkins server. For more information, see Managing plugins >Installing plugins.
Set the IBM UrbanCode Deploy server configuration and authentication settings in Jenkins.
1. From the homepage click: Manage Jenkins > Global Settings
2. Scroll down to the IBM UrbanCode Deploy Pipeline Plugin Configuration section. Specify the necessary settings to configure an IBM UrbanCode Deploy server where the Jenkins artifacts are to be pushed.
3. After the settings are specified, click Test Connection to confirm that the IBM UrbanCode Deploy server can be reached. Additional UrbanCode Deploy configurations can be added by clicking Add.
4. Note: If using the Docker Compose files and templates from GitHub, you can use the same configuration settings as above. If you specify a different value for Profile Name differently, you must remember it for project configuration.
5. Save your new configuration settings.
Initialize build configuration settings if this has not already been done. This is where the build tool for our simple project is configured. This tutorial uses AntHillPro to build.

From the homepage click: Manage Jenkins > Global Tool Configuration
Scroll down until you locate the Ant section.
Specify the sample configuration as below. If you use another build tool, such as Gradle or Maven, they are initialized in a similar way.
Note: If you are using the templates from GitHub, ensure that the Ant configuration name is Default or remember the name for the project configuration.
Save your new build tool configuration.

At this point, all global configuration requirements are set. We can now create the Jenkins Project, set the specific plug-in properties, and import the build files into UrbanCode Deploy. If you have used the templates from GitHub, a Pipeline job should already exist in the Jenkins server. We will go in-depth into each of the Pipeline’s plug-in configuration examples and properties below.
1. Begin by clicking New Item in the upper left corner of the dashboard.
2. Name your new Job UCD-Pipeline and select the Pipeline job.
3. Click Ok
4. Next you will be taken the Job configuration page. Leave all properties as their defaults and scroll down to the Pipeline section.
5. In this text box we define the complete Jenkins Pipeline process for the UCD-Pipeline project. This example is divided into three parts:
  - Clone the source code
  - Build the project
  - Import of build artifacts into IBM UrbanCode Deploy
  Download the Jenkins Pipeline Script: Walkthrough 1 file and copy and paste it into the Pipeline property field.
  
  Clone
  The clone stage pulls the GitHub repository source code and place it into the job runtime working directory. The important properties to recognize are: class, branches, and userRemoteConfigs.
  - The class property specifies what type of repository we are trying to clone. In this case, we are pulling from GitHub, so it’s GitSCM.
  - The branches property specifies the which branch to pull from Git repository which here is the master branch.
  - The userRemoteConfigs property specifies the exact Git URL to clone. This project clones the IBM UrbanCode’s open source DataPower plug-in.
  Build
  The second stage is the Build process. We use a simple build process and therefore, it can be defined in two commands. The first adds the Ant’s installation directory to the PATH environment variable. This means, when we run the following ant command, the runtime environment knows where to look. Because we configured Ant (or another build tool) through the Global Tool Configuration, we can specify that configuration as a dynamic property: ${tool ‘Default’}. The installation folder is dynamically resolved at runtime based on the tool specified. When the PATH has been set, we can run the simple ant command and build our DataPower project. Note: If you defined a different build tool name replace Default with your build tool name and update the ant command with the appropriate call.
  Import
  The last stage and the most important is the import of these newly created artifacts to IBM UrbanCode Deploy. This last Pipeline script identifies the expected build artifacts and pushes them as an IBM UrbanCode Deploy component version. The class property defines the plug-in we are trying to use, similar to GitSCM and Default before, we are specifying the UCDeployPublisher plug-in, which is the UrbanCode Deploy Pipeline plug-in, that we wish to initialize. Similar to the other steps, the syntax varies from plug-in to plug-in and is important to full utilize the power of the IBM UrbanCode Deploy Pipeline plugin. This Pipeline script, and others, are explained in-depth in the Jenkins Pipeline plug-in documentation. For now, confirm you have specified the correct siteName for your IBM UrbanCode Deploy server and continue on to the next step.
Click Build Now from the Pipeline project's Overview page to clone, build, and import your new component version into your IBM UrbanCode Deploy server. If any steps fail, return to the Project Configuration and make the necessary updates.
On success, navigate to your IBM UrbanCode Deploy server, https://ucd-server:8443 if you are using default values. You should see a new component names UrbanCode Deploy Component named UCD - Pipeline.

This Jenkins build and all future builds will create component versions under this specified component.

Next Steps

For next steps, you can construct an automated deployment process using the new component version. This needs to be configured through the Pipeline syntax. The proper syntax is given in the Jenkins Pipeline documentation. Commonly, an automated process installs the artifacts into a TEST or DEV server for immediate feedback. For example, if importing new plug-ins, perhaps you kick off a process that installs this plugin into a separate IBM UrbanCode Deploy test server (https://github.com/IBM-UrbanCode/Plugin-Installer-UCD). Once the proper due diligence is complete, an administrator can then manually deploy and progress the artifacts through the other environments. Using additional Pipeline scripting, you can further automate your build and deploy process by constructing components from a template or assign them to an application. Note: If you open the UCD-Pipeline configuration, notice that the source configuration is not configured. Because artifacts are being pushed, instead of pulled into IBM UrbanCode Deploy, the component does not know how to retrieve additional artifacts. That logic, as it is preferred and explained above, is not included in IBM UrbanCode Deploy.

Freestyle Project

If Pipeline as Code is not your goal, this process can also be achieved through the Freestyle project. This is a good approach if you do not expect your deployment scenario to frequently change or a first time Jenkins user. Because you have already pre-configured many of the configuration (UrbanCode Deploy Server, An, etc), I will touch on many of the values. The new Freestyle Jenkins project, like the Pipeline project, it is divided into the same three stages: Clone, Build, and Import.

Clone

First, we need to clone the repository to the working directory. The below Source Code Management configuration will retrieve the same DataPower plug-in GitHub repository and place it on the Jenkins server. Add a GitHub project name, such as the demo project: DataPower Configuration Manager (DCM), as the Repository URL parameter. All other source configuration parameters can remain as the default.

Build

Next, we will use the build tool (Ant) which we configured in step 4 above. Find the Add build step button and specify the Ant Version created in Jenkin’s Global Tool Configuration settings.

Import

The last piece is where we will use the Jenkins Pipeline plug-in as a build step where all parameters can be given through a simplified form. The below configuration screenshot mirrors the Pipeline script that we used before. I highly recommend you compare the two configuration methods to better understand the Pipeline configuration settings. Like the previous Pipeline project, save this configuration and Build the project to see a new Component Version Import in IBM UrbanCode Deploy.

Scenario: Snapshot Deployment

Now that we understand how to import newly built artifacts into IBM UrbanCode Deploy, we can then deploy them into different environments. As outline above, this would be the third part of our standard use case. However, the simple deployment has already been outlined in the Jenkins Pipeline plugin Usage section. This walkthrough builds on the previous walkthrough by taking it one step further to deploy a snapshot. In this scenario, we want to test the upgrade process for our deployable artifacts. We know how to get artifacts into IBM UrbanCode Deploy from the above walkthrough and how to perform a standard deployment process. With this simple process, you can deploy the new component versions into a Dev environment. But what if you wanted to create a repeatable and testable environment. What if you want to quickly spin up an pre-configured environment for someone else to test against. Snapshots enable a deployment to rollback or reconfigure to a known state. The baseline deployments allows us to initialize our testing environment with a confirmed state on every run. In this walkthrough, we use a new Jenkins project to deploy a manually created snapshot into our Dev environment. A snapshot is a static record of the application, component versions, environments, and application process used to create the current deployed version. Snapshots are paramount in quickly performing rollbacks and configuring standardized environments. If used appropriately, automated tests can be combined with a snapshot to test various deployments against a standardized environment. Then, once the environment needs to be wiped or reset, simply deploy the snapshot again and your environment will be back to its baseline state. Our largest IBM UrbanCode Deploy users tend take full advantage of snapshots in their production deployments. If snapshots are a new concept to you, review the Creating and Deploying Snapshots tutorial. This tutorial guides you through a realistic scenario of creating a snapshot, configuring the environments, and deploying the snapshot using the user interface. Understanding how a snapshot fits in the context of the IBM UrbanCode Deploy helps your understanding of this walkthrough. The Snapshot Deploymen walkthrough assumes you have a snapshot configured and available to deploy to an application environment.

Walkthrough: Snapshot Deployment

Start your IBM UrbanCode Deploy, Agent and Jenkins servers. Install the Jenkins Pipeline plug-in and configure the IBM UrbanCode Deploy server in Jenkins. See the first three steps of the Component Import walkthrough for details.
Create and configure an application, environment, application process, and snapshot in IBM UrbanCode Deploy.

Below is a screenshot of a sample application and configuration. Below is a sample application called Jenkins Pipeline Snapshot Demo with the Dev environment and a successful Version 33 deployment of the previous UCD-Pipeline component.
As shown in the UrbanCode Deploy – Pipeline component version created in the Component Version Import walkthrough has been deployed. Note: A simple Component Process was created that could then run through an Application Process. This was not explained above. If you are uncertain on how to create an application and make a successful deployment, see the Sample Hello World Deployment tutorial.
After you have an Application with a previously deployed version, create a snapshot. Click the Snapshot tab, then the Create Snapshot button. Follow the dialog and create a new snapshot named Base Configuration. Select one of the versions that has already been successfully deployed to your application. Remember this snapshot name as is used in the Jenkins configuration. Again, if snapshots are new, see Creating Snapshots tutorial. The below screenshot is the sample snapshot created using the version 33 from the previously deployed application.

Now that we have our IBM UrbanCode Deploy configuration and snapshot ready, we are ready to configure the Jenkins project.

Create a new Jenkins pipeline project as explained in Component Version Import walkthrough steps 5a to 5d.
Download the Jenkins Snapshot Deployment Pipeline script and copy it into the pipeline script property field. You should immediately notice one major difference with the previous Pipeline script: the Clone and Build steps are not defined. The Jenkins Pipeline plug-in is not limited to deploying locally built artifacts. It can interact with IBM UrbanCode Deploy's API and deploy artifacts that potentially never built in Jenkins. In this example, we are simulating this scenario by taking artifacts that have been saved in IBM UrbanCode Deploy’s Codestation repository. The four required properties specify the Application, Environment, Application Process, and Snapshot version. The first three properties were taken from my specific Application. These properties will likely be different from your own demo application. The snapshot name specified is the same snapshot I created manually in IBM UrbanCode Deploy. The snapshot specified in the deployVersions property, must follow the format: SNAPSHOT=snapshot_name. The deployVersions property is robust and not limited to snapshots. If you do not have a snapshot available, you can specify components and versions. For example: if you followed the above walkthrough you could specify: UCD-Pipeline:1 which is in the format: component_name:version. This deploys version 1 of the UCD-Pipeline component. As an aside, the Jenkins Pipeline plugin is not limited to using snapshots, it can create them too. This feature is enabled using the createSnapshot property. In these scenarios, components and their associated versions would be specified to create new snapshots, instead of replicating the deployed snapshot. Lastly, the deployOnlyChange property deploys the snapshot regardless of what version was last deployed on the environment. This is a good property to set for testing, but may not be applicable for all production scenarios.
In my configuration, I will deploy the Base Configuration snapshot to the Dev environment on the Jenkins Pipeline Snapshot Demo application using the Deploy Demo application process. Once your Jenkins pipeline script is configured, click the Save button.

Click Build Now from the Pipeline project’s overview page to initiate, from Jenkins, your snapshot deployment using IBM UrbanCode Deploy.

Watch the Jenkins Pipeline view and logs to confirm it completed successfully. If any steps fail, return to the Project Configuration and make the necessary updates.
Lastly, switch over to the IBM UrbanCode Deploy to view the Application Process’s logs. Depending on your process, you may be able to see it in action. In the deployment history, you should see something similar to the below screenshot. Notice that the Snapshot value for our most recent deployment is Base Configuration which is the same snapshot I specified in my Jenkins script.

Freestyle Project

The same deployment scenario can be configured in a Jenkins Freestyle project. The below snapshot mirrors the Pipeline script configuration in the Snapshot walkthrough. As explained before, because we are specifying a snapshot in IBM UrbanCode Deploy, we do not need to specify a repository or build tool. The Base Configuration snapshot is already configured in IBM UrbanCode Deploy and contains all of the information I need to reconstruct our baseline environment.