DevOps Automation

However, although this is really easy, it comes at a price, well several actually:

1. You must nominate at least one agent (normally dedicated) to importing new component versions.
   • So there is a license cost straight away or you accept that the agent might also have to do other work which may slow down the import.  Also you need to consider what additional firewall ports you might need opened in this case.
2. The component can be configured to require manual importing of a new version which means you must tell it when to do it:
   • If you forget to tell it to import or you don’t request all of the components in your application to update, you might have inconsistent versions between components.
   • This takes your time and introduces a further delay between the completion of the build and having a new version ready to deploy, slowing your CI/CD pipeline.
3. You set the component to automatically import new versions.  A nominated agent now polls the repository every few minutes looking for any new versions to import:
   • Polling introduces a delay into your CI/CD pipeline, new versions will be delayed before they are discovered and made available.
   • When you scale up in a large enterprise with 100’s or even 1000’s of components this is not an insignificant load.  So you either need more agents or wait longer.
   • For the vast majority of poll cycles, the agent will find nothing for any given component so its wasted effort.

So, although its easy to do, just configuring the source code plugin, select your agent and tick a couple of boxes it has a cost.

Gotchas

With some of the source plugins you have to consider that the polling agent won’t know when a component version is complete, it will start downloading as soon as it sees a new version is available.  So, for example using the Versioned File System source plugin, as soon as the new directory appears and the poll cycle starts, it will start downloading.

So why should you care?  Well, if you’re still copying the files into the new directory and the agent starts downloading you might randomly get incomplete component versions because not all the files were placed or maybe even incomplete files depending on how your copy works.  To avoid this, you should create the directory someplace else on the same filesystem first.  Then you should move the folder into the correct location for the agent to discover it.  In this way the creation of the new component version becomes atomic – its either there or its not so you can avoid the possibility of it not discovering everything.

Is there a better way?

I believe there is.  I think using the udclient API to publish your build assets directly into UCD makes more sense.  There is no polling of a repository and therefore no delay introduced into your CI/CD pipeline.

You can trigger the import directly from your build if it is successful and it’s pretty easy to do.  We just add a few udclient commands to your build script.  This also gives you more control over what order things get imported in.

Some of the source code plugins do effectively the same thing. Any plugin that is a plugin to your build system/repository tool will likely push the assets into UCD in a similar manner to what is described.  eg Jenkins Plugin

There are several udclient commands we will make use of:

• createVersion
  This creates the new version container to receive the version assets
• addVersionFiles
  This command uploads the files into the version container.  You can add from several sources with multiple commands if you wish
• addVersionLink
  Optionally add an http link back to your build result ensuring traceability back to your build.  You can add more than one link if you want.  You likely don’t have this option with most source code plugins
• addVersionStatus
  Optionally add one or more UCD statuses to the new version.  Not available with all source plugins
• setVersionProperty
  Optionally create properties values on your new version passing information about the build for example build reference.

This approach is quite flexible and optionally allows you to make use of more UCD capabilities all automatically from your build.  It also doesn’t tie up an agent and there is no delay in your CI/CD pipeline waiting for the next source import cycle.

Here are some examples of the various commands.  First we’ll create a new container for a component version and then add files to it:

You can perform several addVersionFiles calls if you wish.  However, once the version is deployed for the first time the contents are locked.  There are other parameters as well that allow you to be more specific about what files should be uploaded and also one for preserving the execute permissions of files if desired.  This might make your process simpler as well.

This example adds a link to my build result so that I can traverse from my component version in UCD back into my build tool.  This helps to maintain the traceability of my version back up the CI/CD pipeline.  I can add multiple links if I want.

Next Here I’m going to add a status to say that my build ran the code quality check tool

Lastly, I can add a new property or change the value of an existing one on a component version provided the version hasn’t been deployed yet.  This request requires information in a JSON file (or fed to standard input).  My file looked like this:

Next Time

In the next part of this article, we will explore the udclient API further to discover what other features we have available for uploading new component versions.