DevOps Automation

Shell steps are often overused, and they become just a way of doing stuff in the way you’ve always done it which rather devalues many of the benefits of UCD.

They should be regarded as the step of last resort rather than the kind of step you do everything with. The first thing you look for should be a plugin step that is designed to do what you want. You could even write your own plugin of steps that encapsulate frequently needed patterns. Check out the UrbanCode plugins website to see if there is a plugin that meets your needs. Only a small number of the available plugins are installed by default but the number of available is growing all the time.

I’ve come across organizations that have adopted UCD and virtually every step in their component process is a shell step. Sometimes, you find the shell step just calls a script which is not even held or controlled by UCD thus breaking the best practice of having only a single source of truth. We now have scripts that we can’t be sure will be the same script used all the way through the environment stack on the route to live. I’ve also seen the scripts all bundled together in another component which then creates deployment dependencies between components and in some cases even applications.
This behavior comes about because that is the way they have always done things, but really, they aren’t getting all the value they could out of UCD and they have not de-risked their deployment processes as much as they could / should have done.

Also if I write a shell script, these are often platform dependent. If I had a java application, I could deploy it with a shell script on Linux but what if I suddenly need to deploy it on windows. It won’t work. There are even differences between different Linux dialects. Plugins steps are often (but not always) platform independent.

I’ve seen processes that consist of perhaps two steps, a Download Artefacts and a shell step that does everything else. Doing everything in one step you lose out on the opportunities of performing steps in parallel thus losing opportunities for reducing deployment times.
People rarely write scripts that have any parallelism in them because, let’s face it, it’s hard and they rarely check the outcome of every command for errors which means that the shell step may contain failures that UCD doesn’t know about. The shell step only reports the outcome of the last command in the script, its exit code. So, if the last step in your script is: echo “Step Complete”

The echo is always going to work and so will the step in UCD’s eyes unless you take measures to process the return codes or write post-processing scripts to search the log for errors.

Another danger sign is scripts that rely on values from the login environment that the agent runs as or that you impersonate. All property values used in a deployment process should devolve from UCD properties otherwise you aren’t in full control of the settings used in deployments – again introducing risk and of course we have no guaranteed way of tracking what values were used after the event. For example, if we’re setting up a completely new environment, what would populate these initially? What guarantees do we have that they won’t be changed unexpectedly?

Quite often you see a useful piece of the script gets copied and pasted between many process steps, including of course, the bugs they contain. Finding and fixing all the places the bug was pasted to is a time-consuming and by no means certain process. You may also find that the pasted version diverges slightly with each instance created.

So ‘first-class’ plugin steps are the preferred way to go. That’s not to say that shell steps don’t have a useful place in our toolbox, but if you find that most of your component process steps are shell steps you are missing a trick. Plug-ins encapsulate the knowledge required to do things so that everyone else doesn’t need to be an expert. They just need to know how to use the steps.

If the tool you always reach for is a hammer, you will see every problem as a nail.

There are a growing number of available plugins that cover many areas of popular technologies. They might not do everything you want, but often they will. If you have something ‘special’ in your environment, consider writing your own plugin with the steps you need to encapsulate the functionality you require. If you find a bug in your plugin, you fix it, upload the fixed plugin and UCD will automatically update all processes that contain references to the old step. It does this by creating a new process version using the updated step. If you want it to, UCD can still use the old process version that is on your route-to-live, so you aren’t introducing more risk. Of course, if the risk of not having the update is judged as worse, UCD can accommodate that as well.

Plugins are not hard to write, but of course how hard depends largely on what you want the plugin to do. The basic framework though is quite straightforward.