DevOps Automation

Don't Modify an Existing Plug-in Yourself

Why not?  I can download it and I can see what the problem is and it's an easy fix!

Doing this has consequences, some obvious and some less so.  Firstly, let's make a distinction between plug-ins provided with the server and ones you add later from the website.

If you were to modify an existing plug-in, then the main consequence is that you would be cut off from all future updates.  If you create a new version, and then subsequently the author releases a new version, one of these situations can arise:

1. The new originator authored version might have a newer or the same version version  number as your updated plug-in and would just overwrite your changes if you install it.  But there might be some additional issues if the original authored new version changed the interface and the version numbers are the same.  UCD might not pickup the interface changes.
2. Your version introduced interface changes which a new originator authored version knows nothing about and doesn't know how to handle the upgrade.

Basically, you have dead-ended the plug-in in your environment.  You're on a divergent path and now it's going to be tricky to get back on-track.

Remember the distinction we made about server provided ones?  Well there is an additional consequence here.  If you update one of these, and dead-end the plug-in, you could break the capability of upgrading your UCD server in the future.  This is really bad and potentially could leave you with a partially upgraded server.

So the basic rule is:  If you didn't write it, don't modify it.  Make your own plug-in.

Don't Develop / Update Plug-ins on your live UCD Instance

The reason for this may not be immediately obvious, so let's look at what happens when you install a new plug-in version into a UCD server.

Apart from the obvious uploading of the new plug-in zip file, UCD unpacks it and then examines the plugin.xml and upgrade.xml files to determine what's changed. 

Now there are a few possibilities here and we can exploit some of these to make updating plug-ins a bit simpler.

If we are only changing the scripts that constitute a plug-in, we can get away without updating the plug-ins version number until we're ready to install it on our live UCD instance.  UCD will still upload the updated plug-in and it will replace the old plug-in.  So I can more rapidly go through the change/build/test cycle.  However, when I'm ready to release it, I must update the version number of the plug-in before installing in my live system.

If we're changing the interface we must update the plug-in version number anyway so that UCD processes the interface changes.   So, what does UCD need to do?

In this case, UCD has a potentially much bigger task.  It must update the plug-in reference(s) in every process that uses a step from the updated plug-in and secondly it must adjust the definition of every step that has an interface change described in the plugin.xml and upgrade.xml files that are bundled in every plug-in.

To avoid breaking processes that might be referenced in a locked snapshot which we don't want to change, UCD automatically makes a new version of the process with the required changes in it.  Exactly when UCD does this isn't known except to say that processes will be updated before they are accessed or used.  So for example, when I go to edit an affected process I will see a new process version.  It's a bit like the Schrödinger's cat conundrum.  Does the act of looking at the process cause it to change or did it happen sometime before?  For our purposes, the outcome is the same.

Developing a new plug-in isn't such a big deal but upgrading a well-used plug-in can have severe consequences if you get it wrong.  Also, remember that when you install or update a plugin, the steps the plugin provides will be immediately exposed in the process designer step palette for anyone to use.

Therefore, develop / update plug-ins in a separate instance of UCD where the consequences are contained.  Test carefully to make sure your upgrade.xml is correct and that your changes actually do what you expect them to do.

How Can I Fix a Plug-in or Extend One Without Hurting Myself?

The simple answer is to write a new one of your own to provide a fixed implementation or an additional step(s).  The design of the plug-in metadata allows you to contribute the steps in a plug-in to any drawer you like in the process designer's palette.  From an end users point of view you can make it appear that there are new steps in the original plug-in without having to actually modify the plug-in itself.  If you're modifying an existing step, it would be best to give it a new name, so you know which is your updated one and which is the original.  I would suggest you only use the updated one where the original won't do.  If in the future the original is updated and makes the modified version unnecessary, you could always go back and remove it from use.  This would of course be a manual process of identifying  the step(s)

Updating a step can be as simple (subject to licensing) copying the broken or deficient step from the original plug-in into your own plug-in and fixing it.

Make sure your plug-in ID that appears in the plugin.xml isn't the same as the original plug-in.  These ID's are hidden so make it obvious that you built it.  Ensure the ID is unique to you.  If you use the original ID, UCD will think you're trying to update the original plug-in.  Don't go there.

Of course, if the plug-in was developed by you from scratch, then you have complete control and can freely update it provided nobody else in your organisation thinks the same thing!!  So there needs to be a process to control plug-in development.  Use and SCM tool to control and manage versions.

So administrators should do their due diligence when installing or updating plugins to make sure that nothing bad is going to happen.   Especially when installing home grown plug-ins.

What Makes a Good Plug-in Step?

This one is trickier to answer but essentially it should be:

Self-Contained
The step should have everything it needs in the plug-in or you should declare prerequisites.  By this I mean if you're writing a plug-in to automate functions of a particular tool, you need to be explicit about what version(s) of that tool you have tested with.  Ideally the plug-in should check that.  If there are any configuration assumptions, they should be specified.  A plug-in should normally be dedicated to a single task. 

There is an argument for a plug-in that contains disparate single steps where otherwise you would end up with a load of single step plug-ins.  This is perhaps more a matter of style.  However, you should perhaps consider the impact of the number of potential process updates with such an arrangement. 

Stateless
UCD Steps are essentially stateless.  The agent that executes them treats all steps as individual jobs that get executed in the own processes.  So, you can't rely on any context of an earlier step unless the prior step preserved information by means of step (or other properties) or by leaving them in a file in the agents working directory.  If you need certain information for the step to execute, it should be provided to the step through its interface.  This is the dialogue you complete for each step in the process designer.  So, if you need a connection ID, user / password, token etc., pass it to the step.  Don't rely on picking it up out of some magic location on a target machine.  These would be hidden from UCD and then not form part of the step history of properties.  Leaving stuff about on a target is also a potential security risk.  If your process fails for some reason, it might get left behind.  Also don't rely on environment variables established via a login profile.  This again hides information which should be in UCD and also opens the door to unauthorised changes and the potential for deployment failures across the environment stack because something is missing.

Focused
A step should seek to do one thing and do that one thing well.  We shouldn't have steps that perhaps have a parameter which specifies which behaviour you want.  For example, I could have a step that creates / updates /deletes information in an integrated tool.  I could make the operation a parameter.  While this saves me a bit of code, it's not focused.  UCD provides a number of icons for steps (based on the step name), but you can only have one on a step.  So, in the process designer, you'd just see one icon on the step regardless of function selected.  This kind of obfuscates what the step does when browsing a process.

Encapsulate
The steps in a plug-in should ideally be closely related and together perform some useful set of functions that encapsulate knowledge that means others can leverage it without having to be an expert themselves.  So, they are typically focused on a technology, tool or some capability - like say text manipulation or managing a Tomcat server to name but two. 

Contribute a Plugin
If you have success with a plugin you have written, or want to more examples on how to write a plugin, be sure to visit the community site here and follow the directions: https://github.com/UrbanCode/contributions

Summary

Writing your own plug-ins is a great way to encapsulate process steps that often get written as shell steps.  Plug-ins are normally written in groovy which will work on Windows or Linux.  The groovy interpreter is installed as part of the UCD agent so its guaranteed to be available.  Shell steps are normally bash or some other Linux shell or a command prompt on Windows and of course Windows and Linux would need entirely different implementations.  Something else you can more easily accomplish seamlessly in a plug-in step.

Whenever you see a script type step which appears in some form or other repeated in several places, you've found a good candidate for a plug-in step.  Think of reuse of process steps by shell steps as more cut-and-paste, while writing a plug-in is more like writing a reusable procedure or function.  When you cut-and-paste something, you cut-and-paste the bugs in it as well.  We all know which approach requires the least maintenance. 

The other thing writing a plug-in gives you is robustness.  Shell scripts are usually great, when the work, but they often don't detect all the errors that could occur and so your processes may appear to work when in fact there was an undetected error in your shell script.

Plug-ins can deliver a great deal of benefit, are easier to maintain and generally result in more robust solutions if appropriate care is taken.