One dilemma we often face introducing UrbanCode Deploy (UCD) into a new environment is whether to use CodeStation or an existing 3^rd Party Repository like Git, Nexus, Artifactory etc.

While we can certainly use the source plugins to pull assets from one of these (or other repositories).  [See my blog on CodeStation Push or Pull for the pros and cons. .

Often you may find a client may want you to deploy assets directly from one of the external repositories.  This usually arises because that’s how the client currently does things.

While this is certainly possible, it does introduce some consequences of such an approach which defeat some of the capabilities of UCD itself.

If we start by considering the architecture of UCD we see there are the three main components:

1. Server(s)
2. Relay(s)
3. Agent(s)

Let’s look at the purpose of relays as that is the most relevant in this circumstance:

1. To Allow the UCD server to scale to a larger number of agents than you could directly connect to the server.
2. To allow agent traffic to pass to the server without having to open firewall ports for every agent.
3. To cache assets that pass through the relay so that other agents connected to the same relay can get them without traffic coming directly from the server again.

In this case points 2 and 3 are the relevant things to consider

Pass Firewalls

A key purpose of the relay is to allow traffic to pass firewalls back to the server without opening individual ports for each agent behind the relay.  So, we only need to open the ports for the relays and the agents just need to be able to talk to the relay.  We can add news agents and just connect them to the relevant relay(s).

If we have the agent download direct from a 3^rd party repository, there are consequences.  Firstly, the agent must be able to talk to the repository to fetch the assets.

If we consider one of the DevOps practices is single-source-of-truth, this implies that all agents must fetch assets from that one repository instance.  So, given that different deployment environments are likely in different firewall regions, it follows that firewall ports would have to be opened for every agent to access the repository.  This is a lot more to maintain and it also means that introducing new agents then places an additional workload on the infrastructure teams to open those ports and therefore delays in getting the agent commissioned.

If there are separate independent repositories for each environment, then we break the single-source-of-truth best practice and open the door to their being accidental (or deliberate) differences in assets between environments.

Other Compromises

Using an external 3^rd party can also lead to making other compromises, particularly in configuration.  You may then find that since all the assets come from the 3^rd party repository then there is a tendency for configuration to come from there as well.  So that leads us to environment specific configurations and we’re heading into anti-pattern territory.  Assets that contain configuration information for unrelated environments.  Our configuration information should be held and maintained in UCD where the proper controls are in place and we have facilities to deal with separation of concerns issues.  If we have our configurations for all environments in our assets, then anyone with access to one environment can see the configurations for all.  Is this desirable?

Alternatively, you might also see another anti-pattern arise, environment specific builds to address this.  Unfortunately, this itself introduces more problems.  It means that we end up with different component versions for each environment and that means we then can’t use UCD Snapshots.  This in turn means there is a risk of deploying the wrong version into an environment.  This is quite aside from the main issue that we’re not actually testing the exact same thing in each environment.

Traffic Reduction

Another purpose of the relay is to make sure that assets are made available closer to the agent by optionally caching information as it passes through the relay.  The relay contains a cut-down codeStation that holds the complete versions of assets that are passed through it.  So, this doesn’t break the single-source -of-truth practice as its simple an extension of the servers’ repository.

If my agent needs something the relay has, it can be satisfied from this local copy.  However, if we look further into the agents’ asset downloading strategy, we can see that the Download Artefacts step has optimisations to only download changed files.

Going direct to a 3^rd party repository negates these two benefits and I always have to get the full asset (unless the 3^rd party repository provides similar capabilities).

Summary

While it’s entirely possible for a process step to fetch assets from a 3^rd Party repository, it is far more efficient to publish assets directly into codeStation at build time and perhaps also publish them into the 3^rd Party repository at the same time.  This does come at some cost of disk space. CodeStation isn’t really designed to be a forever-store, but more for assets that are to be deployed, are currently deployed or were recently deployed.  After that such artifacts really lose general relevance.  But if you need a complete historical reference, for perhaps regulatory reasons, then they would be in the 3^rd party repository.

Deploying assets from code station is usually much more efficient because of the optimisation strategies in UCD than it is to fetch them from a 3^rd Party repository directly by the agents.

One exception to this guidance is containers. Leaving container images in their registries and simply tracking their versions in UrbanCode Deploy is still a best practice.

Always challenge old ways of working when adopting new tools.  While the existing practices may have served a client well for many years, maintaining them with new tooling can lead to not realizing the full potential of the new tooling.

#urbancodedeploy