Logging onto machines, particularly production machines, is not a DevOps best practice, so how can we get logs to diagnose faults?
Particularly with Production systems, it's a poor DevOps practice to login to machines to make changes. Ideally everything should happen through your automated systems so there is no need. This prevents little 'tweaks' being done that might fix a short-term issue but long term causes drift in the platform with possible repercussions later. It also means that if a new system is required, we might not actually have recorded all the setup required because a fix was put in outside of the automated deployment processes. This makes ongoing support and maintenance much harder.
So, we limit access to machines to stop this exact kind of issue. Nobody can just 'tidy-up' while they are there anymore. Or just make that little configuration change to make it a 'comfortable' place to work.
However, in order to determine the cause of an issue on a platform, developers often need the appropriate logs and dumps to analyse.
So how do we reconcile the need to keep people off the servers but also provide access to the necessary data for fault analysis?
The same can also be true if you have a support case with a 3rd party software vendor. They will need data to resolve the product issue. IBM Support for example often has a 'must gather' bundle for products and it's a lot easier if these are automated.
OK, so we can quite easily create ourselves processes to capture the necessary logs, dumps, configuration etc, but where to put that bundle? I need it to go somewhere that my agent can write to off of the server and I don't really want to have to get firewall ports opened.
So why not use the capabilities of UrbanCode to upload the bundle and make it available through the UCD GUI for download and subsequent analysis?
The technique I describe here uses a UCD Component version to hold the uploaded logs. In this example I created two components:
- Log Receiver
This component will receive new component versions containing the 'mustGather' assets. Nothing is ever deployed from this component and the component doesn't need to be part of an application. This component could be shared by many applications or as appropriate in your organisation.
- Log Sender
This is a component that holds the must gather process, but you could put it in any component that is part of your application or use a generic process for example.
The Log Receiver component is setup to have its own clean-up rules to avoid it accumulating lots of old stuff and taking up valuable CodeStation disk space. I set the Days to Keep and Versions to Keep both to 1 in the components configuration. So the UCD daily clean-up will remove all but the most recent logs in the event they are not removed in a timely manner.
I created this simple component process (Operational (Without a version)) to capture the logs and upload them into a version of the Log Receiver Component.