DevOps Automation

Tucked away under the Settings tab in UCD (now DevOps Deploy) is the Audit log.  The audit log works tirelessly away in the background to record everything that is changed, or by default, even looked at in your UCD system.

This little regarded feature (if you have no use for it) is on a mission to gobble up database space.  One UCD server I performed an assessment on had been running for several years, they are a large organisation with a heavy workload for UCD.  Their audit log had 675 million rows in it.  The space occupied by this behemoth accounted for around 1/3^rd of the entire UCD database space requirement and made quite a difference to the time it took to do database backups.  There is also the load that it places on the database as it solemnly records all that goes on.  There can also be implications for server upgrades on such a large table.  For example if a new key is added for an existing column.

Now it has to be said that this is a pretty exceptional case, but it illustrates the point.  Have you configured the audit log in your system?  Older versions of UCD had little or no configuration for the audit log, it just does its thing day after day.  But newer versions of UCD (circa 2017) give you options to control it.

I suppose the question is: "Do you actually use the audit log, do you need the information that it gathers ?"  In organisations that have regulatory obligations to fulfill, the answer to this will probably be a resounding yes.  But in others, it may not be as clear cut.  So perhaps then the question becomes: "What do I need to know?"  Do I care that a particular user has accessed objects in the UCD system or do I only care if they have changed an object?  Maybe I have no need for a long term record of either form of access.

Version 6.2.5.0 introduced an option to control whether UCD records events where a user reads an object.  So if you don't really care that someone has looked at objects, you could turn off the read event recording.  This will save a substantial amount of space on a busy system since the read events tend to be more prevalent than the write events.  This is off by default, so it records read events.

Version 6.2.6.0 introduced a means to store audit records offline and to also manage how long UCD kept records in the audit log for.  We'll look at these features in more detail as there are some gotchas surrounding them.

Version 6.2.7.0.ifix01 saw some additional information about the method of authentication a user used to connect to UCD.

To change the settings for the audit log, go here:

The first switch allows you to turn off recording of read events.  If you don't care about these, turn them off.  It won't get rid of historical ones - you are stuck with those until we clean them up.  But at least you won't get any more.

If you can't see these options, then you are on quite an old version of UCD and its time you upgraded to a newer one.  You're missing out on a lot of great stuff.  All version 6.2.x are end of support 30th April 2021, anything earlier are already out of support.  You can find more on Lifecycle support here. 

The next thing you need to decide is whether or not you need all of the audit records forever or only for a time.  If we don't need them forever we can enable Audit Log Cleanup.  But before you do that you should review how many audit records you have and how old the oldest one is.

If you've got a server that has been running for a while, just turning on cleanup with say the default 30 days retention period can have serious implications for your servers performance.  If I've got 5 years of data suddenly going down to 30 days of data means the cleanup task will have a mammoth task.

If you want to get your audit log under control then I would recommend that you find how many days old the oldest audit log entry is and then over time slowly decrease the number of days records UCD holds until you get to the number of days you want.  That way you aren't going to ask the database to delete (potentially many) millions of records and update the indexes for those records all in one go.  You can chip away at it a few days at a time.  How many days you reduce the time to keep by at one go will to an extent depend on how busy your system has been.  Looking at the oldest records when you first adopted UCD may give you a very skewed view as you won't have been doing that much back then.  So have a look at how may records are created in a day at say 6 month intervals during your server's life.  You can use the audit log filter fields on date to give you an idea. 

TIP: If you look at the bottom left of a filtered page, it will tell you how many records exist in the filtered dataset.  If you don't have any filters set, then it will be the total number of records held.

Then you will have a rough idea about how many records you'll be asking UCD to remove if you reduce the keep time by a day or a week or a month.

Offline Storage of Audit Logs

You will probably have noticed that there is a Download as CSV button next to the Audit Log Settings button.  It's tempting to click and save, but again on a very busy system that's been running for a while that can be a VERY big file.  You really don't want to download all of the audit logs as a single file.  It would take a very very long time and consume a large amount of disk space as its not a compressed file.

To do the offline of data in a sensible fashion, you can again use the date filters (or indeed any of the other filters that can restrict the data you actually want to keep).  Once the filter(s) is applied, downloading the records to a CSV file will only download the records you can see in the filtered dataset.  Just be careful how you set your filters to avoid missing records that don't match filters from adjacent downloads.