Cognos Analytics

Where is data uploaded into Cognos Analytics stored?

By KEVIN MCFAUL posted Wed August 17, 2016 01:38 PM

  
In versions of Cognos Analytics prior to 11.0.3, data sets from uploaded files and data modules in snapshot mode are saved directly to the file system. The data sets are stored in a columnar format designed to be loaded into memory quickly on demand when accessed by the Cognos query engine.

As of 11.0.3, when files are uploaded, data is still converted to the columnar format, but it is stored in the Cognos content store. This is done in order to provide a central storage location and so that the life-cycle of the data is better managed when performing tasks such as move/copy and deployment. If the storage capacity of the content store database is a concern in this context, you have the option to use an external object store instead.

Using an external object store for report outputs and data reduces the size of the content store and provides performance improvements for Content Manager. Note that reduced load on the Content Manager may or may not affect user wait times for dashboards and reports. There are many factors that can impact report and dashboard performance including but not limited to network speed. Using an external object store may improve dashboard and report performance in some environments but not in other environments.

Although the data is stored in either an external object store or content store database, it will be automatically extracted to temporary file locations on the Application Tier Component servers when it is being used in dashboards and reports. The default location for these temporary data files is install_location\data\datafiles. These temporary data files will be removed automatically by the Cognos Analytics software. Prior to 11.0.3, the location of directory for data files needed to be set to a network path in distributed installations for Cognos Analytics – this is no longer a requirement, but can still provide benefits by having all servers sharing the same instance of the temporary files.

As of version 11.0.3, this data can be included in deployments by enabling the option to include report output versions* in the export deployment specification.  By default, data that was uploaded to Cognos Analytics in a version prior to 11.0.3 will not be included in these deployments. You can use the following script to migrate data that was uploaded in 11.0.0, 11.0.1 or 11.0.2 from the file system into the content store database or external object store so that it can be included in deployments.

 

As of Cognos Analytics version 11.0.3, there is a utility in /bin and /bin64 named:

  • MigrateUploadedFiles.bat

  • MigrateUploadedFiles.sh


The .bat file is to be run on Windows, while the .sh file is to be run on Unix/Linux based platforms.

Usage:

Options:


-n : Authentication namespace, -u and -p required if specified [optional]
-u : Namespace user, -n and -p required if specified [optional]
-p : Namespace password, -n and -u required if specified [optional]
-df : Data files location [optional]
-params : Specifies program parameters with a file [optional]


Example:

UploadedFileMigrationTool -n namespace -u username -p password


To run, simply run the following command from your /bin64 :

MigrateUploadedFiles.bat


If authentication is required, run:

MigrateUploadedFiles.bat –n -u -p <password


In either case, the user is required to have ‘System Administrator’ role.

The script will run, and the output should be in the form of:

D:\\bin64>MigrateUploadedFiles.bat
running:
log4j:WARN No appenders could be found for logger (org.apache.commons.httpclient.params.DefaultHttpParams).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
The following storeId 'iDBFE011C600F4B5CB62A4CB0CF24B433' is not an Uploaded File and is associated with file: Boston_311_calls_xlsx.parquet
Successfully migrated uploaded file: Legacy1_csv.parquet
The following storeId 'iBD98A1CC5A4247E6933556B44233C92C' is not an Uploaded File and is associated with file: SampleFile_GOSales_xls.parquet
The following storeId 'i6CABB57DA69F4FDD9CB351F6A91B1D38' is not an Uploaded File and is associated with file: Banking_loss_events_xlsx.parquet
The following storeId 'i736C61A6681A41E4A37DA5E709F4C02F' is not an Uploaded File and is associated with file: American_time_use_xlsx.parquet


*******************************************************************************
Total files to migrate: 5
Number of files migrated: 1
*******************************************************************************
Successfully migrated files have been backed up in: D:\\ \bin64\..\configuration\..\data\datafiles\MigratedUploadedFiles_1465833065909.zip
D:\\bin64>



Footnotes:


*As of 11.0.5 use the option Include uploaded data instead.


#GeneralInformation
#data
#LearnCognosAnalytics
#dataset
#home
#administration
0 comments
55 views

Permalink