Hi all, I would liketo know:
When i save and load a dataset where is saved the dataset?
It is saved in the content store?only the specifications ( xml?json?) Like a normal report spec
Or is it saved all togheter,containing not only the specifications but also the Data  loaded (like an active report )?

Second question is about settings of dataset.
 is it possible to set the timeout of the dataset file stored in the temp folders?
I would like to set for a dataset a validity of 1month, even it is not used so i could reload it only 1 time per month.


Thanks

------------------------------
Magnum BI

https://www.linkedin.com/in/magnum-bi-0297a2214
------------------------------

#CognosAnalyticswithWatson

Datasets are parquet files that are stored in the server. But I don't know where exactly.
https://en.wikipedia.org/wiki/Apache_Parquet

------------------------------
Alejandro Sanchez
------------------------------

Hi

1) Datasets are stored in the content store, and disk/memory. 
Example :If you restart the Cognos service it will load from the ContentStore into disk/memory, a number of datasets will be in-memory
It is possible to fine tune :
https://www.ibm.com/docs/en/cognos-analytics/11.2.0?topic=sets-configuring-compute-service

2) If you make a schedule running once a month, you would automatic keep it for a month.


------------------------------
René Kent Nielsen
Brand manager
CogniTech A/S
Herning
------------------------------

Hi, 

Datasets, in the latest version of CA (I don't remember the exact version but since one of the 11.1 stream versions) , are stored in content store which includes the data as well. The data will be encrypted and compressed to make it secure and efficient for storage and retrieval. A parquet file is created when you first access the data set and stored in the Data Files path you specify in IBM Cognos Configuration which is typically ../data folder. 

I do not know the answer to your second question myself so I will let others weigh in if there is a default setting somewhere. If there is not, an alternative could be to run a scheduled script to remove such files from ../data folder (or wherever you store it based on your configuration) using the file properties like created date and deleting anything that is older than x no. of days. 

Regards

------------------------------
Kiran Passumarthi
www.linkedin.com/in/kiranpassumarthi
------------------------------

By default report output and datasets are stored in Content Manager, this include the actual data.
You can also store it outside of Content Manager, pls see: Use an external object store for report output and datasets - IBM Documentation
The cleanup interval of datasets from local disk is set to 1 week, and is not currently exposed as a configurable setting.

------------------------------
Kind regards,
HENK CAZEMIER
------------------------------

Original Message:
Sent: Fri October 29, 2021 02:35 AM
From: Kiran Passumarthi
Subject: Datasets,temp files and content store

Hi, 

Datasets, in the latest version of CA (I don't remember the exact version but since one of the 11.1 stream versions) , are stored in content store which includes the data as well. The data will be encrypted and compressed to make it secure and efficient for storage and retrieval. A parquet file is created when you first access the data set and stored in the Data Files path you specify in IBM Cognos Configuration which is typically ../data folder. 

I do not know the answer to your second question myself so I will let others weigh in if there is a default setting somewhere. If there is not, an alternative could be to run a scheduled script to remove such files from ../data folder (or wherever you store it based on your configuration) using the file properties like created date and deleting anything that is older than x no. of days. 

Regards

------------------------------
Kiran Passumarthi
www.linkedin.com/in/kiranpassumarthi
------------------------------

Henk, when you say 'cleanup interval of datasets from local disk', are you referring to the files in the .../data directory on the app server?

------------------------------
Ryan Dolley
2019 IBM Champion
www.ibmblueview.com
www.pmsquare.com
Connect with me on LinkedIn: linkedin.com/in/ryandolley
------------------------------

Original Message:
Sent: Fri October 29, 2021 10:49 AM
From: HENK CAZEMIER
Subject: Datasets,temp files and content store

By default report output and datasets are stored in Content Manager, this include the actual data.
You can also store it outside of Content Manager, pls see: Use an external object store for report output and datasets - IBM Documentation
The cleanup interval of datasets from local disk is set to 1 week, and is not currently exposed as a configurable setting.

------------------------------
Kind regards,
HENK CAZEMIER
------------------------------


Original Message:
Sent: Fri October 29, 2021 02:35 AM
From: Kiran Passumarthi
Subject: Datasets,temp files and content store

Hi, 

Datasets, in the latest version of CA (I don't remember the exact version but since one of the 11.1 stream versions) , are stored in content store which includes the data as well. The data will be encrypted and compressed to make it secure and efficient for storage and retrieval. A parquet file is created when you first access the data set and stored in the Data Files path you specify in IBM Cognos Configuration which is typically ../data folder. 

I do not know the answer to your second question myself so I will let others weigh in if there is a default setting somewhere. If there is not, an alternative could be to run a scheduled script to remove such files from ../data folder (or wherever you store it based on your configuration) using the file properties like created date and deleting anything that is older than x no. of days. 

Regards

------------------------------
Kiran Passumarthi
www.linkedin.com/in/kiranpassumarthi

That's correct. It's copied from either Content Manager directly or External Store to local disk. Before a copy is made to local disk, existence is checked and if up to date, copying is avoided. These are the files in <install_dir>/data/datafiles/datasetCache/...

------------------------------
Kind regards,
HENK CAZEMIER
------------------------------

Original Message:
Sent: Fri October 29, 2021 01:52 PM
From: Ryan Dolley
Subject: Datasets,temp files and content store

Henk, when you say 'cleanup interval of datasets from local disk', are you referring to the files in the .../data directory on the app server?

------------------------------
Ryan Dolley
2019 IBM Champion
www.ibmblueview.com
www.pmsquare.com
Connect with me on LinkedIn: linkedin.com/in/ryandolley
------------------------------


Original Message:
Sent: Fri October 29, 2021 10:49 AM
From: HENK CAZEMIER
Subject: Datasets,temp files and content store

By default report output and datasets are stored in Content Manager, this include the actual data.
You can also store it outside of Content Manager, pls see: Use an external object store for report output and datasets - IBM Documentation
The cleanup interval of datasets from local disk is set to 1 week, and is not currently exposed as a configurable setting.

------------------------------
Kind regards,
HENK CAZEMIER


Original Message:
Sent: Fri October 29, 2021 02:35 AM
From: Kiran Passumarthi
Subject: Datasets,temp files and content store

Hi, 

Datasets, in the latest version of CA (I don't remember the exact version but since one of the 11.1 stream versions) , are stored in content store which includes the data as well. The data will be encrypted and compressed to make it secure and efficient for storage and retrieval. A parquet file is created when you first access the data set and stored in the Data Files path you specify in IBM Cognos Configuration which is typically ../data folder. 

I do not know the answer to your second question myself so I will let others weigh in if there is a default setting somewhere. If there is not, an alternative could be to run a scheduled script to remove such files from ../data folder (or wherever you store it based on your configuration) using the file properties like created date and deleting anything that is older than x no. of days. 

Regards

------------------------------
Kiran Passumarthi
www.linkedin.com/in/kiranpassumarthi

File upload and Datasets, both result in Parquet files being created, and saved with the content manager.

Content Manager by default, will store that data within a CM database table, unless CM was configured to use an external store

https://www.ibm.com/docs/en/cognos-analytics/11.1.0?topic=options-use-external-object-store-report-output-datasets

There is no time to live rule you can define re

- automatically deleting Parquet files from the content store

- local copy cached on an instance of a CA app-tier node. they are pruned on a default cycle

If you generate a deployment, which included folders that included Parquet files, that data would be included in the deployment export. When imported into a CA environment, that data will be used to generate new Parquet files on the system being imported into.

------------------------------
NIGEL CAMPBELL
------------------------------