Planning Analytics

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

Hi Bret,

Could you confirm which loggers you have enabled?

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

In our dev environment where I have been testing I had it set to the following but it was creating a ton of log files

#
#  Enable INFO level logging through the shared memory appender, by default.  The server #  will write informational messages, as well as errors and warnings to the log file.
#
log4j.logger.TM1=ALL, S1

# S1 is set to be a SharedMemoryAppender
log4j.appender.S1=org.apache.log4j.SharedMemoryAppender

# Specify the size of the shared memory segment
log4j.appender.S1.MemorySize=100 MB

# Specify the max filesize
log4j.appender.S1.MaxFileSize=100 MB

# Specify the max backup index
log4j.appender.S1.MaxBackupIndex=20

# Specify GMT or Local timezone
log4j.appender.S1.TimeZone=Local

Hi Bret,

Could you confirm which loggers you have enabled?

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

Have a look at this article to get a better handle on what is going into the stargate: https://community.ibm.com/community/user/businessanalytics/blogs/george-tonkin/2023/12/02/get-the-most-out-of-planning-analytics-logging

I have a suspicion that more years are going in than you think, and may just be zero suppressed.

In our dev environment where I have been testing I had it set to the following but it was creating a ton of log files

#
#  Enable INFO level logging through the shared memory appender, by default.  The server #  will write informational messages, as well as errors and warnings to the log file.
#
log4j.logger.TM1=ALL, S1

# S1 is set to be a SharedMemoryAppender
log4j.appender.S1=org.apache.log4j.SharedMemoryAppender

# Specify the size of the shared memory segment
log4j.appender.S1.MemorySize=100 MB

# Specify the max filesize
log4j.appender.S1.MaxFileSize=100 MB

# Specify the max backup index
log4j.appender.S1.MaxBackupIndex=20

# Specify GMT or Local timezone
log4j.appender.S1.TimeZone=Local

Hi Bret,

Could you confirm which loggers you have enabled?

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

------------------------------
Bret Pothier
------------------------------

Hi Bret,

I've used similar techniques to precalculate a view before processing it with a TI process.

The general premise is that the TI process retrieves the result from the cache and doesn't have to recalculate the view while it processes, thus saving a lot of time.

To get the view to cache in the first place requires you to play with the VMM and VMT values, and generally these are designed for end-user views not larger TI view sources.

Your issue sounds like the view size has just exceeded the VMM size, and its going back to non-stargate.

I'd test increasing the VMM size and seeing if this solves the issue.

Good luck

------------------------------
Karl Blackler
Head of Product Development
Cortell Australia Pty Ltd
Brisbane QLD

Original Message:
Sent: Wed March 19, 2025 12:28 PM
From: Bret Pothier
Subject: Stargate View creation time increasing

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

------------------------------
Bret Pothier
------------------------------

Thank you . I will look in to it

Hi Bret,

I've used similar techniques to precalculate a view before processing it with a TI process.

The general premise is that the TI process retrieves the result from the cache and doesn't have to recalculate the view while it processes, thus saving a lot of time.

To get the view to cache in the first place requires you to play with the VMM and VMT values, and generally these are designed for end-user views not larger TI view sources.

Your issue sounds like the view size has just exceeded the VMM size, and its going back to non-stargate.

I'd test increasing the VMM size and seeing if this solves the issue.

Good luck

------------------------------
Karl Blackler
Head of Product Development
Cortell Australia Pty Ltd
Brisbane QLD

Original Message:
Sent: Wed March 19, 2025 12:28 PM
From: Bret Pothier
Subject: Stargate View creation time increasing

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

------------------------------
Bret Pothier
------------------------------

Sachin has an excellent article on analysing Stargates here: https://everanalytics.wordpress.com/2016/03/22/stargate_deepdive/

Thank you . I will look in to it

Hi Bret,

I've used similar techniques to precalculate a view before processing it with a TI process.

The general premise is that the TI process retrieves the result from the cache and doesn't have to recalculate the view while it processes, thus saving a lot of time.

To get the view to cache in the first place requires you to play with the VMM and VMT values, and generally these are designed for end-user views not larger TI view sources.

Your issue sounds like the view size has just exceeded the VMM size, and its going back to non-stargate.

I'd test increasing the VMM size and seeing if this solves the issue.

Good luck

------------------------------
Karl Blackler
Head of Product Development
Cortell Australia Pty Ltd
Brisbane QLD

Original Message:
Sent: Wed March 19, 2025 12:28 PM
From: Bret Pothier
Subject: Stargate View creation time increasing

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

------------------------------
Bret Pothier
------------------------------

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

Since limiting the stargate view to data greater than 2024 drastically reduced the view creation time but shifted the delay to the loading process, it suggests the bottleneck might be in how the data is being processed or transferred. I recommend checking the following: 1) Ensure your stargate view query is optimized and only pulls necessary data, 2) Review system logs or performance metrics to identify resource constraints (e.g., memory, CPU), and 3) Consider breaking the process into smaller chunks or parallelizing tasks if possible.

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

ok so I have looked at it in more detail and I have additional questions, please bear with me.

Details

• The process that runs the stargate view initially runs at 4am daily.
• The process that calls it a second time runs at noon daily.
• I see no VMM and VMT values set from the cube the stargate is generated
• I see no Memory Used for Stored Views or Number of Stored Views populated for the cube the stargate is generated 
• I have checked and the other jobs that run between 4am and noon daily do not touch the cube or dimensions the stargate view is generated
• I can see the view that is created by the stargate process listed in the Views for the cube
  
  Questions
• What would cause the stargate view to be visible in the view list but not show in the StatsByCube?
  I have turned on and left performance monitor on in case it is a timing issue of what is being captured
• If the view is listed and I can open it, why would the second process that calls it need to run it again?
  Would it not see the existing view and skip creating it again or is the stargate view explicitly saying create a new one.
• Do I need to set VMM and VMT values? Does leaving them blank not make it meet the criteria by default?

Thank you for all the help on this, I greatly appreciate it

Since limiting the stargate view to data greater than 2024 drastically reduced the view creation time but shifted the delay to the loading process, it suggests the bottleneck might be in how the data is being processed or transferred. I recommend checking the following: 1) Ensure your stargate view query is optimized and only pulls necessary data, 2) Review system logs or performance metrics to identify resource constraints (e.g., memory, CPU), and 3) Consider breaking the process into smaller chunks or parallelizing tasks if possible.

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

Today I tried the following.

I deleted every element that was not used in the dimensions (around 1000), and I deleted the stored stargate view to see if it would generate a new one and maybe cache it properly showing details in the statsbyCube.

The stargate view still took a long time (2607 seconds) to complete AND/BUT the view was not created under the cube.

------------------------------
Bret Pothier

Original Message:
Sent: Mon March 24, 2025 03:31 PM
From: Bret Pothier
Subject: Stargate View creation time increasing

ok so I have looked at it in more detail and I have additional questions, please bear with me.

Details

• The process that runs the stargate view initially runs at 4am daily.
• The process that calls it a second time runs at noon daily.
• I see no VMM and VMT values set from the cube the stargate is generated
• I see no Memory Used for Stored Views or Number of Stored Views populated for the cube the stargate is generated 
• I have checked and the other jobs that run between 4am and noon daily do not touch the cube or dimensions the stargate view is generated
• I can see the view that is created by the stargate process listed in the Views for the cube
  
  Questions
• What would cause the stargate view to be visible in the view list but not show in the StatsByCube?
  I have turned on and left performance monitor on in case it is a timing issue of what is being captured
• If the view is listed and I can open it, why would the second process that calls it need to run it again?
  Would it not see the existing view and skip creating it again or is the stargate view explicitly saying create a new one.
• Do I need to set VMM and VMT values? Does leaving them blank not make it meet the criteria by default?

Thank you for all the help on this, I greatly appreciate it

------------------------------
Bret Pothier

Original Message:
Sent: Sun March 23, 2025 03:22 PM
From: Dr. Renzo Santoro
Subject: Stargate View creation time increasing

Since limiting the stargate view to data greater than 2024 drastically reduced the view creation time but shifted the delay to the loading process, it suggests the bottleneck might be in how the data is being processed or transferred. I recommend checking the following: 1) Ensure your stargate view query is optimized and only pulls necessary data, 2) Review system logs or performance metrics to identify resource constraints (e.g., memory, CPU), and 3) Consider breaking the process into smaller chunks or parallelizing tasks if possible.

------------------------------
Dr. Renzo Santoro

Original Message:
Sent: Wed March 19, 2025 12:28 PM
From: Bret Pothier
Subject: Stargate View creation time increasing

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance

------------------------------
Bret Pothier
------------------------------

Hi there,

We are on Planning Analytics on Cloud version 11.8.02800.9 (from the configuration tab in Administration)

We have a process that runs daily where it creates a stargate view then uses that recordset to process.  The stargate view used to take approximately 10-15 minutes to run in January but is now taking upwards of 40 minutes. 

What I am seeing is the stargate process taking around 40 minutes to run then another process to load it taking about 6 minutes to load.

I saw the data only existed in the stargate view from 2025 onward but was pulling all fiscal periods back to 2008. I tried limiting the stargate view to greater than 2024 and saw the stargate view creation go to .11 seconds but the second piece to load go to 40 minutes.

I could really use some advice on how to troubleshoot this and any help/recommendations on getting things back to the 10 minute range to process.

Thanks in advance