Planning Analytics

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Hi Philippe,

If you on your data tab start with one variable:

• vTest = 1;
• comment out the rest of the statement

How does it the perform?

Then comment in sections, one by on, from the top. What sections are causing the issue?

I see you use @>'' / @>' ' quite allot. When comparing numbers, it is allot more efficient to use numeric variables, then to convert strings for each test.

• One more question: Are you getting any errors when running your process? It the rollback being triggeret? 

Another small thing. You should move this part to prolog. You can do it once there, no need to do it 15k times on the metadata tab :-)

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Hi Philippe,

If you on your data tab start with one variable:

• vTest = 1;
• comment out the rest of the statement

How does it the perform?

Then comment in sections, one by on, from the top. What sections are causing the issue?

I see you use @>'' / @>' ' quite allot. When comparing numbers, it is allot more efficient to use numeric variables, then to convert strings for each test.

• One more question: Are you getting any errors when running your process? It the rollback being triggeret? 

Another small thing. You should move this part to prolog. You can do it once there, no need to do it 15k times on the metadata tab :-)

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Hi Emil

I try a lot of thing hand have the solution (but not the explanation !!!!!)

I Put "Logging"=NO for these cubes
 }DimensionProperties
 }ElementAttributes_Dim_Article
 }ElementAttributes_Dim_w_Article
 }LocalizedElementAttributes_Dim_Article
 }localizedElementAttributes_Dim_w_Article

I Run my TI with several tests :

1°) without anything in DataSection :
==>    3.65 seconds

é)) With Only First CellPutS : 
  CellPuts( CellGetS('}ElementAttributes_Dim_w_Article',Element, 'NewSsFam'),'}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> 29 minutes
(and lot of locks on }DimensionProperties : Thread xxxxx holds the lock in IX mode)

3°) Try to change this alone instruction by :
 CellPuts( 'Nimportequoi','}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> Manualy killed after 5 minutes

4°) Try to use a static subset (and not a mdx subset) as datasource
==> finished in 4 seconds

Try the complete Process with a static Subset in datasource
==> complete finish in 11 seconds.

Then , I think that use a MDX Subset is the expplanation, I will use A Static Subset, create in other process.

BUT WHY ????

Regards,

Philippe

Hi Philippe,

If you on your data tab start with one variable:

• vTest = 1;
• comment out the rest of the statement

How does it the perform?

Then comment in sections, one by on, from the top. What sections are causing the issue?

I see you use @>'' / @>' ' quite allot. When comparing numbers, it is allot more efficient to use numeric variables, then to convert strings for each test.

• One more question: Are you getting any errors when running your process? It the rollback being triggeret? 

Another small thing. You should move this part to prolog. You can do it once there, no need to do it 15k times on the metadata tab :-)

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Can you share the subset as the MDX will refire if there is a change in the underlying drivers of the dynamic set e.g. if you are filtering on an attribute and you are updating the the attributes, the MDX needs to refresh to ensure you are seeing a current set.

Hi Emil

I try a lot of thing hand have the solution (but not the explanation !!!!!)

I Put "Logging"=NO for these cubes
 }DimensionProperties
 }ElementAttributes_Dim_Article
 }ElementAttributes_Dim_w_Article
 }LocalizedElementAttributes_Dim_Article
 }localizedElementAttributes_Dim_w_Article

I Run my TI with several tests :

1°) without anything in DataSection :
==>    3.65 seconds

é)) With Only First CellPutS : 
  CellPuts( CellGetS('}ElementAttributes_Dim_w_Article',Element, 'NewSsFam'),'}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> 29 minutes
(and lot of locks on }DimensionProperties : Thread xxxxx holds the lock in IX mode)

3°) Try to change this alone instruction by :
 CellPuts( 'Nimportequoi','}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> Manualy killed after 5 minutes

4°) Try to use a static subset (and not a mdx subset) as datasource
==> finished in 4 seconds

Try the complete Process with a static Subset in datasource
==> complete finish in 11 seconds.

Then , I think that use a MDX Subset is the expplanation, I will use A Static Subset, create in other process.

BUT WHY ????

Regards,

Philippe

Hi Philippe,

If you on your data tab start with one variable:

• vTest = 1;
• comment out the rest of the statement

How does it the perform?

Then comment in sections, one by on, from the top. What sections are causing the issue?

I see you use @>'' / @>' ' quite allot. When comparing numbers, it is allot more efficient to use numeric variables, then to convert strings for each test.

• One more question: Are you getting any errors when running your process? It the rollback being triggeret? 

Another small thing. You should move this part to prolog. You can do it once there, no need to do it 15k times on the metadata tab :-)

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Hello George,

The Subset  I use as datasource in on  "Dim_W_Article" dimension and I manage an other  dimension "Dim_Article". (forr me, no need to refresh, because datasource not updated)

The MDX is so simple : << [Dim_W_Article].[Dim_W_Article].MEMBERS >>

I don't understand why this comportment arrive.

i am going to workaround with a static subset created in ohter previous Ti

( SubsetMDXSet with mdx formula followed by  SubsetMDXSet with empty MdxFormula)

I have given thes information to IBM Support to have an explanation (perhaps)

Best regards,

philippe

Can you share the subset as the MDX will refire if there is a change in the underlying drivers of the dynamic set e.g. if you are filtering on an attribute and you are updating the the attributes, the MDX needs to refresh to ensure you are seeing a current set.

Hi Emil

I try a lot of thing hand have the solution (but not the explanation !!!!!)

I Put "Logging"=NO for these cubes
 }DimensionProperties
 }ElementAttributes_Dim_Article
 }ElementAttributes_Dim_w_Article
 }LocalizedElementAttributes_Dim_Article
 }localizedElementAttributes_Dim_w_Article

I Run my TI with several tests :

1°) without anything in DataSection :
==>    3.65 seconds

é)) With Only First CellPutS : 
  CellPuts( CellGetS('}ElementAttributes_Dim_w_Article',Element, 'NewSsFam'),'}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> 29 minutes
(and lot of locks on }DimensionProperties : Thread xxxxx holds the lock in IX mode)

3°) Try to change this alone instruction by :
 CellPuts( 'Nimportequoi','}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> Manualy killed after 5 minutes

4°) Try to use a static subset (and not a mdx subset) as datasource
==> finished in 4 seconds

Try the complete Process with a static Subset in datasource
==> complete finish in 11 seconds.

Then , I think that use a MDX Subset is the expplanation, I will use A Static Subset, create in other process.

BUT WHY ????

Regards,

Philippe

Hi Philippe,

If you on your data tab start with one variable:

• vTest = 1;
• comment out the rest of the statement

How does it the perform?

Then comment in sections, one by on, from the top. What sections are causing the issue?

I see you use @>'' / @>' ' quite allot. When comparing numbers, it is allot more efficient to use numeric variables, then to convert strings for each test.

• One more question: Are you getting any errors when running your process? It the rollback being triggeret? 

Another small thing. You should move this part to prolog. You can do it once there, no need to do it 15k times on the metadata tab :-)

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

HI Philippe, so there can be a huge distinction between elements and members.

Going to try explain and assuming you do not know any of this so apologies if you do and not relevant, but may be helpful to others.

TM1SubsetAll() will give you the elements
[...].Members will give you members

Elements are not always Members as an element that has two parents is essentially two Members, made distinct by their Member_Unique_Name (think of the fully qualified name e.g. [ParentsParent^Parent^Element] when you copy unique name in the old architect set editor.

If you have many rollups in the default hierarchy and you have linked in the Leaf elements into these, there could be many times more Members than Elements.
Furthermore, if for example the element has multiple parents the set is likely to contain the element multiple times and your process would process the same item multiple times.

With all of that being said, your set may just have more members/duplicates than you want but does not explain why it should refire/refresh.
Does your set have an alias set perhaps? Updating the attributes could trigger a refire.

I cannot see that you are making any other changes to the Dim_W_Article dimension i.e. insert or delete elements/members, so not expecting the set to refresh.

Hello George,

The Subset  I use as datasource in on  "Dim_W_Article" dimension and I manage an other  dimension "Dim_Article". (forr me, no need to refresh, because datasource not updated)

The MDX is so simple : << [Dim_W_Article].[Dim_W_Article].MEMBERS >>

I don't understand why this comportment arrive.

i am going to workaround with a static subset created in ohter previous Ti

( SubsetMDXSet with mdx formula followed by  SubsetMDXSet with empty MdxFormula)

I have given thes information to IBM Support to have an explanation (perhaps)

Best regards,

philippe

Can you share the subset as the MDX will refire if there is a change in the underlying drivers of the dynamic set e.g. if you are filtering on an attribute and you are updating the the attributes, the MDX needs to refresh to ensure you are seeing a current set.

Hi Emil

I try a lot of thing hand have the solution (but not the explanation !!!!!)

I Put "Logging"=NO for these cubes
 }DimensionProperties
 }ElementAttributes_Dim_Article
 }ElementAttributes_Dim_w_Article
 }LocalizedElementAttributes_Dim_Article
 }localizedElementAttributes_Dim_w_Article

I Run my TI with several tests :

1°) without anything in DataSection :
==>    3.65 seconds

é)) With Only First CellPutS : 
  CellPuts( CellGetS('}ElementAttributes_Dim_w_Article',Element, 'NewSsFam'),'}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> 29 minutes
(and lot of locks on }DimensionProperties : Thread xxxxx holds the lock in IX mode)

3°) Try to change this alone instruction by :
 CellPuts( 'Nimportequoi','}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> Manualy killed after 5 minutes

4°) Try to use a static subset (and not a mdx subset) as datasource
==> finished in 4 seconds

Try the complete Process with a static Subset in datasource
==> complete finish in 11 seconds.

Then , I think that use a MDX Subset is the expplanation, I will use A Static Subset, create in other process.

BUT WHY ????

Regards,

Philippe

Hi Philippe,

If you on your data tab start with one variable:

• vTest = 1;
• comment out the rest of the statement

How does it the perform?

Then comment in sections, one by on, from the top. What sections are causing the issue?

I see you use @>'' / @>' ' quite allot. When comparing numbers, it is allot more efficient to use numeric variables, then to convert strings for each test.

• One more question: Are you getting any errors when running your process? It the rollback being triggeret? 

Another small thing. You should move this part to prolog. You can do it once there, no need to do it 15k times on the metadata tab :-)

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Thank George,

But my Dim_W_Article dimension has only leaf members, I could Use tm1filterByLevel( Tm1Subsetall( [dim_W_Article] ) , 0 ) I would have the same as "Members".

And During TI execution not nay change are made on my datasource Dimension (Dim_W_Article) , change are made on an iother dimension (Dim_Article).

As some people used to say, "The truth is out there" (I think).

I change my tI and use a static Subset on datasource, then everything works perfectly 

But I still don't know why I have to use this workaround.
If anyone would like to do a test on their own environment ....

Regards,

Philippe

Regards,

Philippe

HI Philippe, so there can be a huge distinction between elements and members.

Going to try explain and assuming you do not know any of this so apologies if you do and not relevant, but may be helpful to others.

TM1SubsetAll() will give you the elements
[...].Members will give you members

Elements are not always Members as an element that has two parents is essentially two Members, made distinct by their Member_Unique_Name (think of the fully qualified name e.g. [ParentsParent^Parent^Element] when you copy unique name in the old architect set editor.

If you have many rollups in the default hierarchy and you have linked in the Leaf elements into these, there could be many times more Members than Elements.
Furthermore, if for example the element has multiple parents the set is likely to contain the element multiple times and your process would process the same item multiple times.

With all of that being said, your set may just have more members/duplicates than you want but does not explain why it should refire/refresh.
Does your set have an alias set perhaps? Updating the attributes could trigger a refire.

I cannot see that you are making any other changes to the Dim_W_Article dimension i.e. insert or delete elements/members, so not expecting the set to refresh.

Hello George,

The Subset  I use as datasource in on  "Dim_W_Article" dimension and I manage an other  dimension "Dim_Article". (forr me, no need to refresh, because datasource not updated)

The MDX is so simple : << [Dim_W_Article].[Dim_W_Article].MEMBERS >>

I don't understand why this comportment arrive.

i am going to workaround with a static subset created in ohter previous Ti

( SubsetMDXSet with mdx formula followed by  SubsetMDXSet with empty MdxFormula)

I have given thes information to IBM Support to have an explanation (perhaps)

Best regards,

philippe

Can you share the subset as the MDX will refire if there is a change in the underlying drivers of the dynamic set e.g. if you are filtering on an attribute and you are updating the the attributes, the MDX needs to refresh to ensure you are seeing a current set.

Hi Emil

I try a lot of thing hand have the solution (but not the explanation !!!!!)

I Put "Logging"=NO for these cubes
 }DimensionProperties
 }ElementAttributes_Dim_Article
 }ElementAttributes_Dim_w_Article
 }LocalizedElementAttributes_Dim_Article
 }localizedElementAttributes_Dim_w_Article

I Run my TI with several tests :

1°) without anything in DataSection :
==>    3.65 seconds

é)) With Only First CellPutS : 
  CellPuts( CellGetS('}ElementAttributes_Dim_w_Article',Element, 'NewSsFam'),'}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> 29 minutes
(and lot of locks on }DimensionProperties : Thread xxxxx holds the lock in IX mode)

3°) Try to change this alone instruction by :
 CellPuts( 'Nimportequoi','}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> Manualy killed after 5 minutes

4°) Try to use a static subset (and not a mdx subset) as datasource
==> finished in 4 seconds

Try the complete Process with a static Subset in datasource
==> complete finish in 11 seconds.

Then , I think that use a MDX Subset is the expplanation, I will use A Static Subset, create in other process.

BUT WHY ????

Regards,

Philippe

Hi Philippe,

If you on your data tab start with one variable:

• vTest = 1;
• comment out the rest of the statement

How does it the perform?

Then comment in sections, one by on, from the top. What sections are causing the issue?

I see you use @>'' / @>' ' quite allot. When comparing numbers, it is allot more efficient to use numeric variables, then to convert strings for each test.

• One more question: Are you getting any errors when running your process? It the rollback being triggeret? 

Another small thing. You should move this part to prolog. You can do it once there, no need to do it 15k times on the metadata tab :-)

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :

Hi Philippe,

I haven't seen any general issues, so it probably comes down to the script. What's the size of the finished dimension, that takes 25 min to load?

Have you tried to isolate what is causing the issues? You can start by commenting your script out on all tabs, and just adding a "dummy" variable on data/metadata (e.g. vDummy = 1;) to force the process to load your source. Then comment in your script on prolog, metadata, data etc. to see where the issue resides.

Could you share the scripts you have done for the process you have not, that takes 25 min? (Prolog, Metadata, Data and epilog). I will be happy to take a look at it.

Hello,

For a new customer (and a new project) , I install Planning Analytics on premice on windows 2016 with these versions :

PAW : 2.0.93

PA 2.0.9.19

(and PAFE 2.0.94.7)

I started development and was confronted with behavior I'd never seen before: locks on }DimensionProperties and a horribly long execution time.

Several explanation :

  1 - Locks

     I launch the execution of a first TI that updates a dimension (metadata and data) and, while it's running, launch the execution of another TI process that updates another dimension (or, alternatively, manually attempt to modify a dimension through the workspace).

    the first TI has put such locks on (a priori) }dimensionproperties that all other actions (other TI or manually dimension update) are in WAIT:IXC until ... the end of the TI !!!

  2 - Duration

Inevitably, my first TI, just to complicate the situation, is very long (strangely and inexplicably long!).

because I had several different files to feed my dimension, I proceeded in two stages:
The first step (with several TIs) feeds a working Dim_w_xxx dimension with its elemensts and attributes.
a second step with a final ti that reads (vis subset) my working dimension and completely manages my final dimension ( prolog=deleteallelement, metadata=add element, component , data read attributes and write attributes).

This last process has a duration of  25' !!!!! ( metadata aproximativly 10'')

i try this :

   Read Work Dimension

   Metadata magagedimension

   data : export file (asciioutput)

 and a second ti read this file and manage attributs (data section)

Total duration : 25''

Thank George,

But my Dim_W_Article dimension has only leaf members, I could Use tm1filterByLevel( Tm1Subsetall( [dim_W_Article] ) , 0 ) I would have the same as "Members".

And During TI execution not nay change are made on my datasource Dimension (Dim_W_Article) , change are made on an iother dimension (Dim_Article).

As some people used to say, "The truth is out there" (I think).

I change my tI and use a static Subset on datasource, then everything works perfectly 

But I still don't know why I have to use this workaround.
If anyone would like to do a test on their own environment ....

Regards,

Philippe

Regards,

Philippe

HI Philippe, so there can be a huge distinction between elements and members.

Going to try explain and assuming you do not know any of this so apologies if you do and not relevant, but may be helpful to others.

TM1SubsetAll() will give you the elements
[...].Members will give you members

Elements are not always Members as an element that has two parents is essentially two Members, made distinct by their Member_Unique_Name (think of the fully qualified name e.g. [ParentsParent^Parent^Element] when you copy unique name in the old architect set editor.

If you have many rollups in the default hierarchy and you have linked in the Leaf elements into these, there could be many times more Members than Elements.
Furthermore, if for example the element has multiple parents the set is likely to contain the element multiple times and your process would process the same item multiple times.

With all of that being said, your set may just have more members/duplicates than you want but does not explain why it should refire/refresh.
Does your set have an alias set perhaps? Updating the attributes could trigger a refire.

I cannot see that you are making any other changes to the Dim_W_Article dimension i.e. insert or delete elements/members, so not expecting the set to refresh.

Hello George,

The Subset  I use as datasource in on  "Dim_W_Article" dimension and I manage an other  dimension "Dim_Article". (forr me, no need to refresh, because datasource not updated)

The MDX is so simple : << [Dim_W_Article].[Dim_W_Article].MEMBERS >>

I don't understand why this comportment arrive.

i am going to workaround with a static subset created in ohter previous Ti

( SubsetMDXSet with mdx formula followed by  SubsetMDXSet with empty MdxFormula)

I have given thes information to IBM Support to have an explanation (perhaps)

Best regards,

philippe

Can you share the subset as the MDX will refire if there is a change in the underlying drivers of the dynamic set e.g. if you are filtering on an attribute and you are updating the the attributes, the MDX needs to refresh to ensure you are seeing a current set.

Hi Emil

I try a lot of thing hand have the solution (but not the explanation !!!!!)

I Put "Logging"=NO for these cubes
 }DimensionProperties
 }ElementAttributes_Dim_Article
 }ElementAttributes_Dim_w_Article
 }LocalizedElementAttributes_Dim_Article
 }localizedElementAttributes_Dim_w_Article

I Run my TI with several tests :

1°) without anything in DataSection :
==>    3.65 seconds

é)) With Only First CellPutS : 
  CellPuts( CellGetS('}ElementAttributes_Dim_w_Article',Element, 'NewSsFam'),'}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> 29 minutes
(and lot of locks on }DimensionProperties : Thread xxxxx holds the lock in IX mode)

3°) Try to change this alone instruction by :
 CellPuts( 'Nimportequoi','}ElementAttributes_Dim_Article',Element,'NewSsFam');
==> Manualy killed after 5 minutes

4°) Try to use a static subset (and not a mdx subset) as datasource
==> finished in 4 seconds

Try the complete Process with a static Subset in datasource
==> complete finish in 11 seconds.

Then , I think that use a MDX Subset is the expplanation, I will use A Static Subset, create in other process.

BUT WHY ????

Regards,

Philippe

Hi Philippe,

If you on your data tab start with one variable:

• vTest = 1;
• comment out the rest of the statement

How does it the perform?

Then comment in sections, one by on, from the top. What sections are causing the issue?

I see you use @>'' / @>' ' quite allot. When comparing numbers, it is allot more efficient to use numeric variables, then to convert strings for each test.

• One more question: Are you getting any errors when running your process? It the rollback being triggeret? 

Another small thing. You should move this part to prolog. You can do it once there, no need to do it 15k times on the metadata tab :-)

Thanks Emil !

I admit I don't understand what's going on either, it's the first time I've been confronted with it.

I already make a test .... Metadata aproximativly 5 seconds (and work as fine I wanted) , Data 25 minutes.

This my TI code :