Cognos Analytics

Cognos Analytics

Connect, learn, and share with thousands of IBM Cognos Analytics users! 

 View Only
Expand all | Collapse all

Dashboard performs worse on an Oracle database than on Cognos Datasets

  • 1.  Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 11 days ago

    Good morning!

    As the title says: I have made a number of datasets in Cognos, and connected them logically in a data module. On this module I built a dashboard that performs perfectly.

    To save the local cognos server, I asked our local ETL specialist to recreated the datasets with ETL, and to write them off in an Oracle database. These tables I have then connected again in a data module, in the exact same way as I did with the datasets. On this module I built an identical dashboard as well. The performance of this dashboard is absolutely horrable, and most visualisations time-out on loading.

    The fact table in question is a table with about 130.000 records, using several foreign keys for which dimensions were built. Most of these dimensions are 1:N to the fact table, but I also built a date dimension that is N:N, covering the time between START_DATE and END_DATE. When using this dimension, the number of records increases to 170 million.

    Am I doing something wrong, or is the discrepancy in performance between the Oracle-database and the Datasets to be expected?

    Thanks for your insights!



    ------------------------------
    Matthijs Oosterloo
    ------------------------------


  • 2.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 9 days ago

    Hi Matthijs,

    You can store a dataset to IBM db2: You can use a db2 database to store data from uploaded files and data sets. The data is stored as relational tables, and managed by the Data sets server connection in Cognos® Analytics.

    Source: Storing data from uploaded files and data sets in Db2 - IBM Documentation

    For this feature, only db2 is supported, not Oracle.

    Best regards,



    ------------------------------
    Patrick Neveu
    BSL Consulting
    IBM Champion
    ------------------------------



  • 3.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 9 days ago

    Hi Matthijs,

    You might also want to check you are using a supported Oracle version, and a supported JDBC driver.

    Best regards,



    ------------------------------
    Patrick Neveu
    BSL Consulting
    IBM Champion
    ------------------------------



  • 4.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 9 days ago

    Hi Matthijs,

    you should also compare the size of your dataset with the number of rows in the table of your Oracle DB.

    The question is: you wrote that there's a n:n relationshiop with the calendar table. Is this relationship resovled on the dataset and the Oracle table as well or dou you have a star schema in the database?



    ------------------------------
    Robert Dostal
    Principal Expert BI
    GEMÜ Gebr. Müller Apparatebau GmbH & Co. KG
    Kupferzell
    ------------------------------



  • 5.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 8 days ago

    Good morning,

    The oracle database only contains tables: no star schema is created there.

    In other words:

    Dataset solution: fact and dimension tables are created as individual tables in datasets. -> The required datasets are logically connected in a datamodule. ->A dashboard is built on the data module

    Database solution: fact and dimension tables are created as individual tables in an oracle database. -> A connection between cognos and the database is made -> The required tables are loaded into a datamodule -> The required tables are logically connected in said datamodule. ->A dashboard is built on the data module

    Each dataset and their database-table-counterpart have the exact same number of rows and columns.



    ------------------------------
    Matthijs Oosterloo
    ------------------------------



  • 6.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 7 days ago
    Edited by John Cusack 6 days ago

    Hi Matthijs

    I would suggest capturing the SQL that is being sent to the Oracle DB to help you figure out what type of SQL is being generated and processed at the DB level.  Take that sql and toss it into SQL Developer/Toad/whatever development tool you have and see if that responds within your expectations.



    ------------------------------
    John Cusack
    Analytica iQ

    ------------------------------



  • 7.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 6 days ago

    There's some more information here.  I've included what I think is particularly relevant below.  
     Why do you think you need a N:N relationship?  Also could you answer Robert's question please.

    https://www.cognoise.com/index.php/topic,38462.msg126163.html

    The START_DATE and END_DATE indeed show the duration of a trajectory. Both columns are used for different use-cases. In this case I want to report on the number of active trajectories at a given time, using a N:N connection between the two dates and a year-date calendar, s.t.:

    FACT1.START_DATE >= CALENDAR.DATE
    FACT1.EIND_DATE <= CALENDAR.DATE



    ------------------------------
    Ian Henderson
    ------------------------------



  • 8.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 6 days ago

    I would also (more or less) presume that the n:n relationship which sounds like a cross join between fact and calendar table is one of the root causes here.
    You wrote that 170M records is the result when you combine fact and calendar table. If this is the end result after querying both tables on the database, then your dataset should also contain 170M records. Is that true?

    I've never experienced a huge discrepancy between a dataset and the database when we talk about smaller tables. Yes, Cognos has to take more steps to get data from a db vendor than from a dataset but that's due to the concept that in a database you usually have multiple tables that have to be joined on-the-fly. The response should be visible in nearly the same amount of time. 

    That said, if your dataset is only 130k rows in size as this is the end result of the summarized query of fact and calendar table but  "on-the-fly" amount of records that must be processed by the database is 170m - then I don't  wonder about bad performance.

    For me it sounds like the issue we once had with our budget data. The data was stored at month level but users wanted to see it on a daily basis to compare iot with YTD actual data. I then joined the budget table with a few millions of records with the calendar table to get the end result on a day level and then summarize that back to a year level. The temporarily amount of data were roughly 260m records and the queries took long time to run - despite the fact that the result was just a frew thousand lines.



    ------------------------------
    Robert Dostal
    Principal Expert BI
    GEMÜ Gebr. Müller Apparatebau GmbH & Co. KG
    Kupferzell
    ------------------------------



  • 9.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 6 days ago

    Thanks for your help! I do not actually combine the two tables into one single dataset if that is what you mean. However, to plot de data with the two tables combined, I suspect something similar to what a combined table would look like is created in the background. Or is that assumption wrong? 

    I understand the example you give me. I understand that combining two big tables might cause issues for Cognos. The strange thing however, is that creating a Dataset with only the trajectory information, and a Dataset with only the calendar information in cognos, and connecting the two in a Datamodule, cause no problems when creating a visualisation on it. However, doing the exact same thing with two oracle tables, that are connected to cognos and then combined in a Datamodule causes the time out. In other words: it seems Cognos can handle data in Datasets a lot more efficient than data from an oracle database table.



    ------------------------------
    Matthijs Oosterloo
    ------------------------------



  • 10.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 6 days ago

    Thanks for your reply.
    I indeed also posted this issue on Cognoise.

    Concerning your question:

    I have a list of clients who are provided a service. At any given point in time, one client may be provided multiple services (in other words: a client can be in several trajectories at a given time). For my business case, I want to know the number of unique clients at a given date.

    To do this, I make a N:N connection between my FACT_Trajectories and DIM_Dates (since one date can be part of many trajectories, and one trajectory can be part of many dates), and put a count distinct on the client number. 



    ------------------------------
    Matthijs Oosterloo
    ------------------------------



  • 11.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 7 days ago

    Hi, I recently attended an excellent webinar titled Cognos Analytics Tips & Tricks and AMA.  If you did not attend I would recommend taking a look.  There was a great deal of valuable material presented about how to evaluate the SQL being generated in CA which I think would be extremely helpful in diagnosing your issue with the Oracle implementation of your data sets.  I have included a link to the replay.  Thanks. https://ibm.webcasts.com/starthere.jsp?ei=1741568&tp_key=8a951355c7



    ------------------------------
    Bob Smith
    ------------------------------



  • 12.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 6 days ago

    Hi,

    without knowing what your model looks like and how you have defined all of the relationships its a bit of a guess but what you are describing with the row counts sounds very much like a modelling issue. 

    Have you created an alias of your calendar/date dimension. One as the start date and one as the end date?

    Standard Star scheme rules would then expect a 1:n join from each new calendar dim to your fact.



    ------------------------------
    jonathan chesterton
    NHS Supply Chain
    ------------------------------



  • 13.  RE: Dashboard performs worse on an Oracle database than on Cognos Datasets

    Posted 6 days ago
    Edited by Ralf Roeber 5 days ago
      |   view attached

    Hello @Matthijs Oosterloo

    Your are stating that you have a performance issue OracleDB vs. Datasets.

    Have you reviewed the resulting data set? Is the data correct? If the data is correct, then you need to tune your sql + Fact-Tables. To do this, start with getting the execution plan of the statement. I have seen things like: outdated indices (the DB ADMs just forgot to schedule the update), wrong estimation of distributed indices, wrong index: combined with reverse search should have been chosen and so on. Materialized Views are your golden sword. This needs someone with ADM access to the DB and good understanding of SQL engine. Happy to help you further on this.

    You are stating that you are using 1:N. I believe you should use 1:1 instead.

    Because Cognos behaves different when using to 1:N vs 1:1.

    1:N, 0:N are stiched queries which need to be stiched (may not be executed 100% on Database Server)

    1:1, 0:1 are one-to many queries, which can be fully executed on the database

    I am attaching a document (in german from 2005 - how time flies by :-D ) which explains the relation of families between 3 villages using mapping tables.

    Depending on your join 1:N or 1:1 or 0:N or 0:1 you receive completely different results - and performance.

    TL;DR: Always use 0:1, 1:1 unless it is understood what 1:N does.

    Let me know, if I should translate it into english.

    I can also provide you with the FM-model, the report and SQL for MS-SQL to populate a sample database.

    Hope this helps.



    ------------------------------
    Ralf Roeber
    https://linkedin.com/in/ralf-roeber/
    ------------------------------

    Attachment(s)