Cognos Analytics

Cognos Analytics

Connect, learn, and share with thousands of IBM Cognos Analytics users! 

 View Only
Expand all | Collapse all

Performance issue with Excel output

  • 1.  Performance issue with Excel output

    Posted Wed March 23, 2022 02:36 AM
    Hi,

    In Cognos 11.R7 we are using a data module to retrieve the data for the report,
    The reports consist of 5 pages with 43 columns and around 95k records per page (around 450k records for all pages).
    and we are using materialized view from DB, "select * from view"(All records) returning in just 5 seconds 
    but in the report, just for 2 pages it's taking around 30mins to download the data in excel, I'm not sure if I include all pages how much time it will take.

    In the data module, I changed measure columns to attribute or identifier where ever it's required.

    someone, please help me to optimize the performance of the report to retrieve the data in excel with minimalistic time.

    Thanks.


    ------------------------------
    P S
    ------------------------------

    #CognosAnalyticswithWatson


  • 2.  RE: Performance issue with Excel output

    Posted Wed March 23, 2022 05:03 AM
    Hi P S,

    You should try to create a PDF output to see if it is an issue with Excel output.

    The other idea would be to create a dataset for this report.

    Best regards,

    ------------------------------
    Patrick Neveu
    Positive Thinking Company
    ------------------------------



  • 3.  RE: Performance issue with Excel output

    Posted Wed March 23, 2022 06:54 AM
    HI Patrick,

    Even PDF also takes the same time as excel.

    can you please help me to create a dataset for the report.

    ------------------------------
    P S
    ------------------------------



  • 4.  RE: Performance issue with Excel output

    Posted Wed March 23, 2022 07:25 AM
    Hi P S,

    did you record the retrieval time for all records from the database? Normally the client fetches only the first n rows. Five seconds sounds a bit fast :).

    If the database is still much faster than the report, I suggest you check the gernated SQL. Maybe there is some local processing on the Cognos server that is slowing down the query. Or the SQL contains operations on the database that slow down the query there.


    ------------------------------
    Robert Dostal
    Team Leader BI
    GEMÜ
    Ingelfingen
    ------------------------------



  • 5.  RE: Performance issue with Excel output

    Posted Wed March 23, 2022 08:24 AM
    Hi Robert,
    Yes, I recorded the retrieval time for all records from the database,(Select * from the table)
    we are using materialized view for this report that the reason its retrieving in 5 seconds

    I checked the Cognos generated query its is also taking 5seconds to retrieve the data,

    is excel is the culprit here? to put the data in all 43 columns for 95k records.

    can you please suggest a solution for report performance?


    Thanks.


    ------------------------------
    P S
    ------------------------------



  • 6.  RE: Performance issue with Excel output

    Posted Wed March 23, 2022 09:43 AM
    Hi P S,

    I've tried a table with 460k records - so much larger than yours. I had no success with Excel either as it took forever. It was no problem to create the files as cvs and import it in excel afterwards. The report run for a few minutes and the file was finished.

    ------------------------------
    Robert Dostal
    Team Leader BI
    GEMÜ
    Ingelfingen
    ------------------------------



  • 7.  RE: Performance issue with Excel output

    Posted Wed March 23, 2022 09:52 AM
    It's a good thought to run in CSV but the challenge is the report has 5 different pages.
    if I run the report in CSV report is downloading for the only first page,
    data loss is for the 4 remaining pages.

    ------------------------------
    P S
    ------------------------------



  • 8.  RE: Performance issue with Excel output

    Posted Wed March 23, 2022 09:57 AM
    In this case a quick solution would be to create a schedule and let the report run as Excel file as background task in Cognos. It should finish...


    ------------------------------
    Robert Dostal
    Team Leader BI
    GEMÜ
    Ingelfingen
    ------------------------------



  • 9.  RE: Performance issue with Excel output

    Posted Wed March 23, 2022 10:05 AM
    Yes,
    but the user wants it to run manually,
    Is there any solution that we can optimize the report performance?

    ------------------------------
    P S
    ------------------------------



  • 10.  RE: Performance issue with Excel output

    Posted Thu March 24, 2022 12:07 PM
    Hi P.S.
    Only about 30% of the formatting option in Cognos Analytics is supported in excel, this is probably causing part of your issue. See details at https://www.ibm.com/docs/en/cognos-analytics/11.1.0?topic=lwprimef-unsupported-cognos-analytics-formatting-1.  I figure the application is attempting to address issues that it is finding and reformatting as best it can and that is what is eating up your time.  Another link that might help, determine some of the issues, or at least let you know what formatting not to use is- https://www.ibm.com/docs/en/cognos-analytics/11.1.0?topic=reporting-limitations-when-producing-reports-in-microsoft-excel-format  Hope this helps some.

    ------------------------------
    Michael Igau
    ------------------------------



  • 11.  RE: Performance issue with Excel output

    Posted Thu March 24, 2022 05:05 PM

    Hi,

    I'm going to paraphrase and slightly edit one of CognosPaul's epic answers from the past, as it neatly sums up what's going on here.

    The 5 seconds query execution time is not a valid comparison as all that's doing is taking a resultset from database A and dumping it into client B. I strongly doubt that the resultset would instantly appear in 5 seconds in a database client tool - even a client designed to handle large amounts of data still needs some processing time to draw the results into the grid.

    Let's talk about the results.

    43 columns * 95,000 records * 5 pages == 20,425,000 unique cells. Each cell has to be processed, formatted, and drawn. Cognos handles this by creating a temporary file on the server containing the query output. It then starts parsing through that file converting the results into an XML file. That file is then compressed. Unless you're talking about the old version of the Excel output, which is really a renamed uncompressed MHT file.

    20.4 million cells will take a long time to process. Even at 1,000 cells a second, we're talking about five and a half hours of processing. Considering it's being completed in 30 minutes, that means Cognos is processing 11,347 cells a second. Probably more because that's not including the time it takes to compress and deliver the report over the network.

    An ETL tool doesn't need to draw the output into a specific format, it can work with the dataset as it stands, which is why it would be so much faster.

    All credit for this goes, of course, to the legend who is Paul. He's a true Cognos great!

    Cheers!

    MF.



    ------------------------------
    Mark Fry
    Technical Consultant
    ------------------------------



  • 12.  RE: Performance issue with Excel output

    Posted Thu March 24, 2022 07:28 PM
    Hi,

    I guess that is a problem with Excel output indeed.

    If you try to export to Excel Data, you will notice that is faster than first one.

    Excel formatting in CA is slower than just exporting data in tabular format.

    I do not know if this problem is related to code/api used to convert report output to Excel format.

    ------------------------------
    JEAM COELHO
    Cognos Solution Architect

    LinkedIn: https://www.linkedin.com/in/jeamcoelho/
    ------------------------------