In general, building reports that show hundred thousands of lines of data is never a good idea. It's much better to show that data grouped or in a graph. That's my quick thought on the matter.

------------------------------
Joeri Willems

Original Message:
Sent: Thu October 09, 2025 09:02 AM
From: Dax Lawless
Subject: Interactive Report Execution Time Limit

I'd like to query the community to see if anyone has come up with a solution regarding users executing Detail reports that cover excessive timeframes (such as an entire month of detail data vs summary data) that results in millions of rows returned.  Ideally there would be a way to break up a large resultset into smaller chunks that are executed sequentially. I'd like to do this as transparently as possible.  

--

Dax Lawless

Cslt III Spec-PS Consulting
Consulting & Advisory Services
Verizon Business Group

M 3195382382

Let's say you have had the discussion with the users and after all the reasons not to do this - they are still insistent on this as a requirement, and that they must have a report with a million rows in it.

There is obviously no point in having the report interactively - what are they going to do with a million rows of data within a browser? By the time they have finished reading a million rows the data will be out of date. I would argue the same for PDF - what would you do with a million rows of data in a PDF?

Which sort of leaves either Excel or one of the other data formats such as CSV.

Within a single report you could parallel run multiple queries - maybe split them up by master detail. But this would only work in Excel format. The other formats just run a single query in the report.

And after the report has finished parallel running the queries there is the overhead of assembling the extracted data in to as single excel file which can't be done parallel.

Whilst Cognos can do this requirement - I would say go back to basics, what exactly is the requirement. Is it a report or data extraction? if the latter then look at just running extracts directly against the database.

I'd like to query the community to see if anyone has come up with a solution regarding users executing Detail reports that cover excessive timeframes (such as an entire month of detail data vs summary data) that results in millions of rows returned.  Ideally there would be a way to break up a large resultset into smaller chunks that are executed sequentially. I'd like to do this as transparently as possible.  

--

Dax Lawless

Cslt III Spec-PS Consulting
Consulting & Advisory Services
Verizon Business Group

M 3195382382

Let's say you have had the discussion with the users and after all the reasons not to do this - they are still insistent on this as a requirement, and that they must have a report with a million rows in it.

There is obviously no point in having the report interactively - what are they going to do with a million rows of data within a browser? By the time they have finished reading a million rows the data will be out of date. I would argue the same for PDF - what would you do with a million rows of data in a PDF?

Which sort of leaves either Excel or one of the other data formats such as CSV.

Within a single report you could parallel run multiple queries - maybe split them up by master detail. But this would only work in Excel format. The other formats just run a single query in the report.

And after the report has finished parallel running the queries there is the overhead of assembling the extracted data in to as single excel file which can't be done parallel.

Whilst Cognos can do this requirement - I would say go back to basics, what exactly is the requirement. Is it a report or data extraction? if the latter then look at just running extracts directly against the database.

I'd like to query the community to see if anyone has come up with a solution regarding users executing Detail reports that cover excessive timeframes (such as an entire month of detail data vs summary data) that results in millions of rows returned.  Ideally there would be a way to break up a large resultset into smaller chunks that are executed sequentially. I'd like to do this as transparently as possible.  

--

Dax Lawless

Cslt III Spec-PS Consulting
Consulting & Advisory Services
Verizon Business Group

M 3195382382

Unfortunately, having "technical limitation" conversations with individual users might work when dealing with an internal organization, there is still no solution to the problem that would prevent any particular user from breaking a system, as ours is intended as a multi-user/multi-organizational system of external users.  Documenting "best practices" doesn't prevent bad user behavior.

@Marc Reed:

In this particular situation, the report in question is intended to be run as a batch report via a schedule that has relatively fixed parameters that results in a csv/excel file that is delivered on the back end via MFT. 

However, this particular user wanted the same detail data that unfortunately resulted in a massive data dump due to the availability of the scope of the prompts in question.   

This impacted the disk read/write performance (due to file caching), and overall system memory usage, which in turn impacted the experience of other users.

As far as solutions are concerned, I was looking for a technical solution that would cancel the report run if it exceeded a certain runtime -- normally this is set via the tuning setting "Queue time limit of report service (seconds)".  However, apparently that setting doesn't work as described, as this report ran (interactively) for several hours.  It is set to something like 240 seconds. 

If there are specifics to the tuning setting that aren't generally available, I'd like to know.

Unfortunately, our system has hundreds of basically what amount to unique reports that makes re-development time-consuming/problematic.  

Let's say you have had the discussion with the users and after all the reasons not to do this - they are still insistent on this as a requirement, and that they must have a report with a million rows in it.

There is obviously no point in having the report interactively - what are they going to do with a million rows of data within a browser? By the time they have finished reading a million rows the data will be out of date. I would argue the same for PDF - what would you do with a million rows of data in a PDF?

Which sort of leaves either Excel or one of the other data formats such as CSV.

Within a single report you could parallel run multiple queries - maybe split them up by master detail. But this would only work in Excel format. The other formats just run a single query in the report.

And after the report has finished parallel running the queries there is the overhead of assembling the extracted data in to as single excel file which can't be done parallel.

Whilst Cognos can do this requirement - I would say go back to basics, what exactly is the requirement. Is it a report or data extraction? if the latter then look at just running extracts directly against the database.

I'd like to query the community to see if anyone has come up with a solution regarding users executing Detail reports that cover excessive timeframes (such as an entire month of detail data vs summary data) that results in millions of rows returned.  Ideally there would be a way to break up a large resultset into smaller chunks that are executed sequentially. I'd like to do this as transparently as possible.  

--

Dax Lawless

Cslt III Spec-PS Consulting
Consulting & Advisory Services
Verizon Business Group

M 3195382382

Unfortunately, having "technical limitation" conversations with individual users might work when dealing with an internal organization, there is still no solution to the problem that would prevent any particular user from breaking a system, as ours is intended as a multi-user/multi-organizational system of external users.  Documenting "best practices" doesn't prevent bad user behavior.

@Marc Reed:

In this particular situation, the report in question is intended to be run as a batch report via a schedule that has relatively fixed parameters that results in a csv/excel file that is delivered on the back end via MFT. 

However, this particular user wanted the same detail data that unfortunately resulted in a massive data dump due to the availability of the scope of the prompts in question.   

This impacted the disk read/write performance (due to file caching), and overall system memory usage, which in turn impacted the experience of other users.

As far as solutions are concerned, I was looking for a technical solution that would cancel the report run if it exceeded a certain runtime -- normally this is set via the tuning setting "Queue time limit of report service (seconds)".  However, apparently that setting doesn't work as described, as this report ran (interactively) for several hours.  It is set to something like 240 seconds. 

If there are specifics to the tuning setting that aren't generally available, I'd like to know.

Unfortunately, our system has hundreds of basically what amount to unique reports that makes re-development time-consuming/problematic.  

Let's say you have had the discussion with the users and after all the reasons not to do this - they are still insistent on this as a requirement, and that they must have a report with a million rows in it.

There is obviously no point in having the report interactively - what are they going to do with a million rows of data within a browser? By the time they have finished reading a million rows the data will be out of date. I would argue the same for PDF - what would you do with a million rows of data in a PDF?

Which sort of leaves either Excel or one of the other data formats such as CSV.

Within a single report you could parallel run multiple queries - maybe split them up by master detail. But this would only work in Excel format. The other formats just run a single query in the report.

And after the report has finished parallel running the queries there is the overhead of assembling the extracted data in to as single excel file which can't be done parallel.

Whilst Cognos can do this requirement - I would say go back to basics, what exactly is the requirement. Is it a report or data extraction? if the latter then look at just running extracts directly against the database.

I'd like to query the community to see if anyone has come up with a solution regarding users executing Detail reports that cover excessive timeframes (such as an entire month of detail data vs summary data) that results in millions of rows returned.  Ideally there would be a way to break up a large resultset into smaller chunks that are executed sequentially. I'd like to do this as transparently as possible.  

--

Dax Lawless

Cslt III Spec-PS Consulting
Consulting & Advisory Services
Verizon Business Group

M 3195382382