In ODM on Kubernetes I have a test suite with 1000 scenarios & almost 8000 rules. When trying to run this test suite from the Decision Center, I get the 504 Bad Gateway error. What tuning can be performed to mitigate this problem?

While running test suites, primarily decision center and decision runner pods are involved. Hence tuning is needed on both these pods.

Decision Runner pod

Ensure that enough memory is allotted to the container.
Ensure your decision server database is on a fast disk.
Create an index on the SCENARIOSUITEELEMENT table in the decision server database as follows:
create index sse_idx1 on SCENARIOSUITEELEMENT ( job_execution_id, element_id, element_type);
Change the timeout value of the load balancer in front of the decision runner to at least 600 (10 minutes).

Decision Center pod
Ensure that enough memory is allotted to the container.
Change the timeout value of the load balancer in front of decision center to at least 600 (10 minutes).
Ensure that you understand and wisely select what needs to be displayed in the test run report based on how costly it is to retrieve from the decision server database and store it in the decision center database.

The number of rules fired => not expansive (just a number) 

The number of executed ruleflow tasks => not expansive (just a number)

The number of rules not fired => not expansive (just a number)

The number of not executed ruleflow tasks => not expansive (just a number)

The list of rules not fired  => this can be expansive depending on the number of rules in the ruleset. It is recommended to not use this or use in exclusion with rules fired.

The list of not executed ruleflow tasks => this can be expansive if there is a large number of ruleflows and tasks. It is recommended to not use this or use in exclusion with list of executed ruleflow tasks. Information about rules will include the same.

The list of rules fired => this can be expansive depending on the number of rules in the ruleset. It is recommended to not use this or use in exclusion with rules not fired.

The list of executed ruleflow tasks => this can be expansive if there is a large number of ruleflows and tasks. It is recommended to not use this or use in exclusion with list of not executed ruleflow tasks. Information about rules will include the same.

The duration (in ms) of execution => not expansive (just a number. The information is not an indication of the production performance)   

The list of rules => very expensive, the entire list of rules in the ruleset =  rules fired + rules not fired, should not be used when running a lot of scenarios.

The list of ruleflow tasks => very expensive, the entire list of tasks in the ruleset =  task executed + tasks not executed, should not be used when running a lot of scenarios.