Did anyone have a case when queries and rules that are using some larger ref set (100k) is matching only part of the results and missing other part?

For example, search with NOT in ref set is also returning part of results, but when searching without NOT it is also returning some results.

We have an open case with IBM support for 3 months, but they have not been able to solve the problem yet.

This is currently most obviously manifesting on ref set used by QVTI app.

Our version is 7.4.3. FP4, IF4.

Hi Igor, you are not allone with such long cases. I experience the same with much more severe problems. I had one open that sounds like beeing the root cause for your problem as well.IJ28797 describes the symptoms related to it. Also you might not use QVM and also have a fixed Version like i had you might still run into this problem. The Support Ingeneer LIAMHFOX told me so and advised me to do the following steps that solved my issues. You should ask Support if you should also do them because it seems not to be an official guide and you need a script that i can not attach here.

1) Clearing expired

Take the script I've attached to the case (RefDataCleanV2.sh) and upload it to your console.

Make the script executable with chmod +x RefDataCleanV2.sh

Run the script with ./RefDataCleanV2.sh &

This will clear expired elements and should improve performance a little. This can take a while depending on how many blocked expired events there are.

2) Adding memory override for performance:

1. Open the file /opt/qradar/conf/spillovercache.properties using the text editor of your choice.
2. Add the following line at the bottom: RefData_48.spillover.threshold=500000
3. Restart Tomcat at your next convenience, or let it crash again.

Not sure if it was necessary but i also did it after the above procedure:

How to clear the Tomcat cache (ibm.com)

Thanks Martin for this useful information. I will ask them for a script and try the

procedure. It looks like very similar problem.