Hello everyone!

We are with the following scenario in our integration servers:

[check the attachment]

And I have the following questions:

1 - Currently, our Terracotta cluster manager is using about 11GB. Is this value normal?

2 - We only have 2 Integration Services running and being managed by Terracota. Is there a way to limit or decrease this memory usage?

3 - We set the “wrapper.java.initmemory” and “wrapper.java.maxmemory” properties to, respectively, 8192 and 12288 to our Integration Service. Why does the IS service is using 20GB of memory? Shouldn’t be something below 12GB?

4 - We set the “wrapper.java.initmemory” and “wrapper.java.maxmemory” properties, respectively, 4096 and 8192 to our Universal Messaging. Why does the UM service is using 14GB of memory? Shouldn’t be something below 8GB?

5 - Our actual Universal Messaging Heap Memory usage is about 400~1500MB and we have about 6000MB of free heap memory available. If we decrease the heap memory max size to 4096MB will we have a problem?

I am asking this because we had to restart our both integration servers and UM to solve a high memory usage, about 92%, that we were having. We are doing a study to optimize this usage.

Thank you!

#webMethods
#webMethods-General
#Integration-Server-and-ESB

Hello Renan,

The memory you are referring in your screenshot is virtual memory not physical memory.

Also by looking at the screenshot, we can say that usage (CPU & Mem)is normal, So not sure what is the actual issue you are facing.

Are you facing any performance issue in your IS, TC or UM ? why you need to restart your server.

Thanks,
Yogesh

#webMethods
#Integration-Server-and-ESB
#webMethods-General

Hello Mr. Yogesh, thank you for your reply!

Hmmm, I did a little search and you are right, the “VIRT” column value isn’t the right one. The right one is the “RES” columns who shows the correct memory usage of the process. Which is pretty normal for me too.

“Are you facing any performance issue in your IS, TC or UM ?”
Actually, no. We are not facing any issue with our programs. We are just concerned about the memory usage.

We are using a software to monitor our integration servers and this program is constantly emiting warnings about the free memory percentage, as yesterday it decreased to 1%. So, we had to restart UM and the IS services trying to release the used memory. It worked, but we are concerned if this high memory consumption will return.

#Integration-Server-and-ESB
#webMethods-General
#webMethods

Thanks for sharing these details. One thing worth checking is the difference between Java heap memory and the total process memory shown by the operating system. Native memory, direct buffers, thread stacks, caching, and JVM overhead can make the real memory usage much higher than the configured heap size. Since your actual UM heap usage appears far below the maximum, gradually reducing the heap and monitoring garbage collection, response times, and stability in a test environment seems like a reasonable approach before applying changes in production.

------------------------------
Dexter Watts Watts
Bulk Devices
Pearland TX
9036774333
------------------------------

Hello! Not sure what's going on but I'm currently unable to see the screenshots. However, I've worked with webMethods for many years and a 12 GB heap for the Integration Server is unusual in my personal experience. I have come across customers that had large heap settings like this before, but they typically ended up there by increasing the max heap size when they ran into a memory issue instead of actually addressing the root cause. Increasing the max heap, however, just typically delays the problem which often manifested itself as a garbage collection issue. Once we identified the root cause, we were able to cut the heap size considerably and eliminate the GC pauses.

Does your IS start by consuming 8GB and then gradually increase to 12GB over time? Or does it use the 12GB right out the gate? A gradual increase over time could indicate a memory leak.

If you don't know what is consuming your heap, then I highly recommend doing a heap dump and then using MAT (https://eclipse.dev/mat/) or something similar to analyze the contents of the dump. The challenge is that dumping and analyzing a 12 GB heap is tough given the large size. If you have a test environment that mimics Production close enough, then I'd reduce the heap size there and perform the exercise against that environment. If you don't and if your heap consumption increases gradually over time, then one option is to reduce your minimum heap size and then take incremental heap dumps as heap consumption increases over time. You can then analyze multiple heap dumps and the difference between one dump and then next should pinpoint the leak (if there is one).

I used to do this type of analysis frequently, especially back when I worked for Software AG. If you have specific questions, feel freel to reach out.

------------------------------
Percio
https://nibl.tech
------------------------------