Informix

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

It's typically not oninit process memory that's getting swapped, but rather allocated shared memory, and this you can protect from swapping by setting it resident - this is the purpose of the RESIDENT onconfig parameter.

HTH,
 Andreas

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Yes, RESIDENT 1 is set.

I also think buffers were in swap. I deactiaved swap totally and its even faster as before now. Not much, but a bit.

Problem is solved, but strange situation. 

It's typically not oninit process memory that's getting swapped, but rather allocated shared memory, and this you can protect from swapping by setting it resident - this is the purpose of the RESIDENT onconfig parameter.

HTH,
 Andreas

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Marc:

You should be setting RESIDENT to -1. Setting it to '1' only sets the initial resident segment and the first virtual segment to be non-swappable while the message and buffer cache segments will still be swappable. Setting RESIDENT to '-1' will set all shared memory segments to non-swabbable including buffer cache segments and any added virtual segments (message segments, unfortunately, are not marked as non-swappable which can slow local shared memory connections).

Art

Yes, RESIDENT 1 is set.

I also think buffers were in swap. I deactiaved swap totally and its even faster as before now. Not much, but a bit.

Problem is solved, but strange situation. 

It's typically not oninit process memory that's getting swapped, but rather allocated shared memory, and this you can protect from swapping by setting it resident - this is the purpose of the RESIDENT onconfig parameter.

HTH,
 Andreas

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Art - 

I have a client whose boxes end up with "out of swap" condition(s) often. Haven't tracked down (yet) what's in swap, but realized they have RESIDENT = 0 on all engines. Changing that and bouncing the engine:

Marc:

You should be setting RESIDENT to -1. Setting it to '1' only sets the initial resident segment and the first virtual segment to be non-swappable while the message and buffer cache segments will still be swappable. Setting RESIDENT to '-1' will set all shared memory segments to non-swabbable including buffer cache segments and any added virtual segments (message segments, unfortunately, are not marked as non-swappable which can slow local shared memory connections).

Art

Yes, RESIDENT 1 is set.

I also think buffers were in swap. I deactiaved swap totally and its even faster as before now. Not much, but a bit.

Problem is solved, but strange situation. 

It's typically not oninit process memory that's getting swapped, but rather allocated shared memory, and this you can protect from swapping by setting it resident - this is the purpose of the RESIDENT onconfig parameter.

HTH,
 Andreas

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Hi Mark,

as you might guess from https://www.ibm.com/support/pages/apar/IC76872, this can result, on AIX, from Informix resident segments competing with OS KAIO for 'pinnable' memory, i.e. memory that can be fixed in RAM (I think it mainly needs to match certain alignment criteria).

Since Informix has no way to anticipate those pinnable memory resources (there might be other Informix instances, other shared memory users, other activities using pinned memory, etc....), it warns the user that ... bad things have already happened due to running out of this resource (AIX apparently doing a sub-optimal job at making sure it reserves was it needs itself).

If, on the other hand, you're confident your AIX lpar always has sufficient memory reserves, you should be safe using Informix RESIDENT setting.  Also, remember this isn't a boolean (ON or OFF), but you always can say "this many shmem segments resident, please".

Then, I also remember having been told that AIX is good enough at memory management and won't even need setting shared memory segments resident - but the experience you're citing seems to indicate to the contrary?

Lastly, maybe going for large/huge memory pages would avoid this whole discussion (as such memory would be exempt from swapping anyway.)

And very lastly:  you haven't even indicated that the swapping incurred by those systems affects Informix (performance wise) or even occurs to Informix shared memory?  What else could be consuming memory? What exactly is incurring "out of swap"?

BR,

 Andreas

Art - 

I have a client whose boxes end up with "out of swap" condition(s) often. Haven't tracked down (yet) what's in swap, but realized they have RESIDENT = 0 on all engines. Changing that and bouncing the engine:

Marc:

You should be setting RESIDENT to -1. Setting it to '1' only sets the initial resident segment and the first virtual segment to be non-swappable while the message and buffer cache segments will still be swappable. Setting RESIDENT to '-1' will set all shared memory segments to non-swabbable including buffer cache segments and any added virtual segments (message segments, unfortunately, are not marked as non-swappable which can slow local shared memory connections).

Art

Yes, RESIDENT 1 is set.

I also think buffers were in swap. I deactiaved swap totally and its even faster as before now. Not much, but a bit.

Problem is solved, but strange situation. 

It's typically not oninit process memory that's getting swapped, but rather allocated shared memory, and this you can protect from swapping by setting it resident - this is the purpose of the RESIDENT onconfig parameter.

HTH,
 Andreas

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Art - 

I have a client whose boxes end up with "out of swap" condition(s) often. Haven't tracked down (yet) what's in swap, but realized they have RESIDENT = 0 on all engines. Changing that and bouncing the engine:

Marc:

You should be setting RESIDENT to -1. Setting it to '1' only sets the initial resident segment and the first virtual segment to be non-swappable while the message and buffer cache segments will still be swappable. Setting RESIDENT to '-1' will set all shared memory segments to non-swabbable including buffer cache segments and any added virtual segments (message segments, unfortunately, are not marked as non-swappable which can slow local shared memory connections).

Art

Yes, RESIDENT 1 is set.

I also think buffers were in swap. I deactiaved swap totally and its even faster as before now. Not much, but a bit.

Problem is solved, but strange situation. 

It's typically not oninit process memory that's getting swapped, but rather allocated shared memory, and this you can protect from swapping by setting it resident - this is the purpose of the RESIDENT onconfig parameter.

HTH,
 Andreas

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Marc:

You should be setting RESIDENT to -1. Setting it to '1' only sets the initial resident segment and the first virtual segment to be non-swappable while the message and buffer cache segments will still be swappable. Setting RESIDENT to '-1' will set all shared memory segments to non-swabbable including buffer cache segments and any added virtual segments (message segments, unfortunately, are not marked as non-swappable which can slow local shared memory connections).

Art

Yes, RESIDENT 1 is set.

I also think buffers were in swap. I deactiaved swap totally and its even faster as before now. Not much, but a bit.

Problem is solved, but strange situation. 

It's typically not oninit process memory that's getting swapped, but rather allocated shared memory, and this you can protect from swapping by setting it resident - this is the purpose of the RESIDENT onconfig parameter.

HTH,
 Andreas

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!

Even when setting it to RESIDENT -1 on RHEL8 it keeps swapping oninit proccesses

oninit 5084 kB
oninit 5076 kB
oninit 5068 kB
oninit 5056 kB
oninit 5056 kB
oninit 5056 kB
oninit 5048 kB
oninit 5040 kB
oninit 5028 kB
oninit 5008 kB
oninit 5008 kB
oninit 4976 kB
oninit 4976 kB
oninit 4972 kB
oninit 4964 kB
oninit 4964 kB
oninit 4960 kB
oninit 4956 kB
oninit 4948 kB
oninit 4948 kB

Marc:

You should be setting RESIDENT to -1. Setting it to '1' only sets the initial resident segment and the first virtual segment to be non-swappable while the message and buffer cache segments will still be swappable. Setting RESIDENT to '-1' will set all shared memory segments to non-swabbable including buffer cache segments and any added virtual segments (message segments, unfortunately, are not marked as non-swappable which can slow local shared memory connections).

Art

Yes, RESIDENT 1 is set.

I also think buffers were in swap. I deactiaved swap totally and its even faster as before now. Not much, but a bit.

Problem is solved, but strange situation. 

It's typically not oninit process memory that's getting swapped, but rather allocated shared memory, and this you can protect from swapping by setting it resident - this is the purpose of the RESIDENT onconfig parameter.

HTH,
 Andreas

Thanks.

Looks like its the case.. i have many entries there with oninit 4784 KB 

How to prevent oninit to get swapped? 

Hello,

you can find swap usage by process with (as root ofc):

for x in /proc/*/status; do grep -q VmSwap $x && awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $x ; done | sort -k 2 -n -r | less

Informix processes are oninit so all of those should have 0 usage.

HTH

Hrvoje

Hello,

maybe last question of 2023 :-)

I noticed slow downs during last week in our single instance on RHEL8 - i suspect storage (SAN) speed slowdowns, but not totally sure.

Also maybe swap oocured with oninit processes ? How can i check to verify this ?

Thanks and good start for 2024!