High Performance Computing Group

 View Only

 `bhosts` and `lshosts` report different host memory sizes

Frank Thommen's profile image
Frank Thommen posted Sat April 19, 2025 02:20 PM

Hi,

the new memory related fields for the bhosts customized output report different total/maximal memory size for hosts than what lshosts reports. Example:

$ lshosts -o "HOST_NAME:20 maxmem:-" myhost
HOST_NAME                maxmem
myhost                  514990M
$ bhosts -o "host_name:20 available_mem:- reserved_mem:- total_mem:-" myhost
HOST_NAME              AVAILABLE_MEM    RESERVED_MEM       TOTAL_MEM
myhost                   10.4 Gbytes    396.9 Gbytes    407.4 Gbytes
$ 

I would have expected, that lshosts' maxmem and bhosts' total_mem are the same (the maximum physical memory available on the cluster node). Do I misinterpret these two values? And if not, why are they not reporting exactly the same value?

Btw. it is also confusing - and somehow irritating -, that e.g. the field host_name must be written in uppercase for lshosts -o but is case insensitive when used with bhosts -o. Field name and field name case sensitivity streamlining between the many commands would greatly help all those having to write scripts and functions for their LSF cluster :-)

Cheers, Frank

YI SUN's profile image
YI SUN posted Sun April 20, 2025 07:40 PM

bhosts takes consideration of reserved memory by jobs (e.g. job asks rusage[mem=<mem_value>]). 

Bernd Dammann's profile image
Bernd Dammann posted Mon April 21, 2025 07:27 AM

Hi Frank,

lshost's maxmem and bhost's total_mem are two different values!  maxmem is the installed memory minus the OS/kernel usage, and total_mem the currently available memory on the node.  Both values can be seen with the free command, and match the values of the LSF commands:

$ ssh myhost free | awk ' /Mem:/ { print $2/1024, $NF/1024/1024 }'
773047 737.026

$ lshosts -o "hname:20 maxmem:-" myhost
HOST_NAME                maxmem
myhost                         773046M
~
$ bhosts -o "host_name:20 available_mem:- reserved_mem:- total_mem:-" myhost
HOST_NAME              AVAILABLE_MEM    RESERVED_MEM       TOTAL_MEM
myhost                   649.6 Gbytes     87.4 Gbytes      737 Gbytes

Reserved_mem is the reservations of the jobs, with "rusage[mem=...]", minus the actual usage (this node has one job, that has reserved 96GB, and uses 8.6GB).  Available_mem is the available memory for a potential job, that could be dispatched to this node, and Total_mem is the sum of the two other fields!

Hope that makes sense!   My rule of thumb: lshosts gives you the 'static' picture, while bhosts presents the dynamic values. 

Regards, Bernd

Related Content