High Performance Computing Group

Hi All, We are trying to configure LSF to allow hierarchical resource allocation in our pool. For example we would like to be able to configure something like:

Such that LSF jobs would be submitted to, for example:
/snackycake/tasting/amami
/thermonuclear/testing


The theory being that the top-level allows executives to make high-level allocation decisions and as you dive into the tree allocation decisions can be made on a more fine-grained basis. But crucially, unused allocation rolls up the tree and is available at each intermediate node before being available to leaves. For example if /snackycake/tasting/salt doesn't use its allocation, that is available to sweet, sour, bitter, and amami before it's made available to /snackycake/randd. If it's not used at that level, it is then available to the /thermonuclear hierarchy in the proportions defined therein.

I previously used a proprietary batch system at a large company that used that kind of system to manage millions of cores, but have so far not found an equivalent capability in LSF. Job groups seem to get me part-way there but there are several problems:

* I don't see a way to disallow implicit creation of job groups
* Job groups appear to only allow specific job-count limits, not percentage-based
* It's not clear that other branches of the tree can utilize unused job slots


Any suggestions for what to look for to achieve this?

Thanks!
Sam

#SpectrumComputingGroup

You may review LSF Fairshare and Guaranteed SLA scheduling policies.
https://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_admin/chap_fairshare_lsf_admin.html
https://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_admin/chap_guar_res_pool.html

Regards,

Yi Sun

------------------------------
YI SUN
------------------------------

Original Message:
Sent: Wed June 10, 2020 08:55 PM
From: Sam Huffman
Subject: Configuring LSF for hierarchical resource allocation

Hi All, We are trying to configure LSF to allow hierarchical resource allocation in our pool. For example we would like to be able to configure something like:

Such that LSF jobs would be submitted to, for example:
/snackycake/tasting/amami
/thermonuclear/testing


The theory being that the top-level allows executives to make high-level allocation decisions and as you dive into the tree allocation decisions can be made on a more fine-grained basis. But crucially, unused allocation rolls up the tree and is available at each intermediate node before being available to leaves. For example if /snackycake/tasting/salt doesn't use its allocation, that is available to sweet, sour, bitter, and amami before it's made available to /snackycake/randd. If it's not used at that level, it is then available to the /thermonuclear hierarchy in the proportions defined therein.

I previously used a proprietary batch system at a large company that used that kind of system to manage millions of cores, but have so far not found an equivalent capability in LSF. Job groups seem to get me part-way there but there are several problems:

* I don't see a way to disallow implicit creation of job groups
* Job groups appear to only allow specific job-count limits, not percentage-based
* It's not clear that other branches of the tree can utilize unused job slots


Any suggestions for what to look for to achieve this?

Thanks!
Sam

#SpectrumComputingGroup