High Performance Computing Group

 View Only
  • 1.  How to add LSF tasks to a running job?

    Posted Tue June 21, 2022 09:21 AM
    Hi! I know that there is a -n option in the bsub command to specify the number of tasks in a job.

    I wonder if there is a way to add or remove tasks once the submitted job is started? Thanks.

    ------------------------------
    Ruian Huang
    ------------------------------

    #SpectrumComputingGroup


  • 2.  RE: How to add LSF tasks to a running job?

    Posted Tue June 21, 2022 12:09 PM
    LSF job resize feature may be an option.
    https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=administration-resizable-jobs

    ------------------------------
    YI SUN
    ------------------------------



  • 3.  RE: How to add LSF tasks to a running job?

    Posted Wed June 22, 2022 10:32 AM
    Thanks! This is my first time lean about the resize feature.

    ------------------------------
    Ruian Huang
    ------------------------------



  • 4.  RE: How to add LSF tasks to a running job?

    Posted Wed June 22, 2022 10:09 AM
    Edited by System Fri January 20, 2023 04:31 PM
    Releasing tasks is much easier than acquiring task in my recollection.  There are also limitations.  So, read carefully.  A lot of big HPC shops use this.  For long tailed job runs where the initial run is in parallel using say the entirety of the node (or several nodes), and the end may take 5-10 minutes stitching together results or generating images and doing cleanup that is single threaded.  This way, the next parallel job can startup while the last job is still performing that long tailed operation.  It does improve throughput at the expense of a little jitter at the job start.

    ------------------------------
    Larry Adams
    ------------------------------



  • 5.  RE: How to add LSF tasks to a running job?

    Posted Wed June 22, 2022 10:36 AM
    Hi Larry. Thank you for your reply.

    But I am not sure getting your point. Are you suggesting that I had better not resize the job but specify all the needed resources at the beginning?

    ------------------------------
    Ruian Huang
    ------------------------------



  • 6.  RE: How to add LSF tasks to a running job?

    Posted Wed June 22, 2022 10:50 AM
    No, I'm providing you a use case of how resize is used in the industry.  It's a simple as that.

    ------------------------------
    Larry Adams
    ------------------------------