IBM Spectrum Computing Group

[LSF Explorer]Clean old data from lsf_current_status index

  • 1.  [LSF Explorer]Clean old data from lsf_current_status index

    Posted Tue January 19, 2021 03:25 AM
    The lsf_current_status index stores the latest status of each LSF job. It is used by bhist command to retrieve job events of specified status. Data in this index will not be cleaned automatically, so that the index will be heavier and heavier as time goes on.
    Old data in lsf_current_status index can be cleaned manually, when under either of the following conditions

    1. Old data is not needed by bhist command;
    2. "-C", "-e", "-d" options are not used by bhist command.
    Run below command to clean data earlier before "2019-01-01".
    Replace {ES_HOST} and {ES_PORT} with real Elasticsearch host and port.

    curl -XPOST "http://{ES_HOST}:{ES_PORT}/lsf_current_status/_delete_by_query" -H 'Content-Type: application/json' -d'

    {

      "query": {

        "bool": {

          "must": [

            {

              "range": {

                "event_time": {

                  "lte": "2019-01-01"

                }

              }

            },

            {

              "terms": {

                "job_status": [

                  "DONE",

                  "DONE+PDONE",

                  "DONE+PERR",

                  "EXIT"

                ]

              }

            }

          ]

        }

      }

    }'

    Refer to https://www.elastic.co/guide/en/elasticsearch/reference/7.10/docs-delete-by-query.html  for more details about 'delete by query API options'.



    ------------------------------
    Edward Deng
    ------------------------------