IBM Security QRadar SOAR

 View Only
Expand all | Collapse all

Help with Scheduler Function

  • 1.  Help with Scheduler Function

    Posted Tue October 27, 2020 10:06 AM
    I thought I would start by giving my use case and see if anyone might be able to help with some struggles I am seeing with the Scheduler function.

    I am using it on an App Host and I need to:
    Run 1 specific artifact rule on all incidents that are active and contain a unique artifact type to my environment every 5 minutes using a Call REST API workflow (from fn_utilities) I've built.

    I am trying to just test this on an artifact inside an incident using the default rule "Schedule a Rule to Run - Artifact" to see how it'll work and it enters the workflow status but never ends up doing anything and never populates in the Scheduler data table. With that said, I would like to automate this so all my incidents automatically re-run this workflow every 5 minutes until they are closed.

    rule

    Does anyone have an idea how to accomplish this or is this a problem with the function hanging? Thanks!

    ------------------------------
    Mr Coco
    ------------------------------


  • 2.  RE: Help with Scheduler Function

    Posted Tue October 27, 2020 12:13 PM
    Have you your sql persistant database configured  and referenced correctly in your app.config ...

    [fn_scheduler]
    timezone = utc
    thread_max = 20
    datastore_dir=<sql db>

    Are there any messages in the logs  you can share that may indicate an issue ?

    ------------------------------
    John Quirke
    ------------------------------



  • 3.  RE: Help with Scheduler Function

    Posted Tue October 27, 2020 01:56 PM

    Mine shows the following:

    # timezone. ex: utc, America/New_York
    timezone = cdt
    # number of rules which can run in parallel
    thread_max = 20
    # directory for the sqlite persistent db
    #datastore_dir =
    # db url if using a postgreSQL DB. Use this with AppHost
    #db_url=postgresql+pypostgresql://username:password@host:port/database

    Does this all have to be configured on my AppHost to work? Seems that db_url is not set but we are using postgres. I will need to review the logs but what I noticed was the Workflow & Action Status running in perpetuity with no change in the state with no jobs loading in the scheduler tables after.


    ------------------------------
    Mr Coco
    ------------------------------



  • 4.  RE: Help with Scheduler Function

    Posted Wed October 28, 2020 05:24 AM

    Hi there,

    You are required to set this line:

    db_url=postgresql+pypostgresql://username:password@host:port/database

    to real values... as an example, you can spin up a container:

    docker|podman run -d --name postgresql_database_test -e POSTGRESQL_USER=user -e POSTGRESQL_PASSWORD=pass -e POSTGRESQL_DATABASE=db -p 5433:5432 rhscl/postgresql-96-rhel7

    The corresponding config line would be:
    db_url=postgresql+pypostgresql://user:pass@<IP|URL>:5433/db

    Hope that helps!



    ------------------------------
    Sean OGorman
    ------------------------------



  • 5.  RE: Help with Scheduler Function

    Posted Mon November 02, 2020 12:33 PM
    Thanks Sean! I appreciate the help.

    We setup Postgres (non-Docker) and configured but it still seems to just be hanging never populating the Scheduler data tables. I don't see any errors or know where to really look for that on AppHost.

    Note, we haven't been using Docker or know what all we should do if we wanted to go that route. Do you have any suggestions for us if we were wanting to install it with a Postgres image that we could use to help us leverage Scheduler? Sorry, this is a little out of our comfort zone...

    ------------------------------
    Mr Coco
    ------------------------------



  • 6.  RE: Help with Scheduler Function

    Posted Wed November 04, 2020 07:07 AM
    Edited by John Quirke Wed November 04, 2020 09:46 AM
    Mr Coco

    Could you send us your logs from app host 

    Administrator Settings - Apps - fn_scheduler - Details - Download logs  (button on bottom right)

    Also  please check you action status from your incident window, under the Actions drop down menu for any errors and share.

    As Sean indicate the db_url has to be un-commented and configured correctly.

    Also please share your updated app.config

    Regards
    John

    Any miss configuration should throw an error in the logs and hopefully will help troubleshooting your issue

    ------------------------------
    John Quirke
    ------------------------------



  • 7.  RE: Help with Scheduler Function

    Posted Wed November 04, 2020 10:49 AM
    Also another question ...are you on an 'Integration server' or 'Apphost' ?
    if you are on an integration server you will need install postgres dependencies if you have not done already
     Use upgrade command if you have alreay installed integration

     pip install --upgrade fn_scheduler_1.0.3.tar.gz[apphost]

    ------------------------------
    John Quirke
    ------------------------------



  • 8.  RE: Help with Scheduler Function

    Posted Thu November 05, 2020 02:26 PM
    Edited by Mr Coco Thu November 05, 2020 02:29 PM
      |   view attached
    We have scheduler running on the AppHost and Postgres (not-containered). I can connect via PSQL remotely just fine to Postgres so it's not a connectivity issue from what I see.

    Unfortunately, the Download Logs gives me an error "Unable to complete the action due to an unexpected error. Access the logs using the following command on the App Host: sudo appHostPackageLogs". I ran the log collection from the CLI and attached but it really doesn't tell me much in the synchronizer or operator log.

    The app and workflow status will just stay running / pending and never do anything.


    Here is the app.config screenshots from the App Host config.
    I appreciate all the help!!!!


    ------------------------------
    Mr Coco
    ------------------------------

    Attachment(s)

    txt
    apphost_logs.txt   557 KB 1 version


  • 9.  RE: Help with Scheduler Function

    Posted Thu November 05, 2020 04:30 PM
    Let me try and set this up in our app host environment (currently on 1.0.1 without PostgreSQL)

    I see our setup doesn't have the postgres setup.
    Which version of fn_scheduler are you on (latest is 1.03)
    I have a remote PostgreSQL database which I can connect to.




    ------------------------------
    John Quirke
    ------------------------------



  • 10.  RE: Help with Scheduler Function

    Posted Thu November 05, 2020 04:45 PM
    We are on 1.0.3.

    I really appreciate it John and Sean! My colleague just updated the app host to the latest version and I can Download Logs now. It seems that I do see some errors of worth now. :)

    See below
    postgresql.exceptions.ClientCannotConnectError: could not establish connection to server
    CODE: 08001
    LOCATION: CLIENT
    CONNECTION: [failed]
    failures[0]:
    NOSSL socket('10.198.112.123', 5432)
    postgresql.exceptions.AuthenticationMethodError: unsupported authentication request '<unknown>'(10)
    CODE: --AUT
    LOCATION: CLIENT
    HINT: 'postgresql.protocol' only supports: MD5, crypt, plaintext, and trust.
    CONNECTOR: [IP4] pq://postgres:***@10.198.112.123:5432/scheduler
    category: None
    DRIVER: postgresql.driver.pq3.Driver

    During handling of the above exception, another exception occurred:

    Note, I have the Postgres conf temporarily setup to allow connections from 0.0.0.0/0 (w/ md5) and Postgres reflects it is using md5 when I run show password_encryption;

    I'm not sure why it would have trouble with negotiations as I can connect fine via psql on other hosts to it. I'm stumped. :(

    ------------------------------
    Mr Coco
    ------------------------------



  • 11.  RE: Help with Scheduler Function

    Posted Thu November 05, 2020 04:51 PM
    This would be the expected connection string to your remote postgres database.

    Ensure that your app host container can reach it ...

    db_url=postgresql+pypostgresql://<user>:<pwd>@<host>:<port>/<db>

    ------------------------------
    John Quirke
    ------------------------------



  • 12.  RE: Help with Scheduler Function

    Posted Thu November 05, 2020 05:30 PM
    Sorry I'm not sure how to do that from the container. I'm a newb there.

    Would this be any easier if this wasn't Postgres running on the host OS but working from container (scheduler) to container (new postgres setup)? I'm not sure if there is a simple pre-configured Postgres container I could load to make it simpler?

    Just thinking how to simplify....

    ------------------------------
    Mr Coco
    ------------------------------



  • 13.  RE: Help with Scheduler Function

    Posted Fri November 06, 2020 08:21 AM
    What version of postgres are you using ?
    The error that is displayed seems to point to an issue with postgres ....once I know what version we can determine if we support that version.
    I am running on postgres 9.6 without issue.

    This seems to be the issue with postgres.....

     ...postgresql.exceptions.AuthenticationMethodError: unsupported authentication request '<unknown>'(10)

    Looks like postgres 10 ... can you confirm ?


    ------------------------------
    John Quirke
    ------------------------------



  • 14.  RE: Help with Scheduler Function

    Posted Fri November 06, 2020 08:29 AM
    I put on version 13

    ------------------------------
    Mr Coco
    ------------------------------



  • 15.  RE: Help with Scheduler Function

    Posted Fri November 06, 2020 10:20 AM
    John you were right. It was a Postgres version issue. Good thinking!

    I have another issue to work out now but I'd be surprised if I don't have it working by EOD:

    Notable Error 1 on Action Status:
    File 'nbtinsert.c', line 433, in _bt_check_unique from SERVER DETAIL: Key (id)=(ruletest_check-2370) already exists.

    Notable Error 2 on Actions Status:
    ConflictingIdError: 'Job identifier (ruletest_check-2370) conflicts with an existing job'

    I will report back progress I've made but we appear to be past one hurdle....

    ------------------------------
    Mr Coco
    ------------------------------



  • 16.  RE: Help with Scheduler Function

    Posted Fri November 06, 2020 10:34 AM
    Great news glad to hear it ...did you  get it working on postgres 13 ?
    or did you revert to an older version ?

    ------------------------------
    John Quirke
    ------------------------------



  • 17.  RE: Help with Scheduler Function

    Posted Fri November 06, 2020 10:38 AM
    Reverted to 9.6.

    ------------------------------
    Mr Coco
    ------------------------------



  • 18.  RE: Help with Scheduler Function

    Posted Fri November 06, 2020 02:52 PM
    Woo hoo! Finally figured it out! Thank you John and Sean for taking the time out! You guys are wonderful for the Resilient community and I am grateful.

    Bonus question?
    If I don't want the scheduler posting Notes or want to be cleaning them up when they are left, do you have any suggestions for doing it? With us looking to poll APIs so frequent, that would leave a lot of notes.

    ------------------------------
    Mr Coco
    ------------------------------



  • 19.  RE: Help with Scheduler Function

    Posted Mon November 09, 2020 04:35 AM
    I didnt think schedule created any notes ...can you post an example.
    Any note creation can be prevented by commenting out that line in the post-processing script for a the associate workflow.

    ------------------------------
    John Quirke
    ------------------------------



  • 20.  RE: Help with Scheduler Function

    Posted Mon November 09, 2020 09:41 AM
    I don't have anything in my post-processing and according to the readme (from the Considerations / Rules section link), this appears to be normal. See example notes it leaves below.

    Scheduled job 'Get Test REST API Call' run on artifacts: 410

    If this isn't something we can remove or delete after, this would be a deal breaker for my incident responders. I see that rules can look at users that add Notes but I have not clue how to script logic to have it delete those notes left behind.


    ------------------------------
    Mr Coco
    ------------------------------



  • 21.  RE: Help with Scheduler Function

    Posted Mon November 09, 2020 11:25 AM
    So fn_scheduler does create notes ...typically this functionality is enable in the post processing script.
    As the post processing script is empty that was why I thought that they weren't created.
    However in this case the creation of notes is enabled in the individual function.This functionality is not easy to remove as it stands.
    However  we do realize that this will be a real concern for customers going forward and we will place this issue under serious consideration.
    We have raised a ticket for our integrations team to address the issue.

    ------------------------------
    John Quirke
    ------------------------------



  • 22.  RE: Help with Scheduler Function

    Posted Mon November 09, 2020 04:17 PM
    Thanks John for championing for us and helping me to figure out the Postgres problem!

    ------------------------------
    Mr Coco
    ------------------------------