IBM Security QRadar SOAR

 View Only
Expand all | Collapse all

app-fn_scheduler database configuration

  • 1.  app-fn_scheduler database configuration

    Posted Mon August 23, 2021 05:55 PM
    Edited by Tyler Bennett Mon August 23, 2021 06:10 PM
    Can I get some guidance on deploying the app-fn_scheduler? I've read the documentation but it's not really straightforward on the setup of the sqllite/psql database connection.

    We are using appHosts.

    I tried the following option in the configuration and touched the file on disk(on the dev apphost)
    datastore_dir = /tmp/app-fn_scheduler.sqllite

    fn-scheduler: selftest: failure selftest output: {'state': 'failure', 'reason': '(sqlite3.OperationalError) unable to open database file\n(Background on this error at: http://sqlalche.me/e/14/e3q8)'} Elapsed time: 0.025000 seconds

    touch /tmp/app-fn_scheduler.sqllite
    chmod 777 /tmp/app-fn_scheduler.sqllite

    -----------------------------
    Tyler Bennett
    ------------------------------


  • 2.  RE: app-fn_scheduler database configuration

    Posted Tue August 24, 2021 10:33 AM
    Hi Tyler,

    I do agree that it's a bit of confusing. The app.config setting `datastore_dir` refers to a directory where the app will create the `scheduler.sqlite` file. So, make sure your entry is a `folder`:

    mkdir /tmp/app-fn_scheduler​


    ------------------------------
    Mark Scherfling
    ------------------------------



  • 3.  RE: app-fn_scheduler database configuration

    Posted Tue August 24, 2021 12:04 PM
    Okay, that makes sense.

    I configured the following folder for testing in dev.
    mkdir /tmp/app-fn_scheduler-sqllite

    Then tries the following two configurations and received the same error messages. 
    datastore_dir = /tmp/app-fn_scheduler-sqllite
    datastore_dir = /tmp/app-fn_scheduler-sqllite/

    Then I made the folder world writeable and get the same error messages.
    chmod 777 /tmp/app-fn_scheduler-sqllite

    datastore_dir = /tmp/app-fn_scheduler-sqllite
    datastore_dir = /tmp/app-fn_scheduler-sqllite/


    Error for all four attempts is as follows

    fn-scheduler: selftest: failure selftest output: {'state''failure''reason''(sqlite3.OperationalError) unable to open database file\n(Background on this error at: http://sqlalche.me/e/14/e3q8)'} Elapsed time: 0.025000 seconds


    ------------------------------
    Tyler Bennett
    ------------------------------



  • 4.  RE: app-fn_scheduler database configuration

    Posted Tue August 24, 2021 12:50 PM

    I followed similar steps:

    $ mkdir /tmp/scheduler.sqlite

    My app.config file:

    [fn_scheduler]
    timezone=America/New_York
    thread_max=20
    datastore_dir=/tmp/scheduler.sqlite
    Then, the sqlite file is created:
    ls /tmp/scheduler.sqlite/scheduler.sqlite


    ------------------------------
    Mark Scherfling
    ------------------------------



  • 5.  RE: app-fn_scheduler database configuration

    Posted Tue August 24, 2021 01:15 PM
    I followed those steps exactly.

    Logged onto the dev apphost and ran
    $ mkdir /tmp/scheduler.sqlite


    I guess I raise a ticket with IBM then?



    ------------------------------
    Tyler Bennett
    ------------------------------



  • 6.  RE: app-fn_scheduler database configuration

    Posted Tue August 24, 2021 04:51 PM
    Hi Tyler,

    I believe the issue is permissions within the container. A better location within the container would be datastore_dir=/var/rescircuits. This location is intended for app file access.

    Keep in mind that containers are ephemeral. They can be restarted and the state of the container is reset, losing your schedules. This is where we now support external a postgre databases. Rather than using datastore_dir=, use db_url=postgresql+pypostgresql://username:password@host:port/database referencing an externally accessible database.

    Regards,
    Mark

    ------------------------------
    Mark Scherfling
    ------------------------------



  • 7.  RE: app-fn_scheduler database configuration

    Posted Tue August 24, 2021 05:17 PM
    So to be clear, it is not recommended to use datastore_dir when configuring this app on an apphost? Do you have any details for how to create the postgres system? I know next to nothing about postgres or container networking and the docs are extremely sparse.

    My assumption is we can create a postgres database on the apphost, then configure the containers to connect to the apphost database. Can you layout the steps to create this database, and allow container access to it?

    ------------------------------
    Tyler Bennett
    ------------------------------



  • 8.  RE: app-fn_scheduler database configuration

    Posted Thu August 26, 2021 09:05 AM
    Edited by John Quirke Thu August 26, 2021 10:27 AM
    Hi Tyler

    On apphost the /tmp folder reference is the location on the container where you would want to configure your sqllite database.
    Typically if you were using an integration server environment (non container) we would be referencing a folder on the integration server.
    If used on apphost and the container were to restart the database would be overwritten.
    When we are using apphost we therefore use  postgres connection to an external database.
    If you would like to test with postgres , I can message you privately and share some quick docker environment setup commands to test.
    I have it running on a linux environment.

    Regards
    John

    ------------------------------
    John Quirke
    ------------------------------



  • 9.  RE: app-fn_scheduler database configuration

    Posted Tue October 24, 2023 05:04 PM

    would you mind sharing with me the docker environment setup commands for postgres?



    ------------------------------
    Ryan Terry
    ------------------------------



  • 10.  RE: app-fn_scheduler database configuration

    Posted Wed October 25, 2023 01:33 AM

    Hey Ryan,

    Check out the video and the documentation I created in the link, documentation in the link in the description of the video, where I explain how to get this all setup! Hope that helps!

    https://www.youtube.com/watch?v=k04zz4Jps8w&t

    Thanks!



    ------------------------------
    Nick Mumaw, GPEN, GPYC
    Cyber Security Specialist - SOAR
    IBM - Security
    ------------------------------