Denver QRadar User Group

 View Only

Ansible Module for Maintaining reference data.

By Chris Schulz posted Thu October 25, 2018 09:00 PM

  

All I have written a ansible module for updating reference data and will put the code on github with a link bellow.

This blog post is quick write up on how to use the module. Hopefully some of you can play with it and give feedback.

Id like to eventually introduce it to a wider audience but this is probably an 'alpha' release. Im a system engineer not a programmer but have used existing modules as templates so hopefully its not too ugly.

 

I'm organizing a reference data specific user group meeting to talk about use cases, writing rules to query reference data and more.

Id like to have this module in a steady state for that, so thanks in advance for any help.

 

Please note the module has been tested with python 2.7 and QradarCE 7.3.1 rest api version 9.1.

 

Im not going to go into ansible because there is a ton of info out there Ill just include some links.

However since this module is not part of the standard library I need to give you some background to use it.

 

To use the module copy the example playbooks and module down from github.

Move the qradar_data.py module in a subdirectory called library and ansible should find it there by default.

 

Example directory layout:

/

─hosts

── test_ref_data_map.yml

── test_ref_data_set.yml

── test_ref_data_table.yml

─── /library

─── qradar_data.py

 

The 'hosts' file is required by ansible-playbook but since we are talking to QRadar's REST API there's not much in it and you shouldn't have to change it unless you want to put other options in there.

 

The module is written to use either a QRadar user name and password or a token. To save having to repeat credentials or hard code in a  playbook the module will also look check environmental variables. This was a bit of a trade off because I can't use ansible modules library to validate credentials but is modeled after how AWS built their module so it should work with Tower or AWX.

 

You can see credentials being set as environmental variables in the example playbooks.

 

  environment:

      console_ip: "qradar.peaklab.local"

      console_user: "admin"

      console_password: "somepassword"

 

Or

 environment:

      console_ip: "qradar.peaklab.local"

      token: "282d5950-ca6a-4cba-9015-19bccd9b784e"

 

Again you would probably want to set the environment variables outside the playbook (like in tower) but the examples show several ways.

 

 

The module allows you to maintain 3 types of reference collection types (SET, MAP, TABLE) and uses yaml to describe complex data for each type. Reference collections also require you to define a data type for different aspects of each collection. It will be easier if I just walk through all three. 

 

SET: takes an array described in yaml by preceding values with a dash.

This example will add 2 IP addresses to reference set 'testset'

  tasks:

    - name: add reference set

       ref_name: 'testset'

       qradar_data:

          ref_col_type: 'SET'

          ref_data_type: 'IP'

          ref_data:

            - '172.0.0.1'

            - '132.0.0.1'

 

Note: indention is important.

 

tasks:  Tell ansible we are going to start a task

name:  Just a description of the task

qradar_data: Name of the module you copied into the library subdirectory

ref_col_type: What type of reference data do you want to add ('SET', 'MAP', 'TABLE')

ref_data_type:  Define a data type ('IP','ALN','ALNIC','PORT')

IP - IP address
Port - IP port
ALN - Alpha numeric

ALNIC - Alpha numeric ignore case ( by storing everything in lower case)

ref_data:  This is the tag that starts the complex data defined in yaml.

 

MAP: takes an dictionary of keys and values described in yaml.

This example will add 2 IP addresses and their hostname to a reference map named 'testmap'

 

  tasks:

    - name: add refernce map

      qradar_data:

        ref_name: 'testmap'

        ref_col_type: 'MAP'

        ref_data_type: 'IP'

        timetolive: "1 month"

        ref_data:

             server1: '10.0.0.1'

             server2: '192.0.0.1'

 In a map  the ref_data_type describes the 'value' of each key.

 

TABLE: takes an dictionary of keys and values described in yaml.

This example will add a server and info about that server to a reference table 'testtable'

   tasks:

    - name: add refernce map

      qradar_data:

        token: "282d5950-ca6a-4cba-9015-19bccd9b784e"

        ref_name: 'testtable'

        ref_col_type: 'TABLE'

        ref_data_type: 'ALN'

        timetolive: "1 month"

        ref_data:

           Server1:

             IP: "127.0.0.1"

             Owner: "IT services"

             App: "DNS Services"

 

In a table the ref_data_type describes the 'outer key' or server name.

 

Each of the examples show you how to add new reference collections, remove individual entries and remove an entire collection.
If some one has questions please add a comment and Ill answer it.

Playbooks are called in the same way any ansible playbook is called, for example:

# ansible-playbook -i ./hosts ./test_ref_data_map.yml

 

I also created a script you can use while playing with reference data.  It will iterate through all collections, you just need to update the connection variables and collection type. If you change the action you can remove all the collections of that type as well so please only use this with your QRadarCE development. 

# q1RefData.py

 

Lastly I have a couple of to-do's already but wanted to get this out as I think it provides a minimum viable product.  

Im missing a couple of reference collections (map of sets specifically). Please comment and let me know how valuable that would be.

Ill be working on bulk upload and hope to have that soon, but again please comment and if enugh people want it Ill get it out sooner. 

 

 

Promised links:

Ansible Module:

https://github.com/DenQRadarUser/qradar_ansible_module

 

Ansible Information:

https://www.ansible.com/

 

Ansible quick start:

https://docs.ansible.com/ansible/latest/user_guide/quickstart.html

 

 

 

 

 

 

 

 

 

 

 

 

0 comments
18 views

Permalink