Special thanks to Taees Eimouri for proofreading this blog in a heartbeat :) Thank you also to Amy Seo and Augustine Chife for their help getting the process together!
Hi guys !
I find that exporting and importing content through the API is great, once you've understood all the pieces you need, it is faster and much easier to automate.
So I am here today to guide you though the process of managing content using the API!
I split the blog into 4 parts:
The procedure described in this blog should work on version 13 and above of the API.
General information
Once you know that a capability exists in the API, it is really easy to work with it, everything is made to provide a maximum of information to the user.
For each endpoint, there is a description of the fields that can be returned, a description of the filters that are expected, and some examples. You can work on your query directly from the web browser or copy the curl command that is automatically generated while you are typing.
All the queries can be executed with a curl command in the following format:
curl -S -X METHOD -u username -H 'Version: version' -H 'Accept: application/zip’'https://myqradar.test/api/config/extension_management/xxx'
Note: Use -k to bypass certificate check
Exporting content
Export of content happens in the endpoint /config/extension_management/
The endpoint takes a filter in the following format:
{ "export_contents": [ { "content_item_ids": [”id1"], "content_type": ”typeA","filters": [ { "filter_name":"filterA"} ] ,"related_content": [ { "content_type":”typeB"}] } } ]
- content_item_ids: This filter is not mandatory. If it is not specified, all the content of a certain type will be exported. Multiple IDs can be added for a type, each separated by a comma.
- content_type: This one is pretty straight forward :) You can specify many content types such as "CUSTOM_FUNCTIONS", "CUSTOM_RULES", "REPORTS", etc.
- filters: This option allows to export content depending on its status.
- related_content: The API will trigger an export of all the dependencies necessary to make the content work. If you want to export extra related content such as groups as an example, you can add this argument to your query.
The full list of arguments that can be passed to each of these can be found in the API in the query builder.
Below is a list of examples that will help you build you own queries
| Specific Custom Function |
{ "export_contents": [{ "content_item_ids": ["2"], "content_type": "CUSTOM_FUNCTIONS"}] } |
| All the Rules |
{ "export_contents": [{ "content_type": "CUSTOM_RULES" }] } |
| Specific Rules |
{ "export_contents": [{ "content_item_ids": ["100066", "100067"],"content_type": "CUSTOM_RULES" }] } |
| Specific Report |
{ "export_contents": [{ "content_item_ids": ["admin#$#34ef8667-27b8-45d2-aea5-2710a33899e8"], "content_type": "REPORTS" }] } |
| All the Reports and groups |
{ "export_contents": [ {"content_type": "REPORTS","related_content": [ { "content_type":"FGROUP_LINKS"}] }] } |
| Specific Report and groups |
{ "export_contents": [{ "content_item_ids": ["admin#$#34ef8667-27b8-45d2-aea5-2710a33899e8"], "content_type": "REPORTS","related_content": [{ "content_type":"FGROUP_LINKS"}] }] } |
| Specific Search |
{ "export_contents": [{ "content_item_ids": ["SYSTEM-58"], "content_type": "SAVED_EVENT_FLOW_SEARCHES" }] } |
| CEP Definition |
{ "export_contents": [{ "content_type": "CUSTOM_PROPERTIES", "content_item_ids": ["9a1e8640-ce6e-4387-bbbe-57a7d4933d86"]}] } |
| All the CEP Definitions |
{ "export_contents": [{ "content_type": "CUSTOM_PROPERTIES"}] } |
| All the CEP Definitions + regular expressions |
{ "export_contents": [{ "content_type": "CUSTOM_PROPERTIES", "related_content": [{ "content_type": "REGEX_EXPRESSIONS"}]}] } |
| All the CEP Definitions +regular expressions + JSON expressions |
{ "export_contents": [{ "content_type": "CUSTOM_PROPERTIES", "related_content": [{ "content_type": "REGEX_EXPRESSIONS"},{ "content_type": "JSON_EXPRESSIONS"}]}] } |
| Specific Regular Expression |
{ "export_contents": [{ "content_type": "REGEX_EXPRESSIONS", "content_item_ids": ["9a1e8640-ce6e-4387-bbbe-57a7d4933d86"]}] } |
| Specific Log Source Type |
{ "export_contents": [{ "content_type": "LOG_SOURCE_TYPES", "content_item_ids": ["4002"]} ] } |
| Custom Log Source Type |
{ "export_contents": [{ "content_type": "LOG_SOURCE_TYPES", "filters": [{ "filter_name":"CUSTOM"}]}] } |
| All Custom Log Source Types and their non deleted Log Sources |
{ "export_contents": [ { "content_type": "LOG_SOURCE_TYPES", "filters": [ { "filter_name":"CUSTOM"} ], "related_content": [{ "content_type": "LOG_SOURCES", "filters": [ { "filter_name":"NONDELETED"} ]}] } ] }
|
| Log Source Extension |
{ "export_contents": [{ "content_type": "LOG_SOURCE_EXTENSIONS", "content_item_ids": ["9"]}] } |
End to end workflow for exporting content
The best way I can explain the end-to-end workflow is through an example !
I will assume that you have the Endpoint content extension on your system (because really you should :-P ), but you can adapt to your own content.
Let's make a goal of exporting all the rules related to Ryuk Ransomware.
Step 1: Find the rules ID
- Go to GET /analytics/rules
- Build you query as bellow:
- Fields: id
- Filter: name ilike "%ryuk%"
The curl command looks like :
curl -S -X GET -u admin -H 'Range: items=0-49' -H 'Version: 17.0' -H 'Accept: application/json' 'https://192.0.2.10/api/analytics/rules?fields=id&filter=name%20ilike%20%22%25ryuk%25%22'
You get results in the following format:
[
{
"id": 102439
},
{
"id": 107939
},
{
"id": 107989
},
{
"id": 108039
}
]
To get to the format expected by the /config/extension_management/ endpoint, you need to do a few text editing (other transformation might be needed for fully automated process):
| Find |
Replace with |
| \n |
|
| }, { "id": |
", " |
| }, { |
"], |
The result looks like :
[ { "id": "102439", "107939", "107989", "108039"} ]
Step 2: Trigger the export
- Go to POST /config/extension_management/extension_export_tasks
- Build you query as below (adapt to the result of the previous query):
- configData: { "export_contents": [{ "content_item_ids": ["102439", "107939", "107989", "108039"], "content_type": "CUSTOM_RULES", "related_content": [ { "content_type": "FGROUP_LINKS"} ] }]}
The curl command looks like:
curl -S -X POST -u admin -H 'Content-Type: application/json' -H 'Version: 17.0' -H 'Accept: application/json' --data-binary '{ "export_contents": [{ "content_item_ids": ["102439", "107939", "107989", "108039"], "content_type": "CUSTOM_RULES", "related_content": [ { "content_type": "FGROUP_LINKS"} ] }]}' 'https://192.0.2.10/api/config/extension_management/extension_export_tasks'
After you execute your query get as a response the following output:
{
"message": "Exporting an extension",
"task_id": 1104
}
This will be useful if you want to check the status of your export. Simply enter this task_id in the endpoint /config/extension_management/extensions_task_status/{status_id}
The curl looks like this :
curl -S -X GET -u admin -H 'Version: 17.0' -H 'Accept: application/json' 'https://192.0.2.10/api/config/extension_management/extensions_task_status/1104'
And the response looks like that:
{
"created": 1663702455938,
"started": 1663702455938,
"completed": 1663702469171,
"cancelled_by": "",
"message": null,
"child_tasks": null,
"task_components": null,
"created_by": "admin",
"name": null,
"modified": 1663702469171,
"progress": 0,
"result_url": "",
"maximum": 0,
"id": 1104,
"cancel_requested": false,
"status": "COMPLETED"
}
Step 3: Downloading the export
This step doesn't really makes sense to execute in the browser as the content will be unreadable.
So you can try the web interface but I am going to give you the curl command directly!
curl -S -X GET -u admin -H 'Version: 17.0' -H 'Accept: application/zip' 'https://myqradar.test/api/config/extension_management/extension_export_tasks/1104/extension_export' > myexport.zip
/!\ Note: To make a valid export, the result is redirected to a zip archive
And that's it ! You content extension is ready.
Installing content
The process for installing content is just like in the UI, the extension needs to be uploaded, and then it can be installed.
Step 1: Upload the content extension
You can do it by using the following command:
curl -S -X POST -u admin -H 'Content-Type: multipart/form-data' -H 'Version: 17.0' -H 'Accept: application/json' -F 'file=@/root/myexport.zip' 'https://myqradar.test/api/config/extension_management/extensions'
Step 2: Retrieve the ID of your extension
That is the last step necessary to actually install the package. For readability, this is how it looks in the API UI:
The curl command is:
curl -S -X GET -u admin -H 'Version: 17.0' -H 'Accept: application/json' 'https://myqradar.test/api/config/extension_management/extensions?fields=id&filter=file_location%3D%22%2Fstore%2Fcmt%2Fexports%2Fmyexport.zip%22'
Grab the ID you get as a response.
Step 3: Installation !
Use this ID in the following curl command with the argument INSTALL and let the magic happen!
curl -S -X POST -u admin -H 'Version: 17.0' -H 'Accept: application/json' 'https://myqradar.test/api/config/extension_management/extensions/v2/83?action_type=INSTALL'
You can check the status of your installation by checking the same endpoint as the one we used during the export:
curl -S -X GET -u admin -H 'Version: 17.0' -H 'Accept: application/json' 'https://myqradar.test/api/config/extension_management/extensions_task_status/1106'
I hope this will help you and make things easier for you !
If you are interested in reading more about QRadar Security Content, you can find the complete list of blog entries here.