IBM Security QRadar SOAR

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

Hi Bo Beckel, 

I have watched the tutorial from the security learning academy specifically the "Build your first Qradar SOAR custom integration using the App Host". I have one glaring question regarding the container implementation part. In the guide video provided you shown example on building the docker images locally then pushing it to the ICR repository.

- I tried to register to the IBM cloud and did not managed to "upgrade account" due to some error (Can't search the solution since its coded)

- You also mentioned that for the Apphost server itself you create different server that points to either icr or quay.io. So if say I want to use other repo for storing my images online (e.g. Dockerhub) will I have to modify the setting on my apphost as well or I can use existing default apphost which I assume available for quay.io

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

Hello,

Great clarifying question. Yep, App Host will work with any Docker registry you want, whether thats ICR, Quay.io, Dockerhub, or something else including a private registry hosted locally or publicly. As long as the app host machine has network access to that registry, you can host your images there. You simply have to change the registry that the app host machine looks for with the manageAppHost registry command.

Hope that clears things up! Let me know if there's more I can answer for you.

Hi Bo Beckel, 

I have watched the tutorial from the security learning academy specifically the "Build your first Qradar SOAR custom integration using the App Host". I have one glaring question regarding the container implementation part. In the guide video provided you shown example on building the docker images locally then pushing it to the ICR repository.

- I tried to register to the IBM cloud and did not managed to "upgrade account" due to some error (Can't search the solution since its coded)

- You also mentioned that for the Apphost server itself you create different server that points to either icr or quay.io. So if say I want to use other repo for storing my images online (e.g. Dockerhub) will I have to modify the setting on my apphost as well or I can use existing default apphost which I assume available for quay.io

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

Hi Bo Beckel,

Thanks for the succinct answer given above, I can clearly understand that after packaging the app you can opt to host it on any repo/storage as container.

For the private registry hosted locally, do you meant that I can package my custom application using "resilient-sdk package" and push it as an image such that everyone with access to that docker image will be able to access it?

Also is it possible that, after installing the same module (resilient-sdk, and other packages need for the function to work) other user will be able to access the function provided that they have the access to the same SOAR environment. Can this method be an alternative, where the custom app is not package.

My understanding is that currently I am able to run the custom application in my org SOAR after running the resilient-circuits run, does this translate as I pass the code to other computer and after they have run resilient-circuits run, they also can run the custom function in the same SOAR environment that I have access to.

Hello,

Great clarifying question. Yep, App Host will work with any Docker registry you want, whether thats ICR, Quay.io, Dockerhub, or something else including a private registry hosted locally or publicly. As long as the app host machine has network access to that registry, you can host your images there. You simply have to change the registry that the app host machine looks for with the manageAppHost registry command.

Hope that clears things up! Let me know if there's more I can answer for you.

Hi Bo Beckel, 

I have watched the tutorial from the security learning academy specifically the "Build your first Qradar SOAR custom integration using the App Host". I have one glaring question regarding the container implementation part. In the guide video provided you shown example on building the docker images locally then pushing it to the ICR repository.

- I tried to register to the IBM cloud and did not managed to "upgrade account" due to some error (Can't search the solution since its coded)

- You also mentioned that for the Apphost server itself you create different server that points to either icr or quay.io. So if say I want to use other repo for storing my images online (e.g. Dockerhub) will I have to modify the setting on my apphost as well or I can use existing default apphost which I assume available for quay.io

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

Hi -

Let me try my best to address each of these questions, please let me know if I'm misunderstanding or forgetting certain details of your question.

For private registry hosting, anyone who has network access and credentials to pull from the registry will be able to run that app with App Host on their SOAR instance. You would simply provide to them the .zip file produced in the /dist folder after resilient-sdk package is run, and ensure that their app host is configured to pull images from the private registry. If that registry is as you say "hosted locally", I'm not sure that would be the best approach. If by locally you mean on your personal machine, there would be complex networking to allow others to pull from the registry. The best would be to set up a private registry on a local network machine so that you can share it with other app host machines.

I think that you are getting a bit confused about running the app on app host vs on what we would call an "integration server." An integration server is when you type "resilient-circuits run" from the command line. This is very much recommended only as a development environment. Once you have completed development of the app, you should run "resilient-sdk package" and subsequently build and push with docker to the private registry. The app from then on should be deployed via the App Host mechanism.

More to the point, your question about proper packages on another machine's environment, and sharing the custom functions, is not relevant if they are running in app host. When running in app host, the app will work properly no matter the environment. If you wish to share a package with another "integration server" (i.e. someone else running "resilient-circuits run" locally), then yes they will have to replicate your python environment and your app.config file to properly get things to run.

I hope this addresses most of your concerns and hopefully illuminates the different ways of running apps in SOAR.

Hi Bo Beckel,

Thanks for the succinct answer given above, I can clearly understand that after packaging the app you can opt to host it on any repo/storage as container.

For the private registry hosted locally, do you meant that I can package my custom application using "resilient-sdk package" and push it as an image such that everyone with access to that docker image will be able to access it?

Also is it possible that, after installing the same module (resilient-sdk, and other packages need for the function to work) other user will be able to access the function provided that they have the access to the same SOAR environment. Can this method be an alternative, where the custom app is not package.

My understanding is that currently I am able to run the custom application in my org SOAR after running the resilient-circuits run, does this translate as I pass the code to other computer and after they have run resilient-circuits run, they also can run the custom function in the same SOAR environment that I have access to.

Hello,

Great clarifying question. Yep, App Host will work with any Docker registry you want, whether thats ICR, Quay.io, Dockerhub, or something else including a private registry hosted locally or publicly. As long as the app host machine has network access to that registry, you can host your images there. You simply have to change the registry that the app host machine looks for with the manageAppHost registry command.

Hope that clears things up! Let me know if there's more I can answer for you.

Hi Bo Beckel, 

I have watched the tutorial from the security learning academy specifically the "Build your first Qradar SOAR custom integration using the App Host". I have one glaring question regarding the container implementation part. In the guide video provided you shown example on building the docker images locally then pushing it to the ICR repository.

- I tried to register to the IBM cloud and did not managed to "upgrade account" due to some error (Can't search the solution since its coded)

- You also mentioned that for the Apphost server itself you create different server that points to either icr or quay.io. So if say I want to use other repo for storing my images online (e.g. Dockerhub) will I have to modify the setting on my apphost as well or I can use existing default apphost which I assume available for quay.io

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

Hi Bo Bleckel,

Thanks for your prompt reply and elaborate answer. Since applying some of your pointer, I have encountered roadblock when deploying my app in the SOAR where the error occur say "the tag cant be found".From my understanding, 

Default apphost is configured to connect to quay.io, however user are free to set the apphost to point to their choice of registry (whether it is locally or cloud hosted).

Although there are some limitations in doing so, like apphost can only point towards single registry (correct me if I am wrong here) example shown in the documentation below:
- https://www.ibm.com/docs/en/sqsp/50?topic=repository-mirroring-quayio

So in order for me to setup my apphost to point to a custom registry, I would need to do the following things:
- use resilient-package to build the application .zip file (the command above will generate .zip in the dist of project directory)
- use "docker tag package_name:version" in my development machine
- user "docker push" to my custom registry of choice
- update the app.json (particularly in the image value)
- setup the configuration in my apphost to reflect connection to the custom registry
- pull the docker image 
- install the .zip as usual (using the SOAR) application
Is there any steps that I have missed 

Currently following the guide from the video below:
https://www.youtube.com/watch?v=Gkk3af0hCq0

Also from the course catalog, Is there any comprehensive guide followup to the resilient-sdk where they install to custom registry. 

Again, thank you for your guidance on the topic at hand. Sorry for the long thread.

Regards,

Luqman

Hi -

Let me try my best to address each of these questions, please let me know if I'm misunderstanding or forgetting certain details of your question.

For private registry hosting, anyone who has network access and credentials to pull from the registry will be able to run that app with App Host on their SOAR instance. You would simply provide to them the .zip file produced in the /dist folder after resilient-sdk package is run, and ensure that their app host is configured to pull images from the private registry. If that registry is as you say "hosted locally", I'm not sure that would be the best approach. If by locally you mean on your personal machine, there would be complex networking to allow others to pull from the registry. The best would be to set up a private registry on a local network machine so that you can share it with other app host machines.

I think that you are getting a bit confused about running the app on app host vs on what we would call an "integration server." An integration server is when you type "resilient-circuits run" from the command line. This is very much recommended only as a development environment. Once you have completed development of the app, you should run "resilient-sdk package" and subsequently build and push with docker to the private registry. The app from then on should be deployed via the App Host mechanism.

More to the point, your question about proper packages on another machine's environment, and sharing the custom functions, is not relevant if they are running in app host. When running in app host, the app will work properly no matter the environment. If you wish to share a package with another "integration server" (i.e. someone else running "resilient-circuits run" locally), then yes they will have to replicate your python environment and your app.config file to properly get things to run.

I hope this addresses most of your concerns and hopefully illuminates the different ways of running apps in SOAR.

Hi Bo Beckel,

Thanks for the succinct answer given above, I can clearly understand that after packaging the app you can opt to host it on any repo/storage as container.

For the private registry hosted locally, do you meant that I can package my custom application using "resilient-sdk package" and push it as an image such that everyone with access to that docker image will be able to access it?

Also is it possible that, after installing the same module (resilient-sdk, and other packages need for the function to work) other user will be able to access the function provided that they have the access to the same SOAR environment. Can this method be an alternative, where the custom app is not package.

My understanding is that currently I am able to run the custom application in my org SOAR after running the resilient-circuits run, does this translate as I pass the code to other computer and after they have run resilient-circuits run, they also can run the custom function in the same SOAR environment that I have access to.

Hello,

Great clarifying question. Yep, App Host will work with any Docker registry you want, whether thats ICR, Quay.io, Dockerhub, or something else including a private registry hosted locally or publicly. As long as the app host machine has network access to that registry, you can host your images there. You simply have to change the registry that the app host machine looks for with the manageAppHost registry command.

Hope that clears things up! Let me know if there's more I can answer for you.

Hi Bo Beckel, 

I have watched the tutorial from the security learning academy specifically the "Build your first Qradar SOAR custom integration using the App Host". I have one glaring question regarding the container implementation part. In the guide video provided you shown example on building the docker images locally then pushing it to the ICR repository.

- I tried to register to the IBM cloud and did not managed to "upgrade account" due to some error (Can't search the solution since its coded)

- You also mentioned that for the Apphost server itself you create different server that points to either icr or quay.io. So if say I want to use other repo for storing my images online (e.g. Dockerhub) will I have to modify the setting on my apphost as well or I can use existing default apphost which I assume available for quay.io

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

Hi -

My best guess at that error message is that you're missing the proper name in the Docker tag. I would again point you to https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205 and say that you should pay specific attention to the end of the "Build your first Qradar SOAR custom integration using the App Host" course where there is a specific walk through of how to use resilient-sdk and docker to build the image. Make sure that you have the right repository name included in the package command and that it matches the docker build command precisely.

Good luck!

Bo

Hi Bo Bleckel,

Thanks for your prompt reply and elaborate answer. Since applying some of your pointer, I have encountered roadblock when deploying my app in the SOAR where the error occur say "the tag cant be found".From my understanding, 

Default apphost is configured to connect to quay.io, however user are free to set the apphost to point to their choice of registry (whether it is locally or cloud hosted).

Although there are some limitations in doing so, like apphost can only point towards single registry (correct me if I am wrong here) example shown in the documentation below:
- https://www.ibm.com/docs/en/sqsp/50?topic=repository-mirroring-quayio

So in order for me to setup my apphost to point to a custom registry, I would need to do the following things:
- use resilient-package to build the application .zip file (the command above will generate .zip in the dist of project directory)
- use "docker tag package_name:version" in my development machine
- user "docker push" to my custom registry of choice
- update the app.json (particularly in the image value)
- setup the configuration in my apphost to reflect connection to the custom registry
- pull the docker image 
- install the .zip as usual (using the SOAR) application
Is there any steps that I have missed 

Currently following the guide from the video below:
https://www.youtube.com/watch?v=Gkk3af0hCq0

Also from the course catalog, Is there any comprehensive guide followup to the resilient-sdk where they install to custom registry. 

Again, thank you for your guidance on the topic at hand. Sorry for the long thread.

Regards,

Luqman

Hi -

Let me try my best to address each of these questions, please let me know if I'm misunderstanding or forgetting certain details of your question.

For private registry hosting, anyone who has network access and credentials to pull from the registry will be able to run that app with App Host on their SOAR instance. You would simply provide to them the .zip file produced in the /dist folder after resilient-sdk package is run, and ensure that their app host is configured to pull images from the private registry. If that registry is as you say "hosted locally", I'm not sure that would be the best approach. If by locally you mean on your personal machine, there would be complex networking to allow others to pull from the registry. The best would be to set up a private registry on a local network machine so that you can share it with other app host machines.

I think that you are getting a bit confused about running the app on app host vs on what we would call an "integration server." An integration server is when you type "resilient-circuits run" from the command line. This is very much recommended only as a development environment. Once you have completed development of the app, you should run "resilient-sdk package" and subsequently build and push with docker to the private registry. The app from then on should be deployed via the App Host mechanism.

More to the point, your question about proper packages on another machine's environment, and sharing the custom functions, is not relevant if they are running in app host. When running in app host, the app will work properly no matter the environment. If you wish to share a package with another "integration server" (i.e. someone else running "resilient-circuits run" locally), then yes they will have to replicate your python environment and your app.config file to properly get things to run.

I hope this addresses most of your concerns and hopefully illuminates the different ways of running apps in SOAR.

Hi Bo Beckel,

Thanks for the succinct answer given above, I can clearly understand that after packaging the app you can opt to host it on any repo/storage as container.

For the private registry hosted locally, do you meant that I can package my custom application using "resilient-sdk package" and push it as an image such that everyone with access to that docker image will be able to access it?

Also is it possible that, after installing the same module (resilient-sdk, and other packages need for the function to work) other user will be able to access the function provided that they have the access to the same SOAR environment. Can this method be an alternative, where the custom app is not package.

My understanding is that currently I am able to run the custom application in my org SOAR after running the resilient-circuits run, does this translate as I pass the code to other computer and after they have run resilient-circuits run, they also can run the custom function in the same SOAR environment that I have access to.

Hello,

Great clarifying question. Yep, App Host will work with any Docker registry you want, whether thats ICR, Quay.io, Dockerhub, or something else including a private registry hosted locally or publicly. As long as the app host machine has network access to that registry, you can host your images there. You simply have to change the registry that the app host machine looks for with the manageAppHost registry command.

Hope that clears things up! Let me know if there's more I can answer for you.

Hi Bo Beckel, 

I have watched the tutorial from the security learning academy specifically the "Build your first Qradar SOAR custom integration using the App Host". I have one glaring question regarding the container implementation part. In the guide video provided you shown example on building the docker images locally then pushing it to the ICR repository.

- I tried to register to the IBM cloud and did not managed to "upgrade account" due to some error (Can't search the solution since its coded)

- You also mentioned that for the Apphost server itself you create different server that points to either icr or quay.io. So if say I want to use other repo for storing my images online (e.g. Dockerhub) will I have to modify the setting on my apphost as well or I can use existing default apphost which I assume available for quay.io

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

Suppose that container image path is like this:

   <registry-host>:<registry-port>/<repository-name>/<app-name>:<app-version>

These values may come from the following configurations: 

(1) registry during apphost setup

          manageAppHost registry --registry=<registry-host>:<registry-port> [--user=<user>]

(2) during application packaging

          resilient-sdk -p <path_to_directory> --repository-name <repository-name>

(3) during generating source distribution

          python setup.py sdisk

             setup.py(

                 setup(

                      name="<app-name>"

                      version="app-version>"

(2) and (3) may be seen from: 

        cat app.json | jq .current_installation.executables[0].image

default <repository-name> is ibmresilient. So you may need to specify your <respository-name> instead.

I think custom container image should match the above settings. 

Hi -

My best guess at that error message is that you're missing the proper name in the Docker tag. I would again point you to https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205 and say that you should pay specific attention to the end of the "Build your first Qradar SOAR custom integration using the App Host" course where there is a specific walk through of how to use resilient-sdk and docker to build the image. Make sure that you have the right repository name included in the package command and that it matches the docker build command precisely.

Good luck!

Bo

Hi Bo Bleckel,

Thanks for your prompt reply and elaborate answer. Since applying some of your pointer, I have encountered roadblock when deploying my app in the SOAR where the error occur say "the tag cant be found".From my understanding, 

Default apphost is configured to connect to quay.io, however user are free to set the apphost to point to their choice of registry (whether it is locally or cloud hosted).

Although there are some limitations in doing so, like apphost can only point towards single registry (correct me if I am wrong here) example shown in the documentation below:
- https://www.ibm.com/docs/en/sqsp/50?topic=repository-mirroring-quayio

So in order for me to setup my apphost to point to a custom registry, I would need to do the following things:
- use resilient-package to build the application .zip file (the command above will generate .zip in the dist of project directory)
- use "docker tag package_name:version" in my development machine
- user "docker push" to my custom registry of choice
- update the app.json (particularly in the image value)
- setup the configuration in my apphost to reflect connection to the custom registry
- pull the docker image 
- install the .zip as usual (using the SOAR) application
Is there any steps that I have missed 

Currently following the guide from the video below:
https://www.youtube.com/watch?v=Gkk3af0hCq0

Also from the course catalog, Is there any comprehensive guide followup to the resilient-sdk where they install to custom registry. 

Again, thank you for your guidance on the topic at hand. Sorry for the long thread.

Regards,

Luqman

Hi -

Let me try my best to address each of these questions, please let me know if I'm misunderstanding or forgetting certain details of your question.

For private registry hosting, anyone who has network access and credentials to pull from the registry will be able to run that app with App Host on their SOAR instance. You would simply provide to them the .zip file produced in the /dist folder after resilient-sdk package is run, and ensure that their app host is configured to pull images from the private registry. If that registry is as you say "hosted locally", I'm not sure that would be the best approach. If by locally you mean on your personal machine, there would be complex networking to allow others to pull from the registry. The best would be to set up a private registry on a local network machine so that you can share it with other app host machines.

I think that you are getting a bit confused about running the app on app host vs on what we would call an "integration server." An integration server is when you type "resilient-circuits run" from the command line. This is very much recommended only as a development environment. Once you have completed development of the app, you should run "resilient-sdk package" and subsequently build and push with docker to the private registry. The app from then on should be deployed via the App Host mechanism.

More to the point, your question about proper packages on another machine's environment, and sharing the custom functions, is not relevant if they are running in app host. When running in app host, the app will work properly no matter the environment. If you wish to share a package with another "integration server" (i.e. someone else running "resilient-circuits run" locally), then yes they will have to replicate your python environment and your app.config file to properly get things to run.

I hope this addresses most of your concerns and hopefully illuminates the different ways of running apps in SOAR.

Hi Bo Beckel,

Thanks for the succinct answer given above, I can clearly understand that after packaging the app you can opt to host it on any repo/storage as container.

For the private registry hosted locally, do you meant that I can package my custom application using "resilient-sdk package" and push it as an image such that everyone with access to that docker image will be able to access it?

Also is it possible that, after installing the same module (resilient-sdk, and other packages need for the function to work) other user will be able to access the function provided that they have the access to the same SOAR environment. Can this method be an alternative, where the custom app is not package.

My understanding is that currently I am able to run the custom application in my org SOAR after running the resilient-circuits run, does this translate as I pass the code to other computer and after they have run resilient-circuits run, they also can run the custom function in the same SOAR environment that I have access to.

Hello,

Great clarifying question. Yep, App Host will work with any Docker registry you want, whether thats ICR, Quay.io, Dockerhub, or something else including a private registry hosted locally or publicly. As long as the app host machine has network access to that registry, you can host your images there. You simply have to change the registry that the app host machine looks for with the manageAppHost registry command.

Hope that clears things up! Let me know if there's more I can answer for you.

Hi Bo Beckel, 

I have watched the tutorial from the security learning academy specifically the "Build your first Qradar SOAR custom integration using the App Host". I have one glaring question regarding the container implementation part. In the guide video provided you shown example on building the docker images locally then pushing it to the ICR repository.

- I tried to register to the IBM cloud and did not managed to "upgrade account" due to some error (Can't search the solution since its coded)

- You also mentioned that for the Apphost server itself you create different server that points to either icr or quay.io. So if say I want to use other repo for storing my images online (e.g. Dockerhub) will I have to modify the setting on my apphost as well or I can use existing default apphost which I assume available for quay.io

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.

To Yohji Amano,

I generally understand the process that need to achieve to use the application on local Apphost server, however there is still issues occurring when fetching the image. Below are the detailed steps that I understand needed to be applied after finishing the development of application:

- packaging the application; 

- building the docker image from the package created:

- login to the image repository (currently using default quay.io):

- pushing the image to the repository:

- setting the app.config: api key settings, 

- deploy the application to the correct apphost, which the following error occur:

I have also applied the setup.py setup with the app_name and current version which is 1.0.0 matching the tag. Is there any configuration that I have missed? I am considering the following might be an issue:

• using default dockerfile
• pushing base image on quay.io without additional builds/setup applied

I appreciate if you can guide me on the steps that I might have wrongly applied, or missed it entirely. Thanks for the previous advice.

Suppose that container image path is like this:

   <registry-host>:<registry-port>/<repository-name>/<app-name>:<app-version>

These values may come from the following configurations: 

(1) registry during apphost setup

          manageAppHost registry --registry=<registry-host>:<registry-port> [--user=<user>]

(2) during application packaging

          resilient-sdk -p <path_to_directory> --repository-name <repository-name>

(3) during generating source distribution

          python setup.py sdisk

             setup.py(

                 setup(

                      name="<app-name>"

                      version="app-version>"

(2) and (3) may be seen from: 

        cat app.json | jq .current_installation.executables[0].image

default <repository-name> is ibmresilient. So you may need to specify your <respository-name> instead.

I think custom container image should match the above settings. 

Hi -

My best guess at that error message is that you're missing the proper name in the Docker tag. I would again point you to https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205 and say that you should pay specific attention to the end of the "Build your first Qradar SOAR custom integration using the App Host" course where there is a specific walk through of how to use resilient-sdk and docker to build the image. Make sure that you have the right repository name included in the package command and that it matches the docker build command precisely.

Good luck!

Bo

Hi Bo Bleckel,

Thanks for your prompt reply and elaborate answer. Since applying some of your pointer, I have encountered roadblock when deploying my app in the SOAR where the error occur say "the tag cant be found".From my understanding, 

Default apphost is configured to connect to quay.io, however user are free to set the apphost to point to their choice of registry (whether it is locally or cloud hosted).

Although there are some limitations in doing so, like apphost can only point towards single registry (correct me if I am wrong here) example shown in the documentation below:
- https://www.ibm.com/docs/en/sqsp/50?topic=repository-mirroring-quayio

So in order for me to setup my apphost to point to a custom registry, I would need to do the following things:
- use resilient-package to build the application .zip file (the command above will generate .zip in the dist of project directory)
- use "docker tag package_name:version" in my development machine
- user "docker push" to my custom registry of choice
- update the app.json (particularly in the image value)
- setup the configuration in my apphost to reflect connection to the custom registry
- pull the docker image 
- install the .zip as usual (using the SOAR) application
Is there any steps that I have missed 

Currently following the guide from the video below:
https://www.youtube.com/watch?v=Gkk3af0hCq0

Also from the course catalog, Is there any comprehensive guide followup to the resilient-sdk where they install to custom registry. 

Again, thank you for your guidance on the topic at hand. Sorry for the long thread.

Regards,

Luqman

Hi -

Let me try my best to address each of these questions, please let me know if I'm misunderstanding or forgetting certain details of your question.

For private registry hosting, anyone who has network access and credentials to pull from the registry will be able to run that app with App Host on their SOAR instance. You would simply provide to them the .zip file produced in the /dist folder after resilient-sdk package is run, and ensure that their app host is configured to pull images from the private registry. If that registry is as you say "hosted locally", I'm not sure that would be the best approach. If by locally you mean on your personal machine, there would be complex networking to allow others to pull from the registry. The best would be to set up a private registry on a local network machine so that you can share it with other app host machines.

I think that you are getting a bit confused about running the app on app host vs on what we would call an "integration server." An integration server is when you type "resilient-circuits run" from the command line. This is very much recommended only as a development environment. Once you have completed development of the app, you should run "resilient-sdk package" and subsequently build and push with docker to the private registry. The app from then on should be deployed via the App Host mechanism.

More to the point, your question about proper packages on another machine's environment, and sharing the custom functions, is not relevant if they are running in app host. When running in app host, the app will work properly no matter the environment. If you wish to share a package with another "integration server" (i.e. someone else running "resilient-circuits run" locally), then yes they will have to replicate your python environment and your app.config file to properly get things to run.

I hope this addresses most of your concerns and hopefully illuminates the different ways of running apps in SOAR.

Hi Bo Beckel,

Thanks for the succinct answer given above, I can clearly understand that after packaging the app you can opt to host it on any repo/storage as container.

For the private registry hosted locally, do you meant that I can package my custom application using "resilient-sdk package" and push it as an image such that everyone with access to that docker image will be able to access it?

Also is it possible that, after installing the same module (resilient-sdk, and other packages need for the function to work) other user will be able to access the function provided that they have the access to the same SOAR environment. Can this method be an alternative, where the custom app is not package.

My understanding is that currently I am able to run the custom application in my org SOAR after running the resilient-circuits run, does this translate as I pass the code to other computer and after they have run resilient-circuits run, they also can run the custom function in the same SOAR environment that I have access to.

Hello,

Great clarifying question. Yep, App Host will work with any Docker registry you want, whether thats ICR, Quay.io, Dockerhub, or something else including a private registry hosted locally or publicly. As long as the app host machine has network access to that registry, you can host your images there. You simply have to change the registry that the app host machine looks for with the manageAppHost registry command.

Hope that clears things up! Let me know if there's more I can answer for you.

Hi Bo Beckel, 

I have watched the tutorial from the security learning academy specifically the "Build your first Qradar SOAR custom integration using the App Host". I have one glaring question regarding the container implementation part. In the guide video provided you shown example on building the docker images locally then pushing it to the ICR repository.

- I tried to register to the IBM cloud and did not managed to "upgrade account" due to some error (Can't search the solution since its coded)

- You also mentioned that for the Apphost server itself you create different server that points to either icr or quay.io. So if say I want to use other repo for storing my images online (e.g. Dockerhub) will I have to modify the setting on my apphost as well or I can use existing default apphost which I assume available for quay.io

Hi Luqman Nur,

I'm glad to see you're working on a custom app! These are great questions, I hope I can help.

The lib directory is a custom directory that was added by the developer of the app that you're reading through. You are welcome to add that to your app if you think it would be helpful organizationally, however, it is not required. You can create python files within the util directory as much as you want, or you can create a lib directory like you see there and add python files there. The components directory is generated by codegen and will contain the bones of the function code that you will need to manually fill out to run the app. You should modify those files generated by codegen, however, you shouldn't add any new files to the components directory.

There is no difference between CP4S and SOAR when it comes to using the SDK and creating an app.

As for the API key question, you should select an api key which you will use to develop your app. If you plan to run it locally on an integration server, whatever api key used for that would make the most sense. You can generate a new api key in the administrator settings under the Users tab. Once you've used resilient-sdk package to package the app, and you subsequently install the app in a new system, the system will automatically generate a new api key for your app.

Finally, the guide you linked is pretty good, but doesn't go into as much detail as possible. There is a good amount of content on the security learning academy (https://www.securitylearningacademy.com/local/navigator/index.php?level=msu2205) which will go into more depth.

To Qradar SOAR community,

Currently I am learning how to create custom application for SOAR. My current steps is to:

• Create function to be use in the custom application. From the SOAR web console > customizations > functions
• Create message destinations 
• Apply the message destination to the previously created application
• Create the local resilient package using " resilient-sdk codegen -p <package_name> -f <function_created_previously> -m <message_destination_created_previously>

There some few points that I had trouble understanding on how to create the custom application and how to get it to work:
- When creating custom application and message destination, there's a field where you have to specify the API key. Is this API key the same as the authentication needed when querying to an API url? Or this is different object entirely

• There also a directory called "lib" under the directory named "function_name" directory, as shown below. However when I create using the resilient-sdk codegen method, the directory lib is not created. 
•   From different package shown, the lib contains the Class Function that will initialise the object used and contains every single function that is used in the components function (which is mostly defining API endpoint). I'm currently following this guide (How To Create Your Own IBM Soar App

  Linkedin		remove preview
  How To Create Your Own IBM Soar App The article describes how to write our own IBM SOAR Apps for IBM Resilient or IBM SOAR on CP4S. View this on Linkedin >

  ) but this one is created on CP4S. Is it possible that these might be the reason where there is some difference on the package created.