Today organizations use variety of security tools which are deployed both on premises and in the cloud. These different security tools generate their own set of events and logs scattered across the enterprise. These different events and logs cannot be interpreted in the same way as they are from different security devices and are based on different programming languages. Due to this very reason a threat can be missed. To solve this problem IBM has come up with Cloud Pak for Security also known as CP4S. CP4S is a hybrid model which is built on open-source standards. CP4S comes under the Cloud Packs of IBM. CP4S will help you to connect all the information scattered among different security tools. This creates a more intensive security analysis tool that uses existing tools in a proper manner and provide a more effective approach for security administrators to identify, detect and respond to security threats. If someone wants to search for data across the different security products present in their deployment, then this can be achieved with the help of CP4S. STIX-shifter is a component of CP4S which makes the interaction of data between different siloed security solutions possible.
What is STIX?
STIX stands for Structured Threat Information Expression. It is a standardized xml programming language that we can use for conveying data about cyber security threat in common language. With STIX we can represent all aspects of suspicion, compromise and attribution clearly with objects and descriptive relationships. For an analyst STIX information can be visually represented or stored as JSON so that it can be quickly readable by machine. This format delivers an efficient and machine-readable way to enable collaborative threat analysis, automated threat information exchange, automated detection of threat and response, and more. STIX is open source and hence allows for integration with existing tools and products. STIX-shifter is part of the Open Cybersecurity Alliance.
What is OCA?
OCA stands for the Open Cybersecurity Alliance. It is an OASIS open project. The aim of OCA is to bridge the gaps between different cybersecurity tools so that these products can freely exchange information with mutual agreement on technologies, standards and procedures. IBM Security is a co-founder and initial contributor to the OCA project. STIX-shifter federated search technology is contributed to OCA by IBM. STIX-shifter is a core capability offered in IBM Cloud Pak for Security. OCA also consists of Kestrel which is a threat hunting language. In this blog our focus will be on STIX-shifter. You can find more information about OCA in the link below:
You can find more information on the current projects in OCA in the links below:
What is STIX-shifter?
STIX-shifter is an open-source python library. STIX-shifter is an IBM collaboration with the members of Open Cybersecurity alliance. Stix-shifter federates data from different security tools. It is at the heart of IBM CP4S providing SOC team the ability to pull insights from different SIEM, EDR and Data Protection tools into a single platform. STIX-shifter is an open-source python library that enables software to connect to products that house data repositories. STIX-shifter makes use of the STIX patterns to transform the output into data that mostly looks and behaves the same. One of the remarkable things about STIX-shifter is its ability to create search patterns. These search patterns are available for all three types of security data source types like network, file, log, and more. Since it covers all three data types, we can create complicated queries and analytics that is scattered in multiple domains including SIEM, Endpoint, Network, and file levels.
STIX-shifter comes with a bundled script that can be used to translate STIX patterns into a native data source query. Translated json data source query result in STIX bundle of observable object. This can be used to send a query to a data source via a transmission option. As a CLI tool we can use STIX-shifter to create searches or queries as part of orchestration workflow to enrich IOCs or artifacts for SIEM or to integrate workflow between multiple tools or to create cross platform playbook. As a library you can integrate STIX-shifter in your own tools to add query or enrichment functionality. STIX-shifter can also be used to provide a common way to query data or to provide a standard way to integrate with your own solution to access data. STIX-shifter can natively search data for products that currently support STIX and TAXII. The biggest advantage of STIX-shifter is that the data resides where it is. You do not need to copy the data from data source to CP4S. STIX-shifter will only connect to the data repository and query or extract the data that has been asked for.
Why would someone want to use STIX-shifter instead of native query?
The biggest advantage of STIX-shifter is that you do not need to copy the data from data source to CP4S. This means that the data resides where it is. STIX-shifter will only connect to the data repository and query the data. STIX-shifter can help organisations and analysts benefit with data and tools that they already have by creating a common way to query the information available. STIX-shifter enriches data empowering security operation and inter-operability of security products. With the help of STIX-shifter one can make searches and/or queries as part of threat orchestration workflow. Also, you can make use of STIX-shifter if you have security data that can be made available, and you want to create or contribute an adapter.
How to setup STIX-shifter?
To install STIX-shifter you will need to setup a virtual environment of Ubuntu. Other flavours of Unix also work. However, for the sake of simplicity, we would cover the steps for Ubuntu in this blog. If you already have an existing Ubuntu setup, then you do not need this virtual environment.
To setup a virtual environment below is the requirement:
- Ubuntu – version 20.04
- Python – version 3.6 or above
Below are the steps to setup a virtual environment on MAC OS. Similar steps can be followed for Windows OS as well:
- You can go to site: https://www.virtualbox.org/. Here you will get an option saying “Download VirtualBox 6.1”. Click on that option. VirtualBox 6.1 is the latest version while writing this blog.
- Once you will click on the “Download VirtualBox 6.1” option you will be redirected to a page that will consists of below options under title “VirtualBox 6.1.32 platform packages”:
- From the above list we will click on “OS X hosts” which will then start downloading VirtualBox.dmg file. Once this file in downloaded we will click on that .dmg file and it will provide us with two options given below:
- We would need to now double click on the VirtualBox.pkg. Select “continue” when prompted:
- After that we will get an option to change install location of virtual box. If you don’t want to change the location, then you can proceed by clicking “install”. After this you will need to provide your password and then click on “install software”:
- At first the installation will fail since it will seek for few permissions. To rectify this issue you will need to go to “System Preferences”:
- After this click on “Security and Privacy” icon. This will open up new window. In the new window you will need to click on “allow” and click on lock icon. It will ask for your password. Enter your password when prompted:
- Once the above changes are done then just click on “close” and then click on “keep”:
- Now again you will need to follow the steps 3 to 5 again. This time it will install the virtual box successfully as we have already provided the required permission:
- Now click on “close” and then click on “move to trash”. Once this is done go to launchpad. Here you will be able to see below icon for VirtualBox:
- Click on this icon to launch VirtualBox App. Now you need iso of Ubuntu OS to install Ubuntu on VirtualBox.
- To download the Ubuntu iso file you will need to go to: https://ubuntu.com/. Under download section you will get an option of “Ubuntu Desktop”. From here you can download Ubuntu version 20.04 LTS:
- Once you have downloaded the ISO file you will be able to install Ubuntu on your VirtualBox. You will just need to click on “new” and proceed with further steps of installing Ubuntu:
Once Ubuntu is installed you can do the following checks for python and pip version.
By default, Ubuntu 20.04 version consists of python 3. This can be verified by running below command:
It will appear like this:
Pip is a standard package manager used to install and maintain packages for Python. To check whether pip 3 is installed, run the following command:
It will appear like this:
Once we are done with creating the virtual environment with the above steps, we can now proceed with the installation of STIX-shifter.
You will need to open the above link and click on code. This will provide an option to copy the link. Click on copy option as shown in the below screenshot. This will copy the link.
Once that is done, go to your virtual environment (Ubuntu) and open a terminal. Here you will need to clone the STIX-shifter repository by running the below command:
#git clone <type the link you copied from git hub> -b master
It will appear like this:
When the copy is complete go to STIX-shifter directory to verify the files by running the command given below:
Now we are ready to install STIX-shifter. It is recommended to use pip for installing the STIX-shifter. Also, there are two prerequisite packages that needs to be installed. This will include the package of STIX-shifter connector module to complete a STIX-shifter connector installation.
To install all the packages you will need to follow the below procedure:
- Install main STIX-shifter package using the command given below:
#pip install stix-shifter
- Install STIX-shifter Utility package using the command given below:
#pip install stix-shifter-utils
- Install desired STIX-shifter connector module package using the command given below:
#pip install stix-shifter-modules-<module name>
Example: pip install stix-shifter-modules-qradar
Using the above steps STIX-shifter has now been installed and is ready for running queries on data sources.
For sake of simplicity, running queries using our above setup would be covered in another Blog.
If at any point in time you have any questions or have any comments or want to discuss this further, feel free to get in touch with us and we would be more than happy to answer any of your queries: