Security Global Forum

My Introduction to QRadar Log Management for WebSEAL Administrators

I’ve been working with IBM Security Access Manager for Web, aka WebSEAL, for a very long time now. Most recently I’ve been responsible for deploying and using our WebSEAL technology, in the appliance form called ISAM for Web, as part of IBM’s Platform-as-a-Service offering known as Bluemix.

Check out the Single Sign On Service for our first entry point into SSO for Bluemix. 

In this role I’ve really had to adopt somewhat of an operations role, and one of the more enlightening and interesting aspects of that role has been deploying and learning a little about QRadar. In particular I’ve been using QRadar as a destination for WebSEAL request logs and exploring search and reporting capabilities. As a complete newbie to QRadar I must say I was very impressed with how quickly I was able to become a source of some very interesting and useful reporting information and I’d like to share some of the configuration tips and tricks I picked up along the way.

If you are a customer of our WebSEAL software or appliance offerings (versions that support sending log information to a syslog target), and are considering QRadar as a SIEM solution, then I hope you find what I’ve been doing useful. I’m absolutely certain there are a lot of things I haven’t learned about running QRadar so think of this article as a good starting point rather than “the definitive guide”, and definitely consider getting a QRadar services professional to help with your own deployment and configuration planning.

Systems View of WebSEAL and QRadar

Essentially this diagram shows the type of environment I was trying to establish – using QRadar as a log target for WebSEAL request logs. WebSEAL has the built-in ability to send these via the remote syslog protocol.

Configuring WebSEAL to send formatted request log entries to QRadar

Note that there were some specific constraints and requirements in the environment that I was setting up that led me down this particular configuration path. They may not apply to you, however here’s some that did apply to me:

• I preferred not to turn on WebSEAL XML-based auditing as the records are quite large, requiring a lot of storage, and they didn’t actually contain all the information I wanted to log and perform queries on
• I wanted to log custom attributes from user credentials on each WebSEAL access
• I wanted to be able to trace all requests in a particular user session, both authenticated and unauthenticated to aid in debugging user authentication flows

You might not have known this, however WebSEAL can be configured to have a custom format for the request log, and can send that request log off to an external source using the syslog protocol. For the various WebSEAL instances in my environment, I want to send events to the same single QRadar server, and I wanted the events to have a custom format.

First let’s take a look at the commented documentation in the webseal configuration file that describes the customization possibilities of the request log. These appear in the [logging] stanza:

# The request-log-format to be written to the request log.
# The following directives can be used to customize the log format.
#
# %a: Client IP Address
# %A: Local IP Address
# %b: Bytes in the response excluding HTTP headers in CLF format: ‘-‘ instead of 0 when no bytes are returned.
# %B: Bytes in the response excluding HTTP headers
# %{attribute}C: Attribute from the TAM credential named ‘Attribute’
# %{cookie}e: Contents of the Cookie ‘cookie’ in the request
# %{cookie}E: Contents of the Cookie ‘cookie’ in the response
# %d: Transaction identifier, or session sequence number.
# %F: Time taken to serve the request in microseconds
# %h: Client host
# %H: Request protocol
# %{header}i: Contents of the Header ‘header’ in the request
# %j: The name of the junction servicing the request
# %l: Client logname (RFC 1314) (default -)
# %m: Request method (i.e. GET, POST, HEAD)
# %{header}o: Contents of the Header ‘header’ in the response
# %p: Port over which the request was received
# %q: The decoded query string (prepended with ‘?’ or empty)
# %Q: The raw query string (prepended with ‘?’ or empty).
# %r: First line of the request with decoded URL
# %R: First line of the request with decoded URL including HTTP://HOSTNAME
# %s: Response status
# %t: Time in Common Log Format format
# %{format}t: The time in the given format
# %T: Time taken to serve the request in seconds, or part thereof
# %u: Remote user
# %U: The URL requested
# %v: Canonical ServerName of the server servicing the request
# %z: The decoded path string
# %Z: The raw path string

Some of this will be very useful to us, particularly some of the credential attributes. In the case of my WebSEAL systems, authentication is done via federated SSO, and a mapping rule, and credentials ALWAYS are created with a couple of extra attributes – userRealm and email. I wanted to be able to query on these attributes in QRadar, hence I want them in the request log when available. There are also some other interesting attributes I found useful. Here’s my servers setting for the request-log-format:

request-log-format = TS:%{%Y-%m-%dT%H:%M:%S}t U:%{AZN_CRED_PRINCIPAL_NAME}C AL:%{AUTHENTICATION_LEVEL}C SI:%{tagvalue_session_index}C T:%d M:%m E:%{email}C R:%{userRealm}C ST:%s B:%B F:%F URL:%U

Notice how there are several attributes which come from the credential, and there are several other “standard” attributes as well. Each is prefixed with a short identifier, such as “TS:”. These prefixes are used as simple field separators, and as you will see later allow us to easily extract the various attribute values.

A brief explanation of each of these components of the request log format follows:

Besides the format of the request log, the other part of the configuration for sending request log entries to QRadar is the syslog target information. This appears in the [aznapi-configuration] stanza of the webseal configuration file:

logcfg = http.clf:rsyslog server=10.81.101.126,port=514,log_id=webhttp

Note that in this entry the QRadar machine is specified by the IP address 10.81.101.126.

Other WebSEAL configuration options

One thing I wanted to be able to do in QRadar was track a browser-user’s progress through a login – from unauthenticated through to an authenticated state. To do this on a single session it’s important that WebSEAL uses session indexes, and maintains the same session index while transitioning from unauthenticated to authenticated. This is not the default behaviour. The required settings for doing this are:

[session]
user-session-ids=yes
create-unauth-sessions=yes

That covers all the required WebSEAL configuration settings – I just set these the same for each WebSEAL in the environment. Let’s now look at the QRadar setup and configuration.

QRadar Virtual Appliance and Fixpack Installation

For installation of the QRadar virtual appliance I got the install manual and more-or-less followed it verbatim. I applied the latest fixpack available at the time, information came from the Software upgrade progression for QRadar appliances page. The best piece of advice I can give here is to give QRadar plenty of disk. When you think you have given it enough, double it, add a bit more, and that’s about right.

The system I’ve deployed has the following significant VM characteristics:

• 12G Memory, using a dedicated resource pool in VMWare
• 4 CPU’s (I chose Number of virtual sockets = 4, Number of cores per virtual socket = 1)
• 2 TB thin-provisioned hard drive

It’s important to note that this environment doesn’t process what I would consider a LOT of requests. To give you a rough idea, the total number of request log entries sent by WebSEAL’s to QRadar in this environment in the week prior to me writing this article was right around 1 million. Chicken feed really. I store all request records (no coaelesing), have a log retention policy of 3 months, and at present my 2TB hard disk on my single QRadar appliance is 45% full.

Event Retention

It is important to consider how long you will retain events for, as this will affect performance and required storage. In my environment I wanted to store WebSEAL request log events for 3 months. This can be done with a rule in the Event Retention settings on the Admin panel of the QRadar console, as shown:

Notice the filter of “Event Name is Miscellaneous Webseal-Instance Event”. This is the name of pre-defined event type in QRadar for WebSEAL request log entries. Use this filter.

DSM Parsing Log Source Extension

Note: This piece of configuration informtion is being presented now because it’s useful to have it set up prior to creating log sources in QRadar.

For structed event data (e.g. XML), data source modules (DSM’s) in QRadar can typically extract and interpret all attributes. This is true for the WebSEAL XML-based authentication and authorization audit logs, however in the case of unstructed, variable-format, string-based data such as request log entries, a DSM could not know in advance how to parse out attributes. Log source extensions may be written to assist a DSM in parsing out standard QRadar event attributes that the DSM doesn’t know how to handle. For the request log entries that WebSEAL has been configured to send, there are only two standard QRadar event attributes that apply. These are:

• Log Source Time – the time the event occurred according to the event source (WebSEAL)
• Username – the identity associated with the event

A log source extension has been written to extract these attributes from our request log entries. All other attributes in the request log records sent from WebSEAL are not standard-to-QRadar attributes, and will be treated as custom event properties.

When we configure WebSEAL to send request log events to QRadar, each request log entry contains a timestamp. The format of this timestamp is fixed, and set by WebSEAL. The value will also depend on the timezone that the WebSEAL appliance is set to. The on-the-wire format of a request log entry that WebSEAL will send will be a text string that looks something like:

<114>1 2014-08-24T23:55:46-04:00 web-2 webhttp 5914 – – TS:2014-08-24T23:55:36 -0400 U:unauthenticated AL:- SI:9a5403a0-2c0b-11e4-a170-000c296dc236 T:13 M:POST E:- R:- ST:200 B:6502 F:000012152 URL:/idaas/public/tamlogin.jsp

There are two timestamp in this record. The first is in the syslog header, and is not part of the variable format for a request log entry. This time represents when WebSEAL sends the syslog message to QRadar. It is not particularly useful because these are sent periodically. The second timestamp is the one in the request-log format prefixed by “TS:”. This time is WebSEAL’s view of the time the event occured, which is important, and what we are interested in setting as the Log Source Time. QRadar will record it’s own time of when it received the event (Storage Time), however as WebSEAL batches and sends events periodically (by default 20 seconds), that time is rather uninteresting in any log analysis and will usually be 1 or 2 seconds later than the (equally useless) syslog header time. QRadar doesn’t know how to automatically parse the TS: timestamp, and as such I have written a log source extension which can read and parse the time, and will set the time as the Log Source Time. The log source extension ignores the timezone offset so it is important that when authoring search queries you understand that the time presented in QRadar log records will be the WebSEAL-provided syslog header time from the records, without timezone offset information. The WebSEAL servers in my environment were all set for US-eastern time, and this record was taken during daylight savings, hence the -4:00 offset. As far as QRadar is concerned, the record will be recorded with a Log Source Time of 23:55:36 on 24th August, 2014. Apply your own knowledge of the timezone setting on your WebSEAL system to figure out when that really was.

Another key standard event attribute for QRadar is the Username associated with the event. For our custom syslog event, this is also by the extension. The extension provided here also does that.

The log source extension should be configured using the Admin panel on the QRadar console, as shown:

The contents of the log source extension file is:

<?xml version=”1.0″ encoding=”UTF-8″?>
<device-extension xmlns=”event_parsing/device_extension”>

<pattern id=”allEventNames” xmlns=””><![CDATA[(.*)]]></pattern>

<pattern id=”RequestLogEventPattern” xmlns=””><![CDATA[<114>1 (.*)]]></pattern>

<pattern id=”TimeJan” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-01-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeFeb” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-02-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeMar” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-03-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeApr” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-04-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeMay” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-05-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeJun” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-06-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeJul” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-07-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeAug” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-08-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeSep” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-09-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeOct” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-10-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeNov” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-11-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>
<pattern id=”TimeDec” xmlns=””><![CDATA[TS:([0-9][0-9][0-9][0-9])-12-([0-9][0-9])T([0-9][0-9]):([0-9][0-9]):([0-9][0-9])]]></pattern>

<pattern id=”UsernamePattern” xmlns=””><![CDATA[ U:(.*) AL:]]></pattern>

<match-group order=”1″ description=”IDaaS Request Log Events” xmlns=””>

<matcher field=”DeviceTime” order=”1″ pattern-id=”TimeJan” capture-group=”Jan 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”2″ pattern-id=”TimeFeb” capture-group=”Feb 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”3″ pattern-id=”TimeMar” capture-group=”Mar 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”4″ pattern-id=”TimeApr” capture-group=”Apr 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”5″ pattern-id=”TimeMay” capture-group=”May 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”6″ pattern-id=”TimeJun” capture-group=”Jun 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”7″ pattern-id=”TimeJul” capture-group=”Jul 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”8″ pattern-id=”TimeAug” capture-group=”Aug 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”9″ pattern-id=”TimeSep” capture-group=”Sep 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”10″ pattern-id=”TimeOct” capture-group=”Oct 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”11″ pattern-id=”TimeNov” capture-group=”Nov 2 3:4:5″ enable-substitutions=”true”/>
<matcher field=”DeviceTime” order=”12″ pattern-id=”TimeDec” capture-group=”Dec 2 3:4:5″ enable-substitutions=”true”/>

<matcher field=”EventName” order=”1″ pattern-id=”RequestLogEventPattern” capture-group=”Miscellaneous Webseal-Instance Event” enable-substitutions=”true” />
<matcher field=”UserName” order=”1″ pattern-id=”UsernamePattern” capture-group=”1″/>

<event-match-single event-name=”Miscellaneous Webseal-Instance Event” device-event-category=”Information” send-identity=”OverrideAndNeverSend” />

</match-group>

</device-extension>

After configuring the Log Source Extension, QRadar will prompt you to Deploy Changes. Follow the instructions provided on-screen to do that.

Log Source Configuration

For the most part, QRadar can detect log sources by way of the format of inbound event messages. In the case of our WebSEAL request log events however, the data is not in a structured format that can be auto-detected by QRadar. Therefore I decided to creat the log sources manually and pre-assign them the correct log event type. Further complicating log source creation in my environment was the fact that each physical source (a WebSEAL appliance) was configured with two NIC’s on the same subnet. This had everything to do with the fact that I was running the WebSEAL appliances on Softlayer and I had both a management and application interface on the same internal 10. subnet. What this means is that QRadar could receive events from the same real appliance from two different source IP addresses (the outbound routing on WebSEAL appliance can and sometimes does select either). That in turn means that for each physical WebSEAL appliance I needed to configure two different log sources, one for each IP on the same subnet. Given that I had three physical WebSEAL systems, I confgured 6 log sources in QRadar, as shown below. In your own environment most likely you will only need one log source for each WeBSEAL/appliance machine – it all depends how many NIC’s you have configured on the WebSEAL system which are on the same subnet as the QRadar system.

WebSEAL network configurations

• WebSEAL 1 – M.1 (10.81.101.116), P.1 (10.81.101.115)
• WebSEAL 2 – M.1 (10.81.101.106), P.1 (10.81.101.105)
• WebSEAL 3(development) – M.1 (10.81.101.101), P.1 (10.81.101.100)

QRadar Log Sources

Here is a screenshot showing the QRadar log source configuration for one of those sources. Note that each log source should be configured to use the idaas_dsm_extension log source extension uploaded earlier:

Pay particular attention to the Coalescing Events setting (it needs to be off). If you don’t do that, you will not see all events from WebSEAL appear on QRadar. Don’t be concerned if the Status column for newly-created log sources shows as Error. That simply means it hasn’t received any events recently.

Once again after configuring the log sources QRadar will tell you that a deploy operation is required – go ahead and do that.

Send An Event

Now that QRadar is configured with log sources – let’s send it an event. The best way to do that is to just use a browser to access a page on your WebSEAL system. In the administration console on QRadar, go to the Log Activity view, and watch for something from your newly-configured log sources. It can take up to 20 seconds to arrive because WebSEAL will batch and send the syslog entries. The default Log Activity view is to watch for all events, in real time, from all sources. This should be ok to get started however you will soon want to add a filter to look only for events from your WebSEAL log sources. The first event from a WebSEAL system, when using the default log activity view, should look something like this:

While not very useful yet, even the presence of “unauthenticated” in the Username field tells us that the log source extension we configured earlier is working. Pause the realtime log stream, and double-click on the first event to open it. It should look like this:

Notice the payload. Notice how the TS: timestamp in the payload matches the Log Source Time in the event, and how this is different to the Storage Time and Start Time from the event (which are when QRadar received it). This is also because of the log source extension we configured earlier. There’s also some extra stuff in payload that would be interesting – let’s get it.

Extracting Custom Event Properties

Because we are sending a custom-format syslog string, QRadar doesn’t recognize the various fields within it. There is however a simple way to extract the different properties using regular expression matching. Using the Extract Property button at the top of your event, add a new property with the following options. Carefully check every property on this panel:

Repeat for all of these properties and regular expressions – depending on the request-log-format you have chosen for WebSEAL:

You should be able to see your new properties, where they have values, as shown:

Now you have attributes that are useful, and these can be used in searches and reports. Let’s set up a search which we can use as a default view for the activity log that will show just WebSEAL events.

Creating a Search

In the Log Activity tab of the QRadar admin console, click on Search -> Edit Search. Create a new search, for Real Time (streaming), add your own Column Definitions, and set the Search Parameters to only match Miscellaneous Webseal-Instance Event, as shown:

Click on any of the Filter buttons to return to the Log Activity view and see your search in action. Send some more WebSEAL events using your browser, and watch them arrive fully formatted:

Notice that if a request is authenticated, username information is also displayed. The sequence number can be used, along with the session index, to see the sequential order of requests from a user in a particular session. The query can be saved as the default (using the Save Criteria button) so that each time you launch the QRadar that is what you see.

Advanced queries can now be formulated to discover and report on information including authentication statistics. This will be the topic of my next article!