IBM Security QRadar

Expand all | Collapse all

Azure Active Directory and Office 365 Logging

  • 1.  Azure Active Directory and Office 365 Logging

    Posted Tue September 10, 2019 02:33 PM

    Hello,

    I am working on configuring our Azure Active Directory and Office 365 logging in QRadar on-prem.  I see that there are options to collect data via the Office 365 REST API through the Microsoft Office 365 log source type or via syslog (event hubs) through the Microsoft Azure log source type.

    I initially tried the event hubs and was getting data but was having to manually parse every field and the event types.
    I have since switched to the Office 365 REST API and more of the events/data are being parsed, but I am not getting as much detail on events.

     

    Does anybody have experience collecting data from Azure Active Directory and Office 365?  If so, which log source type(s) are you leveraging to collect the data?

    Seems like the Office 365 REST API option is the best fit (includes Azure Active Directory, Exchange, SharePoint, General, DLP, Service Communications), but it doesn't seem to get as much detail as the event hub option.

     

    Appreciate any experience others can share.

     

    Thanks!

    Tim



    ------------------------------
    Tim
    ------------------------------


  • 2.  RE: Azure Active Directory and Office 365 Logging

    Posted Wed September 11, 2019 03:39 AM
    hey man,

    how did you collect logs from event hubs? i have 4 customer that the event hub dont work to monitor the azure.

    about the OFFICE 365 we use Office 365 REST API its the best way to monitor office 365 mails if you need.

    ------------------------------
    nati nakache
    ------------------------------



  • 3.  RE: Azure Active Directory and Office 365 Logging

    Posted Wed September 11, 2019 07:25 AM

    We got things working with the event hubs by following the documentation on the QRadar support site (link below).  I had to configure a log source, but then another was auto-discovered that had our data being sent from the event hub.  We are only using the event hub for Azure Active Directory data.

    https://www.ibm.com/support/knowledgecenter/en/SS42VS_DSM/com.ibm.dsm.doc/c_dsm_guide_microsoft_azure_overview.html?cp=SS42VS_7.3.2



    ------------------------------
    T
    ------------------------------



  • 4.  RE: Azure Active Directory and Office 365 Logging

    Posted Mon September 23, 2019 09:22 AM
    Hi - we also recently configured an Event Hub - we created the log source as you described, and then a second log source was auto-discovered, and that is the one that sends events back from the hub. The problem is that the protocol is auto-selected as syslog, whereas we selected the EVENT HUB protocol - which is supposed to be more detailed - when we configured the original. Now we don't know how to get the event hub events to be parsed as the event hub protocol.

    ------------------------------
    Amir Perlson
    ------------------------------



  • 5.  RE: Azure Active Directory and Office 365 Logging

    Posted Tue September 24, 2019 11:46 AM
    Hi All,

    This is currently broken.   I'm working w/ IBM Qradar Support Engineers and our Microsoft PFE.  
    We have Office365/AzureAD built and sending events, but not all of their events come via the Office365 DSM, some come via the Azure DSM.
    About 2-3 weeks ago - we had a shared web-session with Qradar Support, Microsoft PFEs, our Microsoft Cloud team, and myself.
    We followed/built the article that sent AzureAD/Office365 Security events via the Graph Security API via Azure Event Hubs to Qradar (On-Prem).   It would connect but then fail within a half-hour. 
    Then a few days later - this article was deleted from the M$ Doco website.   With a lil' bit of detective work - we found the github site that confirmed this.
    Via our M$ PFE - we've contacted Microsoft Development - they relayed that their IBMQradar-MicrosoftCloud Devs will be releasing a new DSM later this year.   

    It's crazy how there are two pipes to connect Qradar to Microsoft Cloud - but multiple places in Microsoft Cloud to enable feeding that data into those two pipes.  And it's definitely not logical about which feed goes to which pipe.

    FYI...
    Troy

    ------------------------------
    Troy Barnhart
    ------------------------------



  • 6.  RE: Azure Active Directory and Office 365 Logging

    Posted Wed September 25, 2019 01:09 PM
    Interested to see how this works out Troy.  Please update this thread with info as it progresses.  Thanks for sharing.  I'm not alone!

    ------------------------------
    Chris Wilhelm
    ------------------------------



  • 7.  RE: Azure Active Directory and Office 365 Logging

    Posted Tue September 24, 2019 11:50 AM
    It's currently broken.   A new DSM is supposedly on the way around the New Year.

    There are two DSM's - Off365 and Azure.   Think of it as "two pipes" to get M$ Cloud events to On-prem Qradar.   But you need to configured M$ Cloud in multiple places to feed data into those two pipes.  Further, complicating it is that AzureAD is in Office365, not Azure, but to configure AzureAD you go thru the Azure portal.   The majority of AzureAD events flow thru the Office365 DSM, but some Security Events flow thru Azure Events Hubs from the Graph API.  

    We were working on this three weeks ago - specifically the Graph API Security events doco - with IBM Support, our MSSP, our M$ PFE, our Azure/Off365 team and myself in a Webex session - flipping between us to get this configured.
    It would initially connect successfully, but then fail.
    The following week after this webex session - the M$ doco site we used was deleted.   With a bit of detective work - we found the Github site where M$ Doco teams works.  And that page was deleted.    The M$ PFEs' were totally surprised.    Lots of Qradar/M$Cloud customers are hanging at this time...

    We've been pushing up our IBM Support Team and our M$ Premier Team to get this fixed sooner...


    ------------------------------
    Troy Barnhart
    ------------------------------



  • 8.  RE: Azure Active Directory and Office 365 Logging

    Posted Wed September 25, 2019 04:06 PM
    Some of the event hub integration must work, our test environment is still actively receiving events from the event hub.  Today I was trying to migrate that to production and things appeared to work (pulled the cert file to my QRadar box), but I am just sitting with a status of "Completed initialization".  In my test environment, I have a status of "Connected.  Collecting events."

    Are you getting anything into QRadar from your event hub?

    If I am unable to get our event hub data going to our production box, I plan on opening a support case.  Hopefully help push the fix to happen sooner.

    ------------------------------
    T
    ------------------------------



  • 9.  RE: Azure Active Directory and Office 365 Logging

    Posted Wed September 25, 2019 04:32 PM
    Our issue was only for feeding those Graph API Security events via Event Hubs.
    Even two PFE's were caught offguard by the article deletion.

    https://docs.microsoft.com/en-us/graph/security-siemintegration < Qradar was removed from this one.

    https://docs.microsoft.com/en-us/graph/security-qradar-siemintegration  < This one was Deleted.


    I found this one still there afterwards.
    https://developer.microsoft.com/en-us/graph/graph/docs/concepts/security-qradar-siemintegration



    ------------------------------
    Troy Barnhart
    ------------------------------



  • 10.  RE: Azure Active Directory and Office 365 Logging

    Posted Tue October 08, 2019 09:21 AM
    Edited by Sophia McCarthy Tue October 08, 2019 09:22 AM
    Hi,

    Azure Event Hubs and Office 365 are individual integrations. You will need a Microsoft Azure DSM and an Azure Event Hub protocol to ingest the following events (Network Security Group (NSG) Flow logs, Network Security Group (NSG) Logs, Authorization, Classic Compute, Classic Storage, Compute, Insights, KeyVault, SQL, Storage, Automation, Cache, CDN, Devices, Event Hub, HDInsight, Recovery Services, AppService, Batch, Bing Maps, Certificate Registration, Cognitive Services, Container Service, Content Moderator, Data Catalog, Data Factory, Data Lake Analytics, Data Lake Store, Domain Registration, Dynamics LCS, Features, Logic, Media, Notification Hubs, Search, Servicebus, Support, Web, Scheduler, Resources, Resource Health, Operation Insights, Market Place Ordering, API Management, AD Hybrid Health Service, Server Management) into QRadar (https://www.ibm.com/support/knowledgecenter/SS42VS_DSM/com.ibm.dsm.doc/t_dsm_guide_microsoft_azure_enable_event_hubs.html?cp=SS42VS_7.3.1)

    As for Office 365, we ingest data from the following content types (Azure Active Directory, Exchange, SharePoint, General, DLP, Service Communications), to which you will need Microsoft Office 365 DSM and Office 365 REST API Protocol and can get the following events if you have the subscriptions to those content types on the Office 365 side (https://docs.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-schema). QRadar's configuration is here, (https://www.ibm.com/support/knowledgecenter/SS42VS_DSM/com.ibm.dsm.doc/c_dsm_guide_microsoft_office_365_overview.html?cp=SS42VS_7.3.1)

    Azure Event Hub is a gateway protocol. Which means, it depends on what services you have enabled in the Azure Portal. If you have Linux /PaloAlto / F5 Networks / CheckPoint data stored in your Azure Event Hub, QRadar would ingest that data and automatically create a log source using our Traffic Analysis engine to determine which event payload belongs to which DSM. If you create an Azure Event Hub log source, you can see that the protocol doesn't actually produce events, it's allowing events to come in and get autodetected as an existing log source, or if it's an event type that isn't recognized by QRadar, it will be unknown or stored to which you can create a uDSM using the DSM Editor to parse and normalize the events.

    If you're looking for native Azure Active Directory events from the Azure Event Hub integration, this is currently in development and targeted for release by the end of the year.

    For Microsoft Graph Security API, this is essentially similar but more refined to which Microsoft products are natively parsed. For Microsoft Graph Security, it can ingest data from Security Center, MCAS, Office 365 ATP, Insight, Azure ATP, WDATP, Azure Identity Protection, Azure Information Protection. With QRadar, we will need to create / update existing DSM to parse the events that would come from Microsoft Graph Security API. In the first release of Microsoft Graph Security API, we will be releasing a DSM for Security Center which is targeted for end of this year.

    We will not be integrating Microsoft Graph Security API with Azure Event Hubs, and they are 2 Gateway protocols, to which we would support separately.

    ------------------------------
    Sophia McCarthy
    QRadar Offering Manager
    IBM Security
    ------------------------------



  • 11.  RE: Azure Active Directory and Office 365 Logging

    Posted Tue January 07, 2020 01:25 PM

    Hi Troy,

     

    Here's the link to the knowledge center for Azure Active Directory;

    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ibm.com%2Fsupport%2Fknowledgecenter%2FSS42VS_DSM%2Fcom.ibm.dsm.doc%2Fc_dsm_guide_ms_azure_active_directory_overview.html%3Fcp%3DSS42VS_7.3.1&amp;data=02%7C01%7CLiz.Tesch%40microsoft.com%7Cc2c1c2db062d45b05d1908d792dc3809%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637139347348459121&amp;sdata=m%2FlAJHU4fEXQecrMy4d2SMmAAtmGVjDk%2FBzgwV15cfM%3D&amp;reserved=0

     

    Let me know if you're having issues.

     

    Thank you,

     

    Sophia Sampath



    ------------------------------
    Troy Barnhart
    ------------------------------



  • 12.  RE: Azure Active Directory and Office 365 Logging

    Posted Thu April 23, 2020 10:30 AM
    Hi Sophia
    I was just wondering if there has been any progress on Microsoft Graph Secure API integration with QRadar?

    ------------------------------
    Viorel Chicu
    ------------------------------



  • 13.  RE: Azure Active Directory and Office 365 Logging

    Posted Thu April 23, 2020 11:39 AM

    +1 to VC's question about M$ Graph Security API.

    We do have the Office365 and AzureAD configured and working successfully.



    ------------------------------
    Troy Barnhart
    ------------------------------



  • 14.  RE: Azure Active Directory and Office 365 Logging

    Posted Wed May 06, 2020 12:34 PM
    Hi Sophia,

    Received this alert today.   It discusses how to use a Universal DSM for M$ Graph Security API.   Are there any updates on this new DSM?

    1. IBM QRadar SIEM: Troubleshooting

      - TITLE: QRadar: Microsoft Graph Security API error 400: 'Invalid ODATA query filter'
      - URL: https://www.ibm.com/support/pages/node/6204112?myns=swgother&mynp=OCSSBQAC&mync=E&cm_sp=swgother-_-OCSSBQAC-_-E
      - ABSTRACT:

    Microsoft™ Graph Security API protocol connections do not receive events and the warning message in the Log Source Management app test tool reports: 'Error received from Microsoft Graph Security API HTTP status Not OK. Status code is 400. Error Description: 'Invalid ODATA query filter'



    - TITLE: QRadar: Microsoft Graph Security API error - 'HTTP status not ok. Status code is 206.'
    - URL: https://www.ibm.com/support/pages/node/6204097?myns=swgother&mynp=OCSSBQAC&mync=E&cm_sp=swgother-_-OCSSBQAC-_-E
    - ABSTRACT:

    Microsoft™ Graph Security API log sources do not receive events and the protocol test tool lists the following: 'Error received from Microsoft Graph Security API HTTP status Not OK. Status code is 206.'



    ------------------------------
    Troy Barnhart
    ------------------------------



  • 15.  RE: Azure Active Directory and Office 365 Logging

    Posted Fri September 25, 2020 05:49 PM
    Hi Sophia,

    I am trying to add MS O365 in to QRadar. I want to know if QRadar will use polling to collect events from O365, means will it use pull mechanism ?
    I need to allow rules on firewall to communicate between O365 and QRadar EC as firewall is there in between. I was thinking what will be the source and destination ?
    Will source be Qradar EC ? 
    Please let me know.

    Thanks,
    Akshay Bahade

    ------------------------------
    Akshay Bahade
    ------------------------------



  • 16.  RE: Azure Active Directory and Office 365 Logging

    Posted Sat September 26, 2020 03:14 PM
    Hello Akshay,

    There is a lot of this online from Microsoft already and you should read about it first.

    It will poll as it the O365 API is REST based. A search has this that explains and from this you can work out the traffic types:

    https://docs.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges?view=o365-worldwide

    MCAS is a different animal and I have posted about this elsewhere on the forum.

    Three things:

    1. I highly recommend setting up the Azure end and then running some wget/curl tests to familiarise yourself with the mode of operation.
    2. Spend some time researching - this will benefit in the long run. 
    3. Don't expect O365 event traffic in real time. I've provided benchmarks on latency before (minutes to days delayed) and provided an O365 latency monitoring dashboard on github.

    Darren H.

    ------------------------------
    Darren H.
    ------------------------------