IBM Security QRadar SOAR

 View Only
Expand all | Collapse all

Issue With fn_microsoft_security_graph Integration / resilient-lib OAuth2ClientCredentialsSession

  • 1.  Issue With fn_microsoft_security_graph Integration / resilient-lib OAuth2ClientCredentialsSession

    IBM Champion
    Posted Thu April 22, 2021 10:42 AM
    Edited by Liam Mahoney Thu April 22, 2021 10:44 AM
    Hello,

    We've been trying to setup the `fn_microsoft_security_graph` integration provided on the community exchange for the past couple of weeks.

    We ran into an issue of the integration not connection to our GCC High instance of Azure, and found it was due to the `DEFAULT_SCOPE` needing to be a different URL than the one coded into the integration.

    Once we changed that URL, the integration started pulling in alerts properly. However, after ~1 hour of running the integration would error out with the following stack trace:

    Traceback (most recent call last):
    File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
    File "/usr/lib64/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
    File "/opt/app-root/lib/python3.6/site-packages/fn_microsoft_security_graph/components/microsoft_security_graph_alerts_integrations.py", line 198, in msg_polling_thread
    alert_list = get_alerts(self.options, self.Microsoft_security_graph_helper)
    File "/opt/app-root/lib/python3.6/site-packages/fn_microsoft_security_graph/components/microsoft_security_graph_alerts_integrations.py", line 290, in get_alerts
    r = ms_graph_helper.ms_graph_session.get(url)
    File "/opt/app-root/lib/python3.6/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 140, in request
    self.update_token()
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 126, in update_token
    self.client_secret, self.scope, self.proxies)
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 86, in authenticate
    r = self.get_token(token_url, client_id, client_secret, scope, proxies)
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 118, in get_token
    return self.post(token_url, data=post_data, proxies=proxies)
    File "/opt/app-root/lib/python3.6/site-packages/requests/sessions.py", line 590, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 140, in request
    self.update_token()

    ... (the same request -> update_token -> authenticate -> get_token -> post lines were repeated many times)

    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 140, in request
    self.update_token()
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 126, in update_token
    self.client_secret, self.scope, self.proxies)
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 86, in authenticate
    r = self.get_token(token_url, client_id, client_secret, scope, proxies)
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 118, in get_token
    return self.post(token_url, data=post_data, proxies=proxies)
    File "/opt/app-root/lib/python3.6/site-packages/requests/sessions.py", line 590, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
    File "/opt/app-root/lib/python3.6/site-packages/resilient_lib/components/oauth2_client_credentials_session.py", line 139, in request
    if self.expiration_time < time.time():
    RecursionError: maximum recursion depth exceeded in comparison


    After some digging into / debugging the integration code, as well as the resilient-lib `OAuth2ClientCredentialsSession` code (and the `requests.Session` code, since `OAuth2ClientCredentialsSession` inherits from `requests.Session`) it appears to me that there's an issue with `resilient_lib's `OAuth2ClientCredentialsSession`.

    The error happens an hour after running as that's when the initial bearer token we get expires. The next time we try to get graph alerts after the bearer token has expired, the `request` method in `OAuth2ClientCredentialsSession` identifies that the bearer token is expired and that a new one needs to be retrieved, however this functionality appears to be broken.

    Below is the process for retrieving a new bearer token:

    1. `OAuth2ClientCredentialsSession.update_token` is called
    2. `OAuth2ClientCredentialsSession.authenticate` is called
    3. `OAuth2ClientCredentialsSession.get_token` is called
    4. `requests.Session.post` is called (through inheritance) - this method calls `self.request` which `OAuth2ClientCredentialsSession` has overwritten
    5. `OAuth2ClientCredentialsSession.request` is called - Here's the issue, this method identifies that the bearer token is expired, and starts the process to retrieve a new bearer token again
    6. `OAuth2ClientCredentialsSession.update_token` is called and the process is repeated until the maximum recursion depth is reached

    So the request to renew the bearer token is never sent, and there ends up being an infinite recursion loop.

    I'm wondering if someone can confirm that this is happening for them as well? 

    Thanks,

    Liam


    ------------------------------
    Liam Mahoney
    ------------------------------


  • 2.  RE: Issue With fn_microsoft_security_graph Integration / resilient-lib OAuth2ClientCredentialsSession

    Posted Wed April 28, 2021 08:53 AM
    Hi Liam,

    Thanks for reporting this issue!

    What URL did you use rather than the 'DEFAULT_SCOPE'?

    Are you running in app host environment?  There was a fix in resilient-lib oauth2 code dealing with session inheritance and I want to confirm which version of resilient-lib is in use here?  We need to rebuild the fn_microsoft_security_graph container with the latest verssion of resilient-lib....but it you are experiencing another problem it would be good to fix that at the same time if possible.

    AnnMarie

    ------------------------------
    AnnMarie Norcross
    ------------------------------



  • 3.  RE: Issue With fn_microsoft_security_graph Integration / resilient-lib OAuth2ClientCredentialsSession

    IBM Champion
    Posted Wed April 28, 2021 09:10 AM
    Hi AnnMarie,

    That is great news, thanks for the update!

    I had to replace "https://graph.microsoft.com/.default" with "'https://graph.microsoft.us/.default" (changed .com to .us). I made a PR that has this DEFAULT_SCOPE value specified in the config file here https://github.com/ibmresilient/resilient-community-apps/pull/71. 

    I was initially running the integration in the app host environment but then I undeployed it from there and started running it locally on my laptop so I could debug the issue. In the environment the integration was running on my laptop I have resilient-lib 40.2.1939 installed.

    Please let me know if you need any other information!

    Thanks,




    ------------------------------
    Liam Mahoney
    ------------------------------