IBM QRadar SOAR

The Bluecoat site review function stopped working recently on both our test and production servers.

We saw the following error in app.log: 



We added a line to the function so that we could debug the result. Now we are seeing the following in app.log:

<section class="site-review-content b-content">
<div class="center_section container">
<h1>Site Review Acceptable Use Information</h1>
<p>
It appears you are using Site Review in an automated fashion, which violates our <a href="https://www.symantec.com/about/legal/blue-coat-legal-archive/website-terms-of-use">Terms
of Use</a> and can result in loss of access to the service.
</p>
<p>
Please contact your Symantec representative for other options.


Is anyone else having this issue? Has Symantec recently added a check to the Sitereview site that detects automated requests?

------------------------------
Michael John Sheahan
------------------------------

We got confirmation from Symantec. They have added controls to the site to prevent automatic requests.

------------------------------
Michael John Sheahan
------------------------------

Original Message:
Sent: Thu July 25, 2019 10:37 AM
From: Michael John Sheahan
Subject: fn_bluecoat_site_review error

The Bluecoat site review function stopped working recently on both our test and production servers.

We saw the following error in app.log: 

2019-07-24 14:45:21,596 ERROR [actions_component] <task[functionworker] (<function _call_the_task at 0x7f1eaf28ed70>, <bluecoat_site_review_lookup[functions.bluecoat_site_review_lookup] (id=64, workflow=bluecoat_site_review_search, user=user@example.com) 2019-07-24 14:45:19.114000> artifact_value=u'www.exampe.com')> (<class 'resilient_circuits.action_message.FunctionException_'>): FunctionException_: <Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/fn_bluecoat_site_review/components/bluecoat_site_review_lookup.py", line 49, in _bluecoat_site_review_lookup_function
    response_json = self.sitereview(self.options['url'], artifact_value)
  File "/usr/local/lib/python2.7/site-packages/fn_bluecoat_site_review/components/bluecoat_site_review_lookup.py", line 78, in sitereview
    dict_to_str = json.dumps(xmltodict.parse(result))
  File "/usr/local/lib/python2.7/site-packages/xmltodict.py", line 330, in parse
    parser.Parse(xml_input, True)
ExpatError: mismatched tag: line 10, column 10

We added a line to the function so that we could debug the result. Now we are seeing the following in app.log:

<section class="site-review-content b-content">
<div class="center_section container">
<h1>Site Review Acceptable Use Information</h1>
<p>
It appears you are using Site Review in an automated fashion, which violates our <a href="https://www.symantec.com/about/legal/blue-coat-legal-archive/website-terms-of-use">Terms
of Use</a> and can result in loss of access to the service.
</p>
<p>
Please contact your Symantec representative for other options.


Is anyone else having this issue? Has Symantec recently added a check to the Sitereview site that detects automated requests?

------------------------------
Michael John Sheahan
------------------------------

Was there any push further to determine what other routes can be used?

------------------------------
Bryan Bowie
------------------------------

Original Message:
Sent: Fri July 26, 2019 06:56 AM
From: Michael John Sheahan
Subject: fn_bluecoat_site_review error

We got confirmation from Symantec. They have added controls to the site to prevent automatic requests.

------------------------------
Michael John Sheahan

Original Message:
Sent: Thu July 25, 2019 10:37 AM
From: Michael John Sheahan
Subject: fn_bluecoat_site_review error

The Bluecoat site review function stopped working recently on both our test and production servers.

We saw the following error in app.log: 

2019-07-24 14:45:21,596 ERROR [actions_component] <task[functionworker] (<function _call_the_task at 0x7f1eaf28ed70>, <bluecoat_site_review_lookup[functions.bluecoat_site_review_lookup] (id=64, workflow=bluecoat_site_review_search, user=user@example.com) 2019-07-24 14:45:19.114000> artifact_value=u'www.exampe.com')> (<class 'resilient_circuits.action_message.FunctionException_'>): FunctionException_: <Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/fn_bluecoat_site_review/components/bluecoat_site_review_lookup.py", line 49, in _bluecoat_site_review_lookup_function
    response_json = self.sitereview(self.options['url'], artifact_value)
  File "/usr/local/lib/python2.7/site-packages/fn_bluecoat_site_review/components/bluecoat_site_review_lookup.py", line 78, in sitereview
    dict_to_str = json.dumps(xmltodict.parse(result))
  File "/usr/local/lib/python2.7/site-packages/xmltodict.py", line 330, in parse
    parser.Parse(xml_input, True)
ExpatError: mismatched tag: line 10, column 10

We added a line to the function so that we could debug the result. Now we are seeing the following in app.log:

<section class="site-review-content b-content">
<div class="center_section container">
<h1>Site Review Acceptable Use Information</h1>
<p>
It appears you are using Site Review in an automated fashion, which violates our <a href="https://www.symantec.com/about/legal/blue-coat-legal-archive/website-terms-of-use">Terms
of Use</a> and can result in loss of access to the service.
</p>
<p>
Please contact your Symantec representative for other options.


Is anyone else having this issue? Has Symantec recently added a check to the Sitereview site that detects automated requests?

------------------------------
Michael John Sheahan
------------------------------

Hi,

I suggest you look at other source/integrations like URLScanIO or URLVOID

------------------------------
BENOIT ROSTAGNI
------------------------------

Original Message:
Sent: Thu April 02, 2020 08:56 AM
From: Bryan Bowie
Subject: fn_bluecoat_site_review error

Was there any push further to determine what other routes can be used?

------------------------------
Bryan Bowie

Original Message:
Sent: Fri July 26, 2019 06:56 AM
From: Michael John Sheahan
Subject: fn_bluecoat_site_review error

We got confirmation from Symantec. They have added controls to the site to prevent automatic requests.

------------------------------
Michael John Sheahan

Original Message:
Sent: Thu July 25, 2019 10:37 AM
From: Michael John Sheahan
Subject: fn_bluecoat_site_review error

The Bluecoat site review function stopped working recently on both our test and production servers.

We saw the following error in app.log: 

2019-07-24 14:45:21,596 ERROR [actions_component] <task[functionworker] (<function _call_the_task at 0x7f1eaf28ed70>, <bluecoat_site_review_lookup[functions.bluecoat_site_review_lookup] (id=64, workflow=bluecoat_site_review_search, user=user@example.com) 2019-07-24 14:45:19.114000> artifact_value=u'www.exampe.com')> (<class 'resilient_circuits.action_message.FunctionException_'>): FunctionException_: <Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/fn_bluecoat_site_review/components/bluecoat_site_review_lookup.py", line 49, in _bluecoat_site_review_lookup_function
    response_json = self.sitereview(self.options['url'], artifact_value)
  File "/usr/local/lib/python2.7/site-packages/fn_bluecoat_site_review/components/bluecoat_site_review_lookup.py", line 78, in sitereview
    dict_to_str = json.dumps(xmltodict.parse(result))
  File "/usr/local/lib/python2.7/site-packages/xmltodict.py", line 330, in parse
    parser.Parse(xml_input, True)
ExpatError: mismatched tag: line 10, column 10

We added a line to the function so that we could debug the result. Now we are seeing the following in app.log:

<section class="site-review-content b-content">
<div class="center_section container">
<h1>Site Review Acceptable Use Information</h1>
<p>
It appears you are using Site Review in an automated fashion, which violates our <a href="https://www.symantec.com/about/legal/blue-coat-legal-archive/website-terms-of-use">Terms
of Use</a> and can result in loss of access to the service.
</p>
<p>
Please contact your Symantec representative for other options.


Is anyone else having this issue? Has Symantec recently added a check to the Sitereview site that detects automated requests?

------------------------------
Michael John Sheahan
------------------------------

FYI - It is possible to pull the category from your local ProxySG if you have one.  An HTTP call to it like below will provide you with HTML output containing the site category...

https://[host]:[port]/ContentFilter/TestUrl/google.com/

------------------------------
Brian Mathias
------------------------------

Original Message:
Sent: Thu April 02, 2020 08:56 AM
From: Bryan Bowie
Subject: fn_bluecoat_site_review error

Was there any push further to determine what other routes can be used?

------------------------------
Bryan Bowie

Original Message:
Sent: Fri July 26, 2019 06:56 AM
From: Michael John Sheahan
Subject: fn_bluecoat_site_review error

We got confirmation from Symantec. They have added controls to the site to prevent automatic requests.

------------------------------
Michael John Sheahan

Original Message:
Sent: Thu July 25, 2019 10:37 AM
From: Michael John Sheahan
Subject: fn_bluecoat_site_review error

The Bluecoat site review function stopped working recently on both our test and production servers.

We saw the following error in app.log: 

2019-07-24 14:45:21,596 ERROR [actions_component] <task[functionworker] (<function _call_the_task at 0x7f1eaf28ed70>, <bluecoat_site_review_lookup[functions.bluecoat_site_review_lookup] (id=64, workflow=bluecoat_site_review_search, user=user@example.com) 2019-07-24 14:45:19.114000> artifact_value=u'www.exampe.com')> (<class 'resilient_circuits.action_message.FunctionException_'>): FunctionException_: <Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/fn_bluecoat_site_review/components/bluecoat_site_review_lookup.py", line 49, in _bluecoat_site_review_lookup_function
    response_json = self.sitereview(self.options['url'], artifact_value)
  File "/usr/local/lib/python2.7/site-packages/fn_bluecoat_site_review/components/bluecoat_site_review_lookup.py", line 78, in sitereview
    dict_to_str = json.dumps(xmltodict.parse(result))
  File "/usr/local/lib/python2.7/site-packages/xmltodict.py", line 330, in parse
    parser.Parse(xml_input, True)
ExpatError: mismatched tag: line 10, column 10

We added a line to the function so that we could debug the result. Now we are seeing the following in app.log:

<section class="site-review-content b-content">
<div class="center_section container">
<h1>Site Review Acceptable Use Information</h1>
<p>
It appears you are using Site Review in an automated fashion, which violates our <a href="https://www.symantec.com/about/legal/blue-coat-legal-archive/website-terms-of-use">Terms
of Use</a> and can result in loss of access to the service.
</p>
<p>
Please contact your Symantec representative for other options.


Is anyone else having this issue? Has Symantec recently added a check to the Sitereview site that detects automated requests?

------------------------------
Michael John Sheahan
------------------------------