I am using version 2.0.1 of fn_utilities and when I launch the Expand URL example function I get an 104 error.

2020-10-23 11:06:32,059 INFO [utilities_expand_url] resilient_url: https://tinyurl.com/m3q2xt
2020-10-23 11:06:32,059 INFO [decorators] [utilities_expand_url] StatusMessage: Starting...
2020-10-23 11:06:32,059 DEBUG [utilities_expand_url] https://tinyurl.com/m3q2xt depth 1
2020-10-23 11:06:32,061 DEBUG [stomp_component] send()
2020-10-23 11:06:32,061 DEBUG [client] Sending SEND frame [headers={'correlation-id': 'invid:167601', 'destination': '/queue/acks.201.fn_utilities'}, body=b'{"message_type": 0, ...', version=1.2]
2020-10-23 11:06:32,062 DEBUG [stomp_component] Message sent
2020-10-23 11:06:32,072 ERROR [utilities_expand_url] [Errno 104] Connection reset by peer
2020-10-23 11:06:32,073 DEBUG [utilities_expand_url] []
2020-10-23 11:06:32,073 DEBUG [decorators] [utilities_expand_url] FunctionResult: <resilient_circuits.action_message.FunctionResult object at 0x7ff5e6f3e860>
2020-10-23 11:06:32,466 DEBUG [actions_component] success! [<resilient_circuits.action_message.FunctionResult object at 0x7ff5e6f3e860>], <utilities_expand_url[functions.utilities_expand_url] (id=17, workflow=utilities_expand_url, user=user@email.com)

I saw in the changelog that between 2.0.0 and 2.0.1 something changed at right for Expand URL function.

Version 2.0.1 -proxy access added to url expand and get ssl certificate functions

Any suggestions ?

Thanks

------------------------------
Lucian Sipos
------------------------------

Hi Lucian

it looks like you had connection issues to your resilient server see the 'stomp' and 'reset by peer messages'.

Are you on an integration server or app host ?
What version of resilient and resilient circuits are you using ?
Do you get this message continuously with this integration or are you also experiencing with other integrations?
Are you using  proxies for fn_utilities in your app.config ?

John

------------------------------
John Quirke
------------------------------

Hi John

Some other Example functions works well, I don't know of connections issues.

My environment is composed as following: Integration server, Resilient 36.2 and resilient-circuits 35. I experience this issue with current integration only.
No proxies are being used.

Thanks

------------------------------
Lucian Sipos
------------------------------

Original Message:
Sent: Thu November 05, 2020 05:35 AM
From: John Quirke
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian

it looks like you had connection issues to your resilient server see the 'stomp' and 'reset by peer messages'.

Are you on an integration server or app host ?
What version of resilient and resilient circuits are you using ?
Do you get this message continuously with this integration or are you also experiencing with other integrations?
Are you using  proxies for fn_utilities in your app.config ?

John

------------------------------
John Quirke
------------------------------

Hi Lucian

Was this a new install of fn_utilities ? or did you upgrade to the latest version ?

Are you on a remote integration  server ?  if so what version of python are you on?

Can you share your fn_utilities section from app.config ?

Are all other fn_utilities functions working ? are there other integrations installed ?

John

------------------------------
John Quirke
------------------------------

Original Message:
Sent: Thu November 05, 2020 09:29 AM
From: Lucian Sipos
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi John

Some other Example functions works well, I don't know of connections issues.

My environment is composed as following: Integration server, Resilient 36.2 and resilient-circuits 35. I experience this issue with current integration only.
No proxies are being used.

Thanks

------------------------------
Lucian Sipos
------------------------------


Original Message:
Sent: Thu November 05, 2020 05:35 AM
From: John Quirke
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian

it looks like you had connection issues to your resilient server see the 'stomp' and 'reset by peer messages'.

Are you on an integration server or app host ?
What version of resilient and resilient circuits are you using ?
Do you get this message continuously with this integration or are you also experiencing with other integrations?
Are you using  proxies for fn_utilities in your app.config ?

John

------------------------------
John Quirke

I did an uninstall/install of fn_utilities. Python version is 3.6.

Other fn_utilities functions work.

This is fn_utilities section:

[fn_utilities]

# For safety, shell_command parameter values are escaped - set to 'sh' (bash) or 'ps' (powershell)
shell_escaping=sh

# NOTE: For safety, you *must* enclose shell-param substitutions in double-quotes.
# The values of these parameters usually includes artifacts or other untrusted data
# that may contain spaces, dashes and other content.

# accepted remote powershell extensions in a comma separated list, example: ps1, psm1, etc
remote_powershell_extensions=ps1

# remote auth transport one of [ntlm, basic]
remote_auth_transport=ntlm

# remote computers
remote_computer=(username:password@server)

# remote shell commands
remote_command=[remote path to script]

# local shell_command default commands (unix)
nslookup=nslookup "{{shell_param1}}"
dig=dig "{{shell_param1}}"
traceroute=traceroute -m 15 "{{shell_param1}}"
whois=whois "{{shell_param1}}"

[some_other_shell_scripts]

# more shell_command examples:
# foo=bash $UTILBIN/foo "{{shell_param1}}"

# on windows, powershell example:
# psinfo=PsInfo.exe -accepteula -nobanner \{{shell_param1}} | ConvertTo-Json

# more shell_command examples: Volatility.
# First param is filename of the memory image, assuming $VOLATILITY_LOCATION is set
# Second param is the profile ("Win7SP0x64" etc)
# imageinfo=python /path/to/vol.py -f "{{shell_param1}}" imageinfo --output=json
# kdbgscan=python /path/to/vol.py -f "{{shell_param1}}" "--profile={{shell_param2}}" kdbgscan --output=json
# psscan=python /path/to/vol.py -f "{{shell_param1}}" "--profile={{shell_param2}}" psscan --output=json
# dlllist=python /path/to/vol.py -f "{{shell_param1}}" "--profile={{shell_param2}}" dlllist --output=json
# (etc)

# directory of xml stylesheets to use for xml transformations
# xml_stylesheet_dir=

max_timer=30d

No other integrations installed. What do you mean by "remote integration server" ? I access it from home.

------------------------------
Lucian Sipos
------------------------------

Original Message:
Sent: Thu November 05, 2020 10:28 AM
From: John Quirke
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian

Was this a new install of fn_utilities ? or did you upgrade to the latest version ?

Are you on a remote integration  server ?  if so what version of python are you on?

Can you share your fn_utilities section from app.config ?

Are all other fn_utilities functions working ? are there other integrations installed ?

John

------------------------------
John Quirke
------------------------------


Original Message:
Sent: Thu November 05, 2020 09:29 AM
From: Lucian Sipos
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi John

Some other Example functions works well, I don't know of connections issues.

My environment is composed as following: Integration server, Resilient 36.2 and resilient-circuits 35. I experience this issue with current integration only.
No proxies are being used.

Thanks

------------------------------
Lucian Sipos


Original Message:
Sent: Thu November 05, 2020 05:35 AM
From: John Quirke
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian

it looks like you had connection issues to your resilient server see the 'stomp' and 'reset by peer messages'.

Are you on an integration server or app host ?
What version of resilient and resilient circuits are you using ?
Do you get this message continuously with this integration or are you also experiencing with other integrations?
Are you using  proxies for fn_utilities in your app.config ?

John

------------------------------
John Quirke

Hi Lucian, when I say remote ... I meant that your integration server doesn't coexist on the resilient server (which is not supported) and it is a separate server
I have shared this issue with the wider integration team and awaiting feedback.
John

------------------------------
John Quirke
------------------------------

Original Message:
Sent: Thu November 05, 2020 11:08 AM
From: Lucian Sipos
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

I did an uninstall/install of fn_utilities. Python version is 3.6.

Other fn_utilities functions work.

This is fn_utilities section:

[fn_utilities]

# For safety, shell_command parameter values are escaped - set to 'sh' (bash) or 'ps' (powershell)
shell_escaping=sh

# NOTE: For safety, you *must* enclose shell-param substitutions in double-quotes.
# The values of these parameters usually includes artifacts or other untrusted data
# that may contain spaces, dashes and other content.

# accepted remote powershell extensions in a comma separated list, example: ps1, psm1, etc
remote_powershell_extensions=ps1

# remote auth transport one of [ntlm, basic]
remote_auth_transport=ntlm

# remote computers
remote_computer=(username:password@server)

# remote shell commands
remote_command=[remote path to script]

# local shell_command default commands (unix)
nslookup=nslookup "{{shell_param1}}"
dig=dig "{{shell_param1}}"
traceroute=traceroute -m 15 "{{shell_param1}}"
whois=whois "{{shell_param1}}"

[some_other_shell_scripts]

# more shell_command examples:
# foo=bash $UTILBIN/foo "{{shell_param1}}"

# on windows, powershell example:
# psinfo=PsInfo.exe -accepteula -nobanner \{{shell_param1}} | ConvertTo-Json

# more shell_command examples: Volatility.
# First param is filename of the memory image, assuming $VOLATILITY_LOCATION is set
# Second param is the profile ("Win7SP0x64" etc)
# imageinfo=python /path/to/vol.py -f "{{shell_param1}}" imageinfo --output=json
# kdbgscan=python /path/to/vol.py -f "{{shell_param1}}" "--profile={{shell_param2}}" kdbgscan --output=json
# psscan=python /path/to/vol.py -f "{{shell_param1}}" "--profile={{shell_param2}}" psscan --output=json
# dlllist=python /path/to/vol.py -f "{{shell_param1}}" "--profile={{shell_param2}}" dlllist --output=json
# (etc)

# directory of xml stylesheets to use for xml transformations
# xml_stylesheet_dir=

max_timer=30d

No other integrations installed. What do you mean by "remote integration server" ? I access it from home.

------------------------------
Lucian Sipos
------------------------------


Original Message:
Sent: Thu November 05, 2020 10:28 AM
From: John Quirke
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian

Was this a new install of fn_utilities ? or did you upgrade to the latest version ?

Are you on a remote integration  server ?  if so what version of python are you on?

Can you share your fn_utilities section from app.config ?

Are all other fn_utilities functions working ? are there other integrations installed ?

John

------------------------------
John Quirke


Original Message:
Sent: Thu November 05, 2020 09:29 AM
From: Lucian Sipos
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi John

Some other Example functions works well, I don't know of connections issues.

My environment is composed as following: Integration server, Resilient 36.2 and resilient-circuits 35. I experience this issue with current integration only.
No proxies are being used.

Thanks

------------------------------
Lucian Sipos


Original Message:
Sent: Thu November 05, 2020 05:35 AM
From: John Quirke
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian

it looks like you had connection issues to your resilient server see the 'stomp' and 'reset by peer messages'.

Are you on an integration server or app host ?
What version of resilient and resilient circuits are you using ?
Do you get this message continuously with this integration or are you also experiencing with other integrations?
Are you using  proxies for fn_utilities in your app.config ?

John

------------------------------
John Quirke

Hi Lucian

Is your postgres database configured on the app host too ?

Could you configure it on another server and retry ...

John

------------------------------
John Quirke
------------------------------

Original Message:
Sent: Thu November 05, 2020 10:28 AM
From: John Quirke
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian

Was this a new install of fn_utilities ? or did you upgrade to the latest version ?

Are you on a remote integration  server ?  if so what version of python are you on?

Can you share your fn_utilities section from app.config ?

Are all other fn_utilities functions working ? are there other integrations installed ?

John

------------------------------
John Quirke
------------------------------


Original Message:
Sent: Thu November 05, 2020 09:29 AM
From: Lucian Sipos
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi John

Some other Example functions works well, I don't know of connections issues.

My environment is composed as following: Integration server, Resilient 36.2 and resilient-circuits 35. I experience this issue with current integration only.
No proxies are being used.

Thanks

------------------------------
Lucian Sipos


Original Message:
Sent: Thu November 05, 2020 05:35 AM
From: John Quirke
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian

it looks like you had connection issues to your resilient server see the 'stomp' and 'reset by peer messages'.

Are you on an integration server or app host ?
What version of resilient and resilient circuits are you using ?
Do you get this message continuously with this integration or are you also experiencing with other integrations?
Are you using  proxies for fn_utilities in your app.config ?

John

------------------------------
John Quirke

Hi Lucian,

The changes in 2.0.1 just added proxy support for expand_url. There should be no changes when using this function without proxy settings in place.

I ran this url in my environment and received the following result:

2020-11-05 14:37:28,089 DEBUG [actions_component] Result: {'urllist': ['http://en.wikipedia.org/wiki/URL_shortening', 'https://en.wikipedia.org/wiki/URL_shortening']}

Do you have a log line with "Result:" to review?

------------------------------
Mark Scherfling
------------------------------

@John Quirke database have nothing to do with AppHost for now.​

@Mark Scherfling In note I printed the results and that's what I have: {u'urllist': []}
​​
Log is the same as one in the first post. Hmmm, so strange..

------------------------------
Lucian Sipos
------------------------------

Original Message:
Sent: Thu November 05, 2020 02:43 PM
From: Mark Scherfling
Subject: fn_utilities: Example Expand URL error: Connection reset by peer

Hi Lucian,

The changes in 2.0.1 just added proxy support for expand_url. There should be no changes when using this function without proxy settings in place.

I ran this url in my environment and received the following result:

2020-11-05 14:37:28,089 DEBUG [actions_component] Result: {'urllist': ['http://en.wikipedia.org/wiki/URL_shortening', 'https://en.wikipedia.org/wiki/URL_shortening']}

Do you have a log line with "Result:" to review?

------------------------------
Mark Scherfling
------------------------------