Hello Patrick,
I have been installed 46.2.19 and still have the issue, but now I see more detailed error:
An error occurred while processing the action acknowledgement. Additional information: Playbook 'IP Address Enrichment' running on Artifact '1.1.1.1' failed. Reason below. Script condition from Condition Point 'Condition point' on flow 'True'. Reason below. None: Script timeout occurred (expected completion no later than 2022-10-31T09:27:00.037Z).
09:14:59.865 [qtp-894207914-3993] WARN v=unknown c.r.s.rest.ExceptionMapperBase - Timeout while running script
com.resilient.scripting.exception.TimeoutScriptingException: Script timeout error
at com.resilient.scripting.cpython.process.CPythonProcessCommsHandler.makeTimeoutException(CPythonProcessCommsHandler.java:243)
at com.resilient.scripting.cpython.process.CPythonProcessCommsHandler.getResult(CPythonProcessCommsHandler.java:187)
at com.resilient.scripting.cpython.process.CPythonProcess.getResult(CPythonProcess.java:46)
at com.resilient.scripting.cpython.CPythonScriptingService.execute(CPythonScriptingService.java:95)
at com.resilient.scripting.rest.PythonREST.execute(PythonREST.java:65)
at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.atmosphere.util.AtmosphereFilterChain.doFilter(AtmosphereFilterChain.java:130)
at org.atmosphere.util.AtmosphereFilterChain.invokeFilterChain(AtmosphereFilterChain.java:95)
at org.atmosphere.handler.ReflectorServletProcessor$FilterChainServletWrapper.service(ReflectorServletProcessor.java:336)
at org.atmosphere.handler.ReflectorServletProcessor.onRequest(ReflectorServletProcessor.java:174)
at org.atmosphere.cpr.AsynchronousProcessor.action(AsynchronousProcessor.java:217)
at org.atmosphere.cpr.AsynchronousProcessor.suspended(AsynchronousProcessor.java:103)
at org.atmosphere.container.Servlet30CometSupport.service(Servlet30CometSupport.java:67)
at org.atmosphere.container.AbstractJetty9AsyncSupportWithWebSocket.service(AbstractJetty9AsyncSupportWithWebSocket.java:161)
at org.atmosphere.container.Jetty93AsyncSupportWithWebSocket.service(Jetty93AsyncSupportWithWebSocket.java:30)
at org.atmosphere.cpr.AtmosphereFramework.doCometSupport(AtmosphereFramework.java:2293)
at org.atmosphere.cpr.AtmosphereServlet.doPost(AtmosphereServlet.java:200)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.co3.json.serialize.OutputFormatServletFilter.doFilter(OutputFormatServletFilter.java:132)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.resilient.scripting.rest.TimingFilter.doFilter(TimingFilter.java:47)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.lang.Thread.run(Thread.java:826)
Caused by: com.resilient.scripting.exception.ScriptTimeoutPyException: None: Script timeout occurred (expected completion no later than 2022-10-31T09:14:44.864Z)
at com.resilient.scripting.exception.ScriptTimeoutPyException.<init>(ScriptTimeoutPyException.java:52)
at com.resilient.scripting.cpython.process.CPythonProcessCommsHandler.makeTimeoutException(CPythonProcessCommsHandler.java:239)
... 75 common frames omitted
09:15:00.022 [Thread-2339] ERROR v=unknown c.r.s.c.p.CPythonProcessCommsHandler - Error while processing script output
java.lang.IllegalStateException: An error occurred while performing a privileged execution
at com.resilient.scripting.security.impl.PrivilegedExecutionServiceImpl.lambda$executePrivileged$0(PrivilegedExecutionServiceImpl.java:49)
at java.security.AccessController.doPrivileged(AccessController.java:691)
at com.resilient.scripting.security.impl.PrivilegedExecutionServiceImpl.executePrivileged(PrivilegedExecutionServiceImpl.java:44)
at com.resilient.scripting.event.impl.DefaultEventHandler.send(DefaultEventHandler.java:66)
at com.resilient.scripting.event.LazyLoader.executeFunction(LazyLoader.java:52)
at com.resilient.scripting.cpython.process.CPythonCommunicator.sendEmailBodyHtml(CPythonCommunicator.java:78)
at com.resilient.scripting.cpython.process.CPythonProcessCommsHandler.handleFrame(CPythonProcessCommsHandler.java:122)
at com.resilient.scripting.cpython.process.CPythonProcessCommsHandler.lambda$handleMessagesFromProcess$0(CPythonProcessCommsHandler.java:143)
at java.lang.Thread.run(Thread.java:826)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error while waiting for results
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:588)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547)
at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:91)
at com.resilient.scripting.security.impl.PrivilegedExecutionServiceImpl.lambda$executePrivileged$0(PrivilegedExecutionServiceImpl.java:47)
... 8 common frames omitted
Caused by: java.lang.RuntimeException: Error while waiting for results
at com.resilient.scripting.event.impl.DefaultEventHandler.lambda$send$1(DefaultEventHandler.java:80)
at com.resilient.scripting.security.PrivilegedCommand.call(PrivilegedCommand.java:74)
at com.resilient.scripting.security.PrivilegedCommand.call(PrivilegedCommand.java:23)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:323)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:69)
at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:36)
at com.resilient.scripting.security.impl.PrivilegedExecutionServiceImpl.lambda$executePrivileged$0(PrivilegedExecutionServiceImpl.java:46)
... 8 common frames omitted
Caused by: java.util.concurrent.TimeoutException: null
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1795)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1939)
at com.resilient.scripting.event.impl.DefaultEventHandler.lambda$send$1(DefaultEventHandler.java:68)
... 18 common frames omitted
Do you know how to fix it?
------------------------------
Gule Fahid
------------------------------
Original Message:
Sent: Thu October 27, 2022 09:26 AM
From: PATRICK MCKENNA
Subject: Issues after upgrade to 46.1.48
Hello!
I believe that you are encountering a problem that was hot-fxed in 46.1.49. Please take a look at IBM Security QRadar SOAR Release Download Locations to download.
-P.J. McKenna
------------------------------
Patrick (PJ) McKenna
SOAR Development
Original Message:
Sent: Wed October 12, 2022 06:59 AM
From: Gule Fahid
Subject: Issues after upgrade to 46.1.48
Some extras related to error "An error occurred while processing the action acknowledgement".
I have errors in /usr/share/co3/logs/client.log like this:
10:37:42.434 [Camel (camel-1) thread #15 - JmsConsumer[acks.%3E]] ERROR [] com.co3.broker.AcknowledgementProcessor - Error processing message: { "message_type": 0, "message": "Completed", "complete": true, "results": { "qrhost": "1.1.1.1", "offenseid": "14828", "current_time": 1665571057000, "assets": [ { "id": "132923", "domain": { "id": "0", "name": "", "__typename": "Domain" }, "hostnames": [ { "id": "76865", "created": "1662232934000", "name": "test", "type": "DNS", "__typename": "AssetHostname" } ], "products": [ { "id": "402454", "productVariantId": 1, "__typename": "AssetProduct" }, { "id": "402455", "productVariantId": null, "__typename": "AssetProduct" }, { "id": "402453", "productVariantId": null, "__typename": "AssetProduct" }, { "id": "402450",
java.lang.IllegalStateException: Unable to commit transaction
at com.co3.jta.JtaTransactionImpl.doCommitTransaction(JtaTransactionImpl.java:205)
at com.co3.jta.JtaTransactionImpl.commitTransaction(JtaTransactionImpl.java:151)
at com.co3.context.DefaultCo3ContextImpl.commitTransaction(DefaultCo3ContextImpl.java:823)
at com.co3.context.DefaultCo3ContextImpl.endTransaction(DefaultCo3ContextImpl.java:1480)
at com.co3.context.DefaultCo3ContextImpl.endTransaction(DefaultCo3ContextImpl.java:1474)
at com.co3.context.DefaultCo3ContextImpl.endTransaction(DefaultCo3ContextImpl.java:1464)
at com.co3.broker.GenericProcessor.establishContextAndProcessMessage(GenericProcessor.java:100)
at com.co3.broker.GenericProcessor.lambda$processMessageAndRetry$3(GenericProcessor.java:86)
at com.co3.broker.GenericProcessor$$Lambda$1298/0x000000001c048ad0.run(Unknown Source)
at net.jodah.failsafe.Functions.lambda$toSupplier$10(Functions.java:240)
at net.jodah.failsafe.Functions$$Lambda$779/0x000000001c0264c0.get(Unknown Source)
at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48)
at net.jodah.failsafe.Functions$$Lambda$789/0x000000000c025cc0.get(Unknown Source)
at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:62)
at net.jodah.failsafe.RetryPolicyExecutor$$Lambda$794/0x00000000f8036f30.get(Unknown Source)
at net.jodah.failsafe.FallbackExecutor.lambda$supply$0(FallbackExecutor.java:43)
at net.jodah.failsafe.FallbackExecutor$$Lambda$798/0x000000000c027eb0.get(Unknown Source)
at net.jodah.failsafe.Execution.executeSync(Execution.java:129)
at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)
at net.jodah.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:210)
at com.co3.broker.GenericProcessor.processMessageAndRetry(GenericProcessor.java:86)
at com.co3.broker.GenericProcessor.lambda$processMessage$4(GenericProcessor.java:219)
at com.co3.broker.GenericProcessor$$Lambda$1293/0x000000001c035fe0.call(Unknown Source)
at com.resilient.guice.ResilientGuiceUtils.lambda$callVoidInRequestScope$0(ResilientGuiceUtils.java:52)
at com.resilient.guice.ResilientGuiceUtils$$Lambda$174/0x00000000d08de9a0.call(Unknown Source)
at com.google.inject.servlet.ServletScopes$4.call(ServletScopes.java:450)
at com.resilient.guice.ResilientGuiceUtils.callInRequestScope(ResilientGuiceUtils.java:73)
at com.resilient.guice.ResilientGuiceUtils.callVoidInRequestScope(ResilientGuiceUtils.java:56)
at com.co3.broker.GenericProcessor.processMessage(GenericProcessor.java:216)
at com.co3.broker.GenericProcessor$GenericProcessorRoute.lambda$configure$0(GenericProcessor.java:206)
at com.co3.broker.GenericProcessor$GenericProcessorRoute$$Lambda$523/0x00000000f90a3e00.process(Unknown Source)
at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:124)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:736)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:696)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:331)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:270)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1237)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1227)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1120)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:830)
Caused by: javax.transaction.RollbackException: The transaction was set to rollback only
at com.atomikos.icatch.jta.TransactionImp.rethrowAsJtaRollbackException(TransactionImp.java:46)
at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:182)
at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:428)
at com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:86)
at com.co3.jta.JtaTransactionImpl.doCommitTransaction(JtaTransactionImpl.java:203)
... 48 common frames omitted
Caused by: com.atomikos.icatch.RollbackException: The transaction was set to rollback only
at com.atomikos.icatch.imp.TransactionStateHandler.commit(TransactionStateHandler.java:215)
at com.atomikos.icatch.imp.CompositeTransactionImp.doCommit(CompositeTransactionImp.java:234)
at com.atomikos.icatch.imp.CompositeTransactionImp.commit(CompositeTransactionImp.java:274)
at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:168)
... 51 common frames omitted
------------------------------
Gule Fahid
Original Message:
Sent: Fri September 30, 2022 02:30 AM
From: Gule Fahid
Subject: Issues after upgrade to 46.1.48
Hello,
After upgrade SOAR from 46.0.8134 to 46.1.48 I got a lot of errors in my playbooks. The errors related to fn_utilites and QRadar Integration apps. The errors happens only if playbook has activation type = Automation. The same playbook with activation type = Manual works flawless. Also my fn_utilites 2.1.1 do no pass selftest and got the error:
fn-utilities:
('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
selftest: failure
selftest output:
{'state': 'failure', 'reason': "('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))"}
Elapsed time: 0.267000 seconds
ERROR: running selftest for App.
Error Code: 1
In client.log I see some errors and warnings like:
04:49:28.604 [Camel (camel-1) thread #11 - JmsConsumer[acks.%3E]] ERROR [] org.hibernate.internal.SessionImpl - HHH000088: Exception in interceptor beforeTransactionCompletion()
06:19:59.441 [Camel (camel-1) thread #14 - JmsConsumer[acks.%3E]] ERROR [] com.co3.broker.AcknowledgementProcessor - Error processing message: { "message_type": 0, "message": "Completed", "complete": true, "results": { "version": "1.0", "success": true, "reason": null, "content": { "timeout_type": "FIRST_SEEN", "number_of_elements": 11360, "data": { "xxx": { "Hostname": { "last_seen": 1656076954240, "first_seen": 1655817626389, "source": "Table_Users_Hostname", "value": "xxx", "domain_id": null }, "Log Source": { "last_seen": 1656076954240, "first_seen": 1655817626389, "source": "Table_Users_Hostname", "value": "xxx", "domain_id": null }, "Logon Time": { "last_seen": 1656079534261, "first_seen": 1655817626389, "source": "Table_Users_Hostname", "value": "1656079481000", "domain_id": null }, "Logon Type": {
java.lang.IllegalStateException: Unable to commit transaction
06:20:00.348 [Camel (camel-1) thread #13 - JmsConsumer[acks.%3E]] WARN [] com.co3.broker.GenericProcessor - GenericProcessor Exception on message
org.camunda.bpm.engine.OptimisticLockingException: ENGINE-03005 Execution of 'UPDATE ExecutionEntity[e5162656-4087-11ed-b9b7-005056a8bf41]' failed. Entity was updated by another transaction concurrently.
In UI I see only one reason:
An error occurred while processing the action acknowledgement.
------------------------------
Gule Fahid
------------------------------