IBM Security QRadar SOAR

 View Only
  • 1.  Custom Threat Source - java.lang.RuntimeException: Unexpected response from...

    Posted Thu October 31, 2019 07:37 AM
    Edited by Lucian Sipos Thu October 31, 2019 07:38 AM
    Hello all

    I created a custom threat source and set it up. It works but the behaviour (or the steps to make it work properly) are unusual. This is what I do to have a result, giving an artifact, from the CTS:

    1. Enabled my CTS from Administration settings
    2. Create a new incident and add an artifact
      1. nothing happens - the artifact stays grey and, sometimes, the checking animation does not even appear - refreshing the page have not effect
      2. this is the client.log for these first steps:
        • 12:26:38.658 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] ERROR com.co3.context.Co3ContextRunnable - Exception in runnable
          java.lang.RuntimeException: Unexpected response from http://MYIP:9000/cts/MYCTS/aacf4e02-f5c2-5702-abc6-7d597ef3f2f2
          at com.co3.threat.CustomThreatService.executeUrl(CustomThreatService.java:263)
          at com.co3.threat.CustomThreatService.execute(CustomThreatService.java:534)
          at com.co3.context.Co3PersistentCommand.lambda$run$0(Co3PersistentCommand.java:78)
          at com.co3.context.Co3PersistentCommand$$Lambda$627.00000000E400F350.run(Unknown Source)
          at com.co3.context.Co3ContextRunnable.runImpl(Co3ContextRunnable.java:244)
          at com.co3.context.Co3ContextRunnable.lambda$runScopedImpl$0(Co3ContextRunnable.java:200)
          at com.co3.context.Co3ContextRunnable$$Lambda$164.00000000B40033D0.call(Unknown Source)
          at com.resilient.guice.ResilientGuiceUtils.lambda$callVoidInRequestScope$0(ResilientGuiceUtils.java:49)
          at com.resilient.guice.ResilientGuiceUtils$$Lambda$53.000000003558BF70.call(Unknown Source)
          at com.google.inject.servlet.ServletScopes$4.call(ServletScopes.java:450)
          at com.resilient.guice.ResilientGuiceUtils.callInRequestScope(ResilientGuiceUtils.java:70)
          at com.resilient.guice.ResilientGuiceUtils.callVoidInRequestScope(ResilientGuiceUtils.java:53)
          at com.co3.context.Co3ContextRunnable.runScopedImpl(Co3ContextRunnable.java:200)
          at com.co3.context.Co3ContextRunnable.run(Co3ContextRunnable.java:186)
          at com.co3.context.Co3PersistentCommand.run(Co3PersistentCommand.java:92)
          at com.co3.schedule.ScheduledJobConsumer.process(ScheduledJobConsumer.java:70)
          at sun.reflect.GeneratedMethodAccessor2666.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
          at java.lang.reflect.Method.invoke(Method.java:508)
          at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
          at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
          at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
          at com.google.common.eventbus.Subscriber.dispatchEvent(Subscriber.java:67)
          at com.google.common.eventbus.Dispatcher$PerThreadQueuedDispatcher.dispatch(Dispatcher.java:108)
          at com.google.common.eventbus.EventBus.post(EventBus.java:212)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus.processCamelExchange(CamelGuavaInterprocessEventBus.java:51)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus.lambda$register$0(CamelGuavaInterprocessEventBus.java:46)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus$$Lambda$148.00000000298A1B50.accept(Unknown Source)
          at com.ibm.eventbus.camel.DefaultCamelActiveMqStrategyImpl$1.lambda$configure$0(DefaultCamelActiveMqStrategyImpl.java:70)
          at com.ibm.eventbus.camel.DefaultCamelActiveMqStrategyImpl$1$$Lambda$156.00000000292CF760.process(Unknown Source)
          at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
          at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
          at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
          at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
          at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:123)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:719)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:679)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:649)
          at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317)
          at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1168)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1160)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1057)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
          at java.lang.Thread.run(Thread.java:812)
          Caused by: java.net.SocketTimeoutException: Read timed out
          at java.net.SocketInputStream.socketRead0(Native Method)
          at java.net.SocketInputStream.socketRead(SocketInputStream.java:127)
          at java.net.SocketInputStream.read(SocketInputStream.java:182)
          at java.net.SocketInputStream.read(SocketInputStream.java:152)
          at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
          at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
          at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
          at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
          at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
          at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
          at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
          at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
          at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
          at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
          at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
          at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
          at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
          at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
          at com.co3.threat.CustomThreatService.executeUrl(CustomThreatService.java:252)
          ... 45 common frames omitted
          12:26:38.659 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] ERROR com.co3.context.Co3ContextRunnable - Exception while running
          java.lang.RuntimeException: Unexpected response from http://MYIP:9000/cts/MYCTS/aacf4e02-f5c2-5702-abc6-7d597ef3f2f2
          at com.co3.threat.CustomThreatService.executeUrl(CustomThreatService.java:263)
          at com.co3.threat.CustomThreatService.execute(CustomThreatService.java:534)
          at com.co3.context.Co3PersistentCommand.lambda$run$0(Co3PersistentCommand.java:78)
          at com.co3.context.Co3PersistentCommand$$Lambda$627.00000000E400F350.run(Unknown Source)
          at com.co3.context.Co3ContextRunnable.runImpl(Co3ContextRunnable.java:244)
          at com.co3.context.Co3ContextRunnable.lambda$runScopedImpl$0(Co3ContextRunnable.java:200)
          at com.co3.context.Co3ContextRunnable$$Lambda$164.00000000B40033D0.call(Unknown Source)
          at com.resilient.guice.ResilientGuiceUtils.lambda$callVoidInRequestScope$0(ResilientGuiceUtils.java:49)
          at com.resilient.guice.ResilientGuiceUtils$$Lambda$53.000000003558BF70.call(Unknown Source)
          at com.google.inject.servlet.ServletScopes$4.call(ServletScopes.java:450)
          at com.resilient.guice.ResilientGuiceUtils.callInRequestScope(ResilientGuiceUtils.java:70)
          at com.resilient.guice.ResilientGuiceUtils.callVoidInRequestScope(ResilientGuiceUtils.java:53)
          at com.co3.context.Co3ContextRunnable.runScopedImpl(Co3ContextRunnable.java:200)
          at com.co3.context.Co3ContextRunnable.run(Co3ContextRunnable.java:186)
          at com.co3.context.Co3PersistentCommand.run(Co3PersistentCommand.java:92)
          at com.co3.schedule.ScheduledJobConsumer.process(ScheduledJobConsumer.java:70)
          at sun.reflect.GeneratedMethodAccessor2666.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
          at java.lang.reflect.Method.invoke(Method.java:508)
          at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
          at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
          at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
          at com.google.common.eventbus.Subscriber.dispatchEvent(Subscriber.java:67)
          at com.google.common.eventbus.Dispatcher$PerThreadQueuedDispatcher.dispatch(Dispatcher.java:108)
          at com.google.common.eventbus.EventBus.post(EventBus.java:212)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus.processCamelExchange(CamelGuavaInterprocessEventBus.java:51)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus.lambda$register$0(CamelGuavaInterprocessEventBus.java:46)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus$$Lambda$148.00000000298A1B50.accept(Unknown Source)
          at com.ibm.eventbus.camel.DefaultCamelActiveMqStrategyImpl$1.lambda$configure$0(DefaultCamelActiveMqStrategyImpl.java:70)
          at com.ibm.eventbus.camel.DefaultCamelActiveMqStrategyImpl$1$$Lambda$156.00000000292CF760.process(Unknown Source)
          at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
          at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
          at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
          at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
          at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:123)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:719)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:679)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:649)
          at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317)
          at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1168)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1160)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1057)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
          at java.lang.Thread.run(Thread.java:812)
          Caused by: java.net.SocketTimeoutException: Read timed out
          at java.net.SocketInputStream.socketRead0(Native Method)
          at java.net.SocketInputStream.socketRead(SocketInputStream.java:127)
          at java.net.SocketInputStream.read(SocketInputStream.java:182)
          at java.net.SocketInputStream.read(SocketInputStream.java:152)
          at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
          at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
          at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
          at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
          at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
          at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
          at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
          at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
          at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
          at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
          at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
          at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
          at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
          at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
          at com.co3.threat.CustomThreatService.executeUrl(CustomThreatService.java:252)
          ... 45 common frames omitted
          12:26:38.669 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] WARN com.co3.threat.ThreatServiceBase - Removing all pending records for artifact: 5324
          12:26:38.669 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] WARN com.co3.threat.ThreatServiceBase - Removing all pending records for artifact: 5324
          12:26:38.679 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] ERROR com.co3.schedule.ScheduledJobConsumer - Executed com.co3.threat.CustomThreatService job c6eb868e-c67a-479b-b672-a3ce4b16a8f9 error, won't retry. Current retry count 0. Max retry count 0 . Exception:
          java.lang.RuntimeException: Unexpected response from http://MYIP:9000/cts/MYCTS/aacf4e02-f5c2-5702-abc6-7d597ef3f2f2
          at com.co3.threat.CustomThreatService.executeUrl(CustomThreatService.java:263)
          at com.co3.threat.CustomThreatService.execute(CustomThreatService.java:534)
          at com.co3.context.Co3PersistentCommand.lambda$run$0(Co3PersistentCommand.java:78)
          at com.co3.context.Co3PersistentCommand$$Lambda$627.00000000E400F350.run(Unknown Source)
          at com.co3.context.Co3ContextRunnable.runImpl(Co3ContextRunnable.java:244)
          at com.co3.context.Co3ContextRunnable.lambda$runScopedImpl$0(Co3ContextRunnable.java:200)
          at com.co3.context.Co3ContextRunnable$$Lambda$164.00000000B40033D0.call(Unknown Source)
          at com.resilient.guice.ResilientGuiceUtils.lambda$callVoidInRequestScope$0(ResilientGuiceUtils.java:49)
          at com.resilient.guice.ResilientGuiceUtils$$Lambda$53.000000003558BF70.call(Unknown Source)
          at com.google.inject.servlet.ServletScopes$4.call(ServletScopes.java:450)
          at com.resilient.guice.ResilientGuiceUtils.callInRequestScope(ResilientGuiceUtils.java:70)
          at com.resilient.guice.ResilientGuiceUtils.callVoidInRequestScope(ResilientGuiceUtils.java:53)
          at com.co3.context.Co3ContextRunnable.runScopedImpl(Co3ContextRunnable.java:200)
          at com.co3.context.Co3ContextRunnable.run(Co3ContextRunnable.java:186)
          at com.co3.context.Co3PersistentCommand.run(Co3PersistentCommand.java:92)
          at com.co3.schedule.ScheduledJobConsumer.process(ScheduledJobConsumer.java:70)
          at sun.reflect.GeneratedMethodAccessor2666.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
          at java.lang.reflect.Method.invoke(Method.java:508)
          at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
          at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
          at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
          at com.google.common.eventbus.Subscriber.dispatchEvent(Subscriber.java:67)
          at com.google.common.eventbus.Dispatcher$PerThreadQueuedDispatcher.dispatch(Dispatcher.java:108)
          at com.google.common.eventbus.EventBus.post(EventBus.java:212)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus.processCamelExchange(CamelGuavaInterprocessEventBus.java:51)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus.lambda$register$0(CamelGuavaInterprocessEventBus.java:46)
          at com.ibm.eventbus.CamelGuavaInterprocessEventBus$$Lambda$148.00000000298A1B50.accept(Unknown Source)
          at com.ibm.eventbus.camel.DefaultCamelActiveMqStrategyImpl$1.lambda$configure$0(DefaultCamelActiveMqStrategyImpl.java:70)
          at com.ibm.eventbus.camel.DefaultCamelActiveMqStrategyImpl$1$$Lambda$156.00000000292CF760.process(Unknown Source)
          at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
          at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
          at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
          at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
          at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:123)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:719)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:679)
          at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:649)
          at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317)
          at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1168)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1160)
          at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1057)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
          at java.lang.Thread.run(Thread.java:812)
          Caused by: java.net.SocketTimeoutException: Read timed out
          at java.net.SocketInputStream.socketRead0(Native Method)
          at java.net.SocketInputStream.socketRead(SocketInputStream.java:127)
          at java.net.SocketInputStream.read(SocketInputStream.java:182)
          at java.net.SocketInputStream.read(SocketInputStream.java:152)
          at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
          at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
          at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
          at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
          at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
          at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
          at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
          at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
          at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
          at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
          at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
          at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
          at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
          at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
          at com.co3.threat.CustomThreatService.executeUrl(CustomThreatService.java:252)
          ... 45 common frames omitted
      3. Without deleting the artifact (but even deleting it, I tested the both cases) I add the same artifact
      4. Refreshing the page make the artifact field of red colour and if I click to see details I can view infos given by my CTS
        1. This is the client.log AFTER I added the second artifact (comes directly after the log lines above):
          • 12:32:39.944 [pool-3-thread-1] INFO com.co3.threat.ThreatFeedManager - Rescanning for incident artifact hits
            12:32:39.944 [pool-3-thread-1] INFO com.co3.threat.ThreatFeedManager - Rescanning for incident artifact hits

    What is going on ? Why adding the same artifact a second time make the CTS to work ? How can I solve the errors in the first log ? I used as a template for my CTS the misp one I can find on resilient-community-apps

    Thanks

    ------------------------------
    Bruce Wayne
    Senior Dark Knight
    ------------------------------


  • 2.  RE: Custom Threat Source - java.lang.RuntimeException: Unexpected response from...

    Posted Fri November 01, 2019 07:25 AM
    Hi Bruce,

    The stack trace shows that Resilient is unable to communicate with your custom threat service: http://MYIP:9000/cts/MYCTS/aacf4e02-f5c2-5702-abc6-7d597ef3f2f2. Confirm that you have resilient-circuits running on your integrations server with your threat service started. 

    Good luck

    ------------------------------
    Mark Scherfling
    ------------------------------



  • 3.  RE: Custom Threat Source - java.lang.RuntimeException: Unexpected response from...

    Posted Mon November 04, 2019 03:39 AM
    Edited by Lucian Sipos Mon November 04, 2019 05:08 AM
    Hi Mark

    Thanks for the answer. Yes, the circuits it's running and the threat service is started.

    The fact is that it works if, like described above, I add the same artifact a second time. At first it won't work (and this is why I create this thread) but at second yes.

    I would like to add this: at first try, in the app.log file I see this log:

    2019-11-04 09:32:12,218 INFO [threat_webservice] <Request POST /cts/MYCTS HTTP/1.1>
    2019-11-04 09:32:12,219 DEBUG [threat_webservice] {u'type': u'net.ip', u'value': u'163.172.40.218'}
    2019-11-04 09:32:12,219 INFO [threat_webservice] 303 See Other: {"retry_secs": 5, "hits": [], "id": "00a2b17c-8c57-5bde-9bfa-bd964c6bec55"}
    2019-11-04 09:32:12,234 INFO [threat_webservice] helper: <net.ip[threat_lookup_helper] (00a2b17c-8c57-5bde-9bfa-bd964c6bec55)>, cts_search.MYCTS
    2019-11-04 09:32:32,937 INFO [searcher] HITS: [Hit([('props', [{'type': 'string', 'name': u'Malware', 'value': 'Heodo'}, {'type': 'string', 'name': u'LastOnline', 'value': '2019-11-04'}, {'type': 'string', 'name': u'Firstseen', 'value': '2019-10-30 15:42:11'}, {'type': 'string', 'name': u'DstPort', 'value': '7080'}, {'type': 'string', 'name': u'Lista', 'value': u'BOTNET_C2_IP_BLOCKLIST_ALL'}, {'type': 'uri', 'name': u'URL', 'value': u'https://feodotracker.abuse.ch/downloads/ipblocklist.txt'}, {'type': 'string', 'name': u'MasterFile Timestamp', 'value': u'2019-10-31 17:45:36'}])])]
    2019-11-04 09:32:32,941 DEBUG [client] Received heart-beat
    2019-11-04 09:32:32,942 DEBUG [client] Received MESSAGE frame [headers={u'expires': u'0', u'Co3ContextToken': u'eyJhbGciOiJIUzI1NiJ9.bnVsbA.cH1P6y_AmRWRTr3dQNNnNr8bpl88i-VT6p95433_KY0', u'ack': u'ID:HOST-36225-1572448310497-32:307', u'timestamp': u'1572856331477', u'JMSXUserID': u'SYSTEM', u'destination': u'/queue/actions.201.fn_elasticsearch', u'correlation-id': u'invid:94427', u'persistent': u'true', u'priority': u'4', u'Co3MessagePayload': u'FunctionDataDTO', u'Co3RemoteAddr': u'127.0.0.1', u'reply-to': u'/queue/acks.201.fn_elasticsearch', u'message-id': u'ID:HOST-43717-1572448359513-3:3:1877:1:1', u'Co3ContentType': u'application/json', u'subscription': u'actions.201.fn_elasticsearch'}, body='{"function":{"creato...', version=1.2]
    2019-11-04 09:32:32,943 DEBUG [stomp_component] Recieved frame MESSAGE

    What concern me is: why do I have a 303 error and an empty hits list (even if, as you can see, my log show that the list is populated) ? If I add the artifact a second time, it work without that 303.

    Also, correct me if I am wrong, does the app.log file is being updated after the client.log ? If yes, then we "return" to the issue above (the stack trace).

    Why, in client.log file, I have a response with an empty hits list ?

    A video of the steps I make actually:



    Thanks

    ------------------------------
    Bruce Wayne
    Senior Dark Knight
    ------------------------------



  • 4.  RE: Custom Threat Source - java.lang.RuntimeException: Unexpected response from...

    Posted Wed November 06, 2019 03:47 AM
    I figured out what the problem was and also I implemented a solution.

    ------------------------------
    Bruce Wayne
    Senior Dark Knight
    ------------------------------



  • 5.  RE: Custom Threat Source - java.lang.RuntimeException: Unexpected response from...

    IBM Champion
    Posted Thu November 07, 2019 11:34 AM
    Hey Bruce,

    It may be helpful if you noted what the problem/solution were, in case others encounter this and find this in the future.

    ------------------------------
    Jared Fagel
    Cyber Security Analyst Intern
    Public Utility
    ------------------------------



  • 6.  RE: Custom Threat Source - java.lang.RuntimeException: Unexpected response from...

    Posted Mon November 11, 2019 04:05 AM
    Edited by Lucian Sipos Mon November 11, 2019 04:07 AM
    The problem was that the result from the CTS (which is a local one, with an in-file search for value) was taking too much from when I add the artifact to when Resilient itself have an "answer" (aka return value).

    I solved by improving the search function in my CTS and by increasing the retry time in app.config (section [custom_threat_service] - first_retry_secs=20 - later_retry_secs=20).

    By now it's working. However I really don't understand the log I posted above (how it's correlated with Mark answer) and Resilient behaviour showed in my video. Where, in the log, is write something that should let me know that "Resilient is unable to communicate with your custom threat service" ?
    Also, @Mark Scherfling what do you mean by "Confirm that you have resilient-circuits running on your integrations server with your threat service started" ? If I have a result in the second try (we talk about submitting two artifacts in 10 seconds where the second one is ok) why CTS should be not started ?

    To be honest, I am confused about the Resilient behaviour I can see and the answers I received.


    ------------------------------
    Bruce Wayne
    Senior Dark Knight
    ------------------------------



  • 7.  RE: Custom Threat Source - java.lang.RuntimeException: Unexpected response from...

    Posted Wed November 20, 2019 11:22 AM
    See Jared, that's why I did not post the answer before. No one answered my questions I made 9 days ago. Why post at this point ?

    ------------------------------
    Bruce Wayne
    Senior Dark Knight
    ------------------------------



  • 8.  RE: Custom Threat Source - java.lang.RuntimeException: Unexpected response from...

    Posted Wed November 27, 2019 03:35 PM
    Well, because now I had the same issue you had. And your solution worked for me too! thanks

    ------------------------------
    Nathan Getty
    ------------------------------