Planning Analytics

Planning Analytics

Get AI-infused integrated business planning

 View Only
Expand all | Collapse all

Experiences with Satellite Connector Reliability

  • 1.  Experiences with Satellite Connector Reliability

    Posted Wed May 07, 2025 10:04 PM

    Hi All,

    Wondering how many are using the Satellite Connector and what the experience has been like so far. We have recently migrated to cloud with a major client and have set up the Satellite Connector agents across 3 servers for high availability as advised by the IBM docs. 

    We have found that despite the high availability setup, the agents all randomly drop connections, causing critical TI processes to fail unpredictably. At worst this causes failures in dimensions rebuilds and leads to a bit of a mess to clean up, at best it results in delays in data being available to users during time-critical period end processes. We were advised that the cause was server side and that IBM have reproduced the behavior internally; however, IBM have also now stated that it is "by design" and will not commit to a fix or a timeline for resolution.

    Has anyone else experienced random failure of TI processes with Satellite Connector and how do you handle it if you do?

    Note: I am aware of and have designed the processes to all run within the 10 minute timeout limit that Satellite Connector introduced and have applied virus scan exclusions for all satellite connector folders on all servers.

    Thanks,

    Joe



    ------------------------------
    Joseph White
    ------------------------------


  • 2.  RE: Experiences with Satellite Connector Reliability

    Posted Thu May 08, 2025 12:55 AM

    Hi Joseph,

    Yes, we are also facing similar issue in our IBM cloud environment but as per IBM suggestions satelite connecter agent need to install in different inhouse premises server but I am not sure is this bug within satelite connecter or as per new setup IBM is recommended as work around steps till resolution.

    Thanks,

    Chandrakant



    ------------------------------
    Chandrakant Jagadale
    ------------------------------



  • 3.  RE: Experiences with Satellite Connector Reliability

    Posted Thu May 08, 2025 01:10 AM

    Chandrakant,

    Out of interest, are you parallel running TI processes over Satellite Connector? How often are you seeing failures?

    Thanks,

    Joe



    ------------------------------
    Joseph White
    ------------------------------



  • 4.  RE: Experiences with Satellite Connector Reliability

    Posted Thu May 08, 2025 07:48 AM

    Hello,

    We have 2 customers with exact same problem of Satellite down random issues causing critical TI processes to fail.

    We contacted IBM support and still waiting for solution.

    BTW - restarting Satellite agent service restores the connection.

    Thanks,

    Gabi



    ------------------------------
    Gabi Cohen
    ------------------------------



  • 5.  RE: Experiences with Satellite Connector Reliability

    Posted Thu May 08, 2025 10:43 AM

    We have the same issue with multiple clients. Issues are intermittent and random. I'm pretty sure IBM is aware and investigating. 



    ------------------------------
    Matthew Berson
    ------------------------------



  • 6.  RE: Experiences with Satellite Connector Reliability

    Posted Thu May 08, 2025 10:04 PM

    Thanks for the feedback. It's reassuring to know that we are not alone, but frustrating that IBM don't seem to have a handle on the issue.

    FYI, we tested the latest version of the Agent 1.2 on our non prod with 3 agents for high availability and are still seeing the forced connection resets which cause TI process failure.



    ------------------------------
    Joseph White
    ------------------------------



  • 7.  RE: Experiences with Satellite Connector Reliability

    Posted Thu May 29, 2025 05:55 AM

    Thanks for letting me know as we are going to update to v1.2 and exclude the satellite agent folders from the anti virus scan just to rule this out.

    Regards 



    ------------------------------
    Jing Yee Harle
    ------------------------------



  • 8.  RE: Experiences with Satellite Connector Reliability

    Posted Tue May 27, 2025 12:30 PM

    Hi

    We experienced same issue on 12/5/25 and 15/5/25.

    IBM reviewed the satellite connectors logs for both days and they have responded today that this is an issue on their side which is currently under investigation.

    See response below:

    I can tell you that this was the expected behaviour caused by Satellite server activities on our side. Your Agent could experience such restarts, error is always the same:

    All connections for U2F0ZWxsaXRlQ29ubmVjdG9yOiJjcmttNnM3ZjEwOGtqMnRubWI5MCI have been destroyed

    This is currently being investigated by Satellite Development team, you should not experience restarts in next days.

    Make sure to:

    1. Exclude Satellite Agent folders from Antivirus scans on your machines.
    2. Make sure to install the newest Agent version (1.2.0).
    3. Notify IBM Support about any new occurrence of that error.

    Regards 



    ------------------------------
    Jing Yee Harle
    ------------------------------



  • 9.  RE: Experiences with Satellite Connector Reliability

    Posted Tue May 27, 2025 10:30 PM

    Hi,

    Yes, I think this is the same if you are seeing TC22 messages in your log file. We've been in regular contact with IBM on this and I'm happy to say that they appear to have a handle on this and the stability issues are now greatly improved, although we might not be at a full solution quite yet.

    Joe



    ------------------------------
    Joseph White
    ------------------------------



  • 10.  RE: Experiences with Satellite Connector Reliability

    Posted 16 days ago

    Hello. IBM Cloud Satellite Connector Architect here. I wanted to update this thread in case others come upon this with what we did to address this with the Planning Analytics team. The method we used to update our service relies on triple redundant everything. 3 pathways, 3 IP addresses, IBM cloud multi-zone regions (MZR). What we learned in working with the Planning Analytics team is that database drivers to do some of the advanced work that happens in Planning Analytics usually don't have database connection pooling or connection failover/retry mechanisms to employ our triple redundant pathway. This isn't the PA team's shortcoming. This is actually how most database client drivers are designed. In order to reduce the disruption to PA and IBM customers, we aligned our updates to the Planning Analytics teams updates (see notices for outages for specifics). I believe this has been quiet now for most users of this for months, but please open a support case if anyone sees issues and we always work together to address issues that arise. 



    ------------------------------
    GERALD COON
    ------------------------------



  • 11.  RE: Experiences with Satellite Connector Reliability

    Posted 15 days ago

    Hi Gerald,

    I can confirm that the Satellite Connector has been very stable for us and all of our customers since you and your team's work on this.

    Thanks for the support!

    Joe



    ------------------------------
    Joseph White
    ------------------------------