IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Broker server restart issue

  • 1.  Broker server restart issue

    Posted Mon July 31, 2017 01:17 PM

    Dear All,

    I’m getting below errors while restarting broker server on windows machine through services.

    1. Error: Broker “broker 1”: Could not create configuration storage for Event Type “canonical 1”, error 1302 occurred.

    My comment: Where broker 1 is the name of the broker created earlier on broker server and canonical 1 is one of the canonical present on IS.
    As per my understanding error related to this broker 1 shouldnt be coming because broker server containing this broker itself doesn’t exist anymore when I check from MWS.However please be noted that broker 1 was present earlier as part of this broker server.

    1. Error: Broker “broker 1”: Out of storage occurred while processing update from Remote Broker “broker 2”. [RESTARTING]

    My comment: Again error message is related to broker 1 which shouldn’t have occurred. broker 1 and 2 are part of same territory.

    1. Unable to initialize basic authentication module. Verify the contents of basic authentication config file.

    My comment: No changes were done in this file. It was working fine earlier.

    Can you please share your thoughts.

    Thanks,
    Ashish


    #broker
    #webMethods
    #Integration-Server-and-ESB
    #Universal-Messaging-Broker


  • 2.  RE: Broker server restart issue

    Posted Mon July 31, 2017 03:43 PM

    Hi Ashish,

    please provide more details about your environment.

    Which wM Version?
    Which Broker Version?
    Any fixes to the above?

    Are both broker servers managed by the same broker monitor?

    What is the Broker Memory/Storage Usage (can be checked via MWS Messaging UI)?

    Regards,
    Holger


    #webMethods
    #broker
    #Universal-Messaging-Broker
    #Integration-Server-and-ESB


  • 3.  RE: Broker server restart issue

    Posted Tue August 01, 2017 05:20 AM

    Hi Holger,

    Please find my comments below.

    wM version: 9.8
    Broker version: 9.6
    Fixes: No

    Each broker server have their own dedicated broker monitor.

    Broker memory/storage usage in MWS I can check only for the broker server (broker 2) which is up. Broker server (broker 1) which is down is not showing anymore in MWS however we have already checked the DB space where broker environment is hosted and its normal.
    Here is the memory usage for Broker server (broker 2).

    Total: 14 GB
    Used: 9 GB
    Free: 5 GB

    Thanks,
    Ashish


    #Universal-Messaging-Broker
    #webMethods
    #broker
    #Integration-Server-and-ESB


  • 4.  RE: Broker server restart issue

    Posted Tue August 01, 2017 10:39 AM

    Hi Ashish,

    you should apply at least latest Fixes for the Broker (Broker Core, C API and Java API) to both installations.
    Update MWS Messaging UI with latest Broker Portal Fix accordingly.

    Is there anything in the broker server log which indicates why broker 1 is going down/restarting?

    What do you mean by DB space?
    Broker is not using any databases except its own storage files.

    Regards,
    Holger


    #webMethods
    #Universal-Messaging-Broker
    #broker
    #Integration-Server-and-ESB


  • 5.  RE: Broker server restart issue

    Posted Tue August 01, 2017 12:15 PM

    Hi Holger,

    Thanks for your quick response. Via update manager I could see these fixes are already applied. Below are the fixes already applied:

    Broker command line 9.6.0 Fix (W64)
    broker core 9.6.0 Fix 7 (W64)
    Broker Java API 9.6.0 Fix4
    Broker Java API shared bundles 9.6 Fix4
    Broker SPM plugin 9.6.0

    For MWS also, broker portal fix is already applied. In broker server log as well same error messages are coming which I have shared earlier. However I have noticed one thing today. When I brought down broker monitor and server of 2nd environment, I was able to successfully restart the broker server of first environment. As soon as I brought up the broker monitor of 2nd environment again, broker server of first environment went down.

    Please be noted that both 1st and 2nd broker environments are running on seperate machine however they are connected via broker territory.

    My bad. I was referring to MWS database.

    Thanks,
    Ashish


    #Integration-Server-and-ESB
    #Universal-Messaging-Broker
    #broker
    #webMethods


  • 6.  RE: Broker server restart issue

    Posted Tue August 01, 2017 01:06 PM

    Hi Ashish,

    latest Fixes are:
    Broker Core Fix14
    Broker ComnmandLine Fix2
    Broker Java API Fix9
    Broker C API Fix3
    Broker Portal 7

    When the two installations are on two different boxes then there should be no port conflict and therefore no reason why the first Broker server is going down when the second one is started.
    Please provide the storage usage for both broker servers (just with one of them started if neccessary).
    Eventually you will have to add additional storage files to the broker storages.
    See Broker Admin Guide for instructions.

    Regards,
    Holger


    #Universal-Messaging-Broker
    #Integration-Server-and-ESB
    #webMethods
    #broker


  • 7.  RE: Broker server restart issue

    Posted Tue August 01, 2017 02:56 PM

    Hi Holger,

    There is a good news. Broker server is up now after changing the client prefix for the broker connection alias on Integration server however broker is showing not connected (activeYes [Not Connected]) on integration servers.

    I restarted the broker server but result is still the same.
    Can you please suggest what can be done to fix the same.

    Thanks,
    Ashish


    #broker
    #Universal-Messaging-Broker
    #webMethods
    #Integration-Server-and-ESB


  • 8.  RE: Broker server restart issue

    Posted Tue August 01, 2017 03:28 PM

    Hi Ashish,

    did you restart the integrationServer after changing the preifx?

    When changing the broker alias you must restart the IntegrationServer immediately as otherwise the changes are lost.
    When the Broker does not reconnect after restart check the server log for related error messages.

    Check the list of clients on the broker server afterwards and eventually remove the old queue clients (those with the old prefix).

    Regards,
    Holger


    #Universal-Messaging-Broker
    #broker
    #webMethods
    #Integration-Server-and-ESB


  • 9.  RE: Broker server restart issue

    Posted Thu August 03, 2017 07:07 AM

    Hi Holger,

    Sorry for the delay. Yes server was restarted. However I got to know from my colleague that they did some other change as well to bring up broker server.
    They removed BrokerConfig.qs* files and then restarted the broker server which created these files again and server came up but this is not the correct solution since deleting these files will cause deletion of all existing broker, client group and document type details thus we reverted back the change.
    Now we are in the same situation. Below is the current utilization of the broker server (broker2) which is up:

    Resource name Total capacity Used Free
    Virtual memory 15.6 GB 8.36 GB 7.24 GB

    As these are storage issues, can it be because of the volume of updates broker 1 is sending at the time of broker server restart to broker 2 is more than the free space available.

    Please share your thoughts.

    Thanks,
    Ashish


    #Universal-Messaging-Broker
    #broker
    #webMethods
    #Integration-Server-and-ESB


  • 10.  RE: Broker server restart issue

    Posted Mon August 07, 2017 06:51 AM

    Hi Holger,

    To provide you an update this issue is resolved now. This was due to BrokerConfig.qs* files got corrupted. We restored these files from the backup of previous date.

    Cheers,
    Ash


    #Universal-Messaging-Broker
    #webMethods
    #broker
    #Integration-Server-and-ESB