Description:

This article is about Cluster Provider initialization exception during the startup of My webMethods Server. The article applies to My webMethods Server versions 10+.

The startup of a My webMethods Server cluster node might fail with the following stack trace:

 2021-01-25 15:53:47 ICT (c.w.p.s.c.i.ClusterProvider:ERROR) [Start Level: Equinox Container: cb90d8aa-84ac-47e9-9818-239e0be842db] [   ]  [  ]- null
com.webmethods.portal.system.init.InitializationException: null
at com.webmethods.portal.system.cluster.impl.ClusterProvider.init(ClusterProvider.java:389) ~[com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
at com.webmethods.portal.system.impl.BaseProvider.activate(BaseProvider.java:146) [com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
at com.webmethods.portal.system.cluster.impl.ClusterProvider.activate(ClusterProvider.java:204) [com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_242]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_242]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_242]
at 
org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1657) [org.eclipse.osgi_3.13.300.0000-0004.jar:3.13.300.0000-0004]
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1558) [org.eclipse.osgi_3.13.300.0000-0004.jar:3.13.300.0000-0004]
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:233) [org.eclipse.osgi_3.13.300.0000-0004.jar:3.13.300.0000-0004]
at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:343) [org.eclipse.osgi_3.13.300.0000-0004.jar:3.13.300.0000-0004]
Caused by: com.webmethods.portal.PortalException
at com.webmethods.portal.system.cluster.impl.ClusterProvider.registerLocalClusterNode(ClusterProvider.java:754) ~[com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
at com.webmethods.portal.system.cluster.impl.ClusterProvider.init(ClusterProvider.java:364) ~[com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
... 135 more
2021-01-25 15:53:47 ICT (c.w.r.p.ShutdownManager:INFO) [tHREAD-13][  ] [  ] - Shutdown Manager running all shutdown hooks.
2021-01-25 15:53:47 ICT (c.w.r.p.ShutdownManager:INFO) [tHREAD-13][  ] [  ] - Portal state : 4
[ProxyLogManager] reset() -> org.apache.logging.log4j.jul.LogManager@79638fc5
2021-01-25 15:53:47 ICT (c.w.p.s.i.i.ClusterPhaseProvider:INFO) [MWS Boostrap Lock Hook][  ] [  ]   -Released cluster bootstrap lock
[Mon Jan 25 15:53:48 ICT 2021] Bootstrap : Stopping...

!ENTRY com.softwareag.platform.hook.shutdown 1 0 2021-01-25 15:53:48.304
!MESSAGE Stopping OSGi framework.
2021-01-25 15:53:48 ICT (o.e.e.logger:INFO) [Wrapper-Shutdown-Hook] [  ] [  ] -Stopping OSGi framework.

!ENTRY com.softwareag.platform.hook.shutdown 1 0 2021-01-25 15:53:48.305
!MESSAGE Framework shutdown timeout set to 120 seconds.Please also check tanuki-wrapper shutdown property, if specified.
2021-01-25 15:53:48 ICT (o.e.e.logger:INFO) [Wrapper-Shutdown-Hook] [  ] [  ] -Framework shutdown timeout set to 120 seconds. Please also check tanuki-wrapper shutdown property, id specified.

Root Cause:

This issue occurs when the values of the NODE_NAME parameter for the instance do not match throughout all required configuration files for My webMethods Server. The NODE_NAME mismatch is usually a result of manual editing of one or more (but not all) of the configuration files.

My webMethods Server stores the node name of a cluster node in the cluster.xml file in the My webMethods Server database, and in the following configuration files on the file system:

1. <SoftwareAG_directory>/MWS/server//config/cluster.node.properties(bak) - the value in this file gets copied to the cluster.xml configuration file in the My webMethods Server database at startup.
2. <SoftwareAG_directory>/profiles/ MWS_/configuration/custom_wrapper.conf (set.NODE_NAME=) - the value in this file gets verified at startup, together with the value in the configuration file in 1. If the values do not match, the server node will not start.
3. <SoftwareAG_directory>/server//bin/instance.properties - the value in this file gets copied to the custom_wrapper.conf configuration file (2.) when creating or updating the My webMethods Server profile.

When updating the instance profile, for example, with the installation of a test patch or official fix, or when using the update command, My webMethods overwrites the NODE_NAME values in the custom_wrapper.conf and cluster.xml configuration files.
If you modified part of the NODE_NAME values manually, and the values in all configuration files no longer match, the My webMethods Server cluster node will fail to initialize with the exception above.

Resolution:

My webMethods Server provides a mechanism for updating the name of a cluster node, using the command line. In order to avoid the issue, described in this article, always use the My webMethods Server command line to modify the NODE_NAME of a cluster node, as follows:

1. Go to <SoftwareAG_directory>/MWS/bin/mws.sh|bat
2. Run the mws update command, as follows:
   mws -s update -Dnode.name=<newNodeName>