webMethods

What product/components do you use and which version/fix level?

webMethods Integrations server-10.11 version
Terracotta-4.3.9

Are you using a free trial or a product with a customer license?

Customer license

What are trying to achieve? Please describe in detail.

Try to bring two IS into cluster

Do you get any error messages? Please provide a full error message screenshot and log file.

2022-07-02 16:49:02 BST [ISS.0095.0013I] (tid=94) Audit Logging initialized.
2022-07-02 16:52:05 BST [ISS.0017.0001E] (tid=94) An error was enc`Preformatted text`ountered parsing SoftwareAG-IS-Core.xml: net.sf.ehcache.CacheException: Unable to load class net.sf.eh
cache.terracotta.ExpressEnterpriseTerracottaClusteredInstanceFactory. Initial cause was org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException
: Unable to create toolkit.
at net.sf.ehcache.util.ClassLoaderUtil.createNewInstance(ClassLoaderUtil.java:91)
at net.sf.ehcache.terracotta.TerracottaClusteredInstanceHelper.newClusteredInstanceFactory(TerracottaClusteredInstanceHelper.java:157)
at net.sf.ehcache.terracotta.TerracottaClient.createNewClusteredInstanceFactory(TerracottaClient.java:180)
at net.sf.ehcache.terracotta.TerracottaClient.createClusteredInstanceFactory(TerracottaClient.java:129)
at net.sf.ehcache.CacheManager.doInit(CacheManager.java:466)
at net.sf.ehcache.CacheManager.init(CacheManager.java:398)
at net.sf.ehcache.CacheManager.<init>(CacheManager.java:269)
at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1108)
at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1084)
at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:374)
at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:428)
at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:33)
at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:13)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.createToolkit(TerracottaToolkitBuilder.java:65)
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.buildToolkit(TerracottaToolkitBuilder.java:58)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.createTerracottaToolkit(ToolkitInstanceFactoryImpl.java:184)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.<init>(ToolkitInstanceFactoryImpl.java:120)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.<init>(ToolkitInstanceFactoryImpl.java:130)
at org.terracotta.modules.ehcache.EnterpriseToolkitInstanceFactory.<init>(EnterpriseToolkitInstanceFactory.java:23)
at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.createToolkitInstanceFactory(EnterpriseTerracottaClusteredInstanceFactory.j
ava:21)
at org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.<init>(TerracottaClusteredInstanceFactory.java:65)
at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.<init>(EnterpriseTerracottaClusteredInstanceFactory.java:15)
at net.sf.ehcache.terracotta.ExpressEnterpriseTerracottaClusteredInstanceFactory.<init>(ExpressEnterpriseTerracottaClusteredInstanceFactory.java:11)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at net.sf.ehcache.util.ClassLoaderUtil.createNewInstance(ClassLoaderUtil.java:73)
... 16 more
Caused by: org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
at com.terracotta.toolkit.api.TerracottaToolkitFactoryService.createToolkit(TerracottaToolkitFactoryService.java:63)
at org.terracotta.toolkit.ToolkitFactory.create(ToolkitFactory.java:100)
at org.terracotta.toolkit.ToolkitFactory.createToolkit(ToolkitFactory.java:84)
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.createToolkit(TerracottaToolkitBuilder.java:63)
... 30 more
Caused by: java.lang.RuntimeException: Unable to create toolkit.
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createToolkit(TerracottaToolkitCreator.java:95)
at com.terracotta.toolkit.api.EnterpriseTerracottaToolkitFactoryService.createToolkit(EnterpriseTerracottaToolkitFactoryService.java:17)
at com.terracotta.toolkit.api.TerracottaToolkitFactoryService.createToolkit(TerracottaToolkitFactoryService.java:58)
... 33 more
Caused by: java.lang.RuntimeException: com.tc.config.schema.setup.ConfigurationSetupException:
*******************************************************************************
Could not fetch configuration data from 2 different configuration sources.  The sources we tried were: the server at 'devtlnx0515.stbc2.jstest2.net:9510', and the serve
r at 'devtlnx0516.stbc2.jstest2.net:9510'.  Fetch attempt duration: 60 seconds.

To correct this problem specify a valid configuration location using the -f/--config command-line options.
*******************************************************************************

at com.terracotta.toolkit.express.TerracottaInternalClientImpl.init(TerracottaInternalClientImpl.java:101)
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createInternalToolkit(TerracottaToolkitCreator.java:154)
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createInternalToolkitSynchronously(TerracottaToolkitCreator.java:111)
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createToolkit(TerracottaToolkitCreator.java:78)
... 35 more
Caused by: com.tc.config.schema.setup.ConfigurationSetupException:
*******************************************************************************
Could not fetch configuration data from 2 different configuration sources.  The sources we tried were: the server at 'devtlnx0515.stbc2.jstest2.net:9510', and the serve
r at 'devtlnx0516.stbc2.jstest2.net:9510'.  Fetch attempt duration: 60 seconds.

To correct this problem specify a valid configuration location using the -f/--config command-line options.
*******************************************************************************

at com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.configurationFetchFailed(StandardXMLFileConfigurationCreator.java:375)
at com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.loadConfigDataFromSources(StandardXMLFileConfigurationCreator.java:264)
at com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.loadConfigAndSetIntoRepositories(StandardXMLFileConfigurationCreator.java:141)
at com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.createConfigurationIntoRepositories(StandardXMLFileConfigurationCreator.java:125)
at com.tc.config.schema.setup.BaseConfigurationSetupManager.runConfigurationCreator(BaseConfigurationSetupManager.java:107)
at com.tc.config.schema.setup.L1ConfigurationSetupManagerImpl.<init>(L1ConfigurationSetupManagerImpl.java:63)
at com.tc.config.schema.setup.StandardConfigurationSetupManagerFactory.getL1TVSConfigurationSetupManager(StandardConfigurationSetupManagerFactory.java:243)
at com.tc.object.DistributedObjectClientFactory.create(DistributedObjectClientFactory.java:92)
at com.terracotta.toolkit.express.CreateClient$ClientCreatorCallableImpl.call(CreateClient.java:123)
at com.terracotta.toolkit.express.TerracottaInternalClientImpl.init(TerracottaInternalClientImpl.java:93)
... 38 more
. See the Integration Server error log for the full stack trace.
2022-07-02 16:52:05 BST [ISS.0095.0042I] (tid=94) The ERRORMESSAGE field in a WMERROR audit record was truncated. CONTEXTID: aed48312-671c-4239-b02f-1d216fcdba8b. MSGID
: be7c18cb-30a1-176a-e353-e19cdf292f8c. Full value: [ISS.0017.0062] Integration Server encountered an error while parsing /app/opt/sag/webmethods/SITISA/IntegrationServ
er/instances/default/config/Caching/SoftwareAG-IS-Core.xml: Unable to load class net.sf.ehcache.terracotta.ExpressEnterpriseTerracottaClusteredInstanceFactory. Initial
cause was org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit..
Caused by: java.lang.RuntimeException: org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
Caused by: org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
Caused by: java.lang.RuntimeException: Unable to create toolkit.
Caused by: java.lang.RuntimeException: com.tc.config.schema.setup.ConfigurationSetupException:
*******************************************************************************
Could not fetch configuration data from 2 different configuration sources.  The sources we tried were: the server at 'devtlnx0515.stbc2.jstest2.net:9510', and the serve
r at 'devtlnx0516.stbc2.jstest2.net:9510'.  Fetch attempt duration: 60 seconds.

To correct this problem specify a valid configuration location using the -f/--config command-line options.
*******************************************************************************

Caused by: com.tc.config.schema.setup.ConfigurationSetupException:
*******************************************************************************
Could not fetch configuration data from 2 different configuration sources.  The sources we tried were: the server at 'devtlnx0515.stbc2.jstest2.net:9510', and the serve
r at 'devtlnx0516.stbc2.jstest2.net:9510'.  Fetch attempt duration: 60 seconds.

To correct this problem specify a valid configuration location using the -f/--config command-line options.
*******************************************************************************

2022-07-02 16:52:05 BST [ISS.0095.0042I] (tid=94) The ERRSTACKTRACE field in a WMERROR audit record was truncated. CONTEXTID: aed48312-671c-4239-b02f-1d216fcdba8b. MSGI
D: be7c18cb-30a1-176a-e353-e19cdf292f8c. Full value: com.tc.config.schema.setup.ConfigurationSetupException:
*******************************************************************************
Could not fetch configuration data from 2 different configuration sources.  The sources we tried were: the server at 'devtlnx0515.stbc2.jstest2.net:9510', and the serve
r at 'devtlnx0516.stbc2.jstest2.net:9510'.  Fetch attempt duration: 60 seconds.

To correct this problem specify a valid configuration location using the -f/--config command-line options.
*******************************************************************************

at com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.configurationFetchFailed(StandardXMLFileConfigurationCreator.java:375)
at com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.loadConfigDataFromSources(StandardXMLFileConfigurationCreator.java:264)
at com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.loadConfigAndSetIntoRepositories(StandardXMLFileConfigurationCreator.java:141)
at com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.createConfigurationIntoRepositories(StandardXMLFileConfigurationCreator.java:125)
at com.tc.config.schema.setup.BaseConfigurationSetupManager.runConfigurationCreator(BaseConfigurationSetupManager.java:107)
at com.tc.config.schema.setup.L1ConfigurationSetupManagerImpl.<init>(L1ConfigurationSetupManagerImpl.java:63)
at com.tc.config.schema.setup.StandardConfigurationSetupManagerFactory.getL1TVSConfigurationSetupManager(StandardConfigurationSetupManagerFactory.java:243)
at com.tc.object.DistributedObjectClientFactory.create(DistributedObjectClientFactory.java:92)
at com.terracotta.toolkit.express.CreateClient$ClientCreatorCallableImpl.call(CreateClient.java:123)
at com.terracotta.toolkit.express.TerracottaInternalClientImpl.init(TerracottaInternalClientImpl.java:93)
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createInternalToolkit(TerracottaToolkitCreator.java:154)
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createInternalToolkitSynchronously(TerracottaToolkitCreator.java:111)
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createToolkit(TerracottaToolkitCreator.java:78)
at com.terracotta.toolkit.api.EnterpriseTerracottaToolkitFactoryService.createToolkit(EnterpriseTerracottaToolkitFactoryService.java:17)
at com.terracotta.toolkit.api.TerracottaToolkitFactoryService.createToolkit(TerracottaToolkitFactoryService.java:58)
at org.terracotta.toolkit.ToolkitFactory.create(ToolkitFactory.java:100)
at org.terracotta.toolkit.ToolkitFactory.createToolkit(ToolkitFactory.java:84)
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.createToolkit(TerracottaToolkitBuilder.java:63)
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.buildToolkit(TerracottaToolkitBuilder.java:58)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.createTerracottaToolkit(ToolkitInstanceFactoryImpl.java:184)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.<init>(ToolkitInstanceFactoryImpl.java:120)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.<init>(ToolkitInstanceFactoryImpl.java:130)
at org.terracotta.modules.ehcache.EnterpriseToolkitInstanceFactory.<init>(EnterpriseToolkitInstanceFactory.java:23)
at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.createToolkitInstanceFactory(EnterpriseTerracottaClusteredInstanceFactory.j
ava:21)
at org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.<init>(TerracottaClusteredInstanceFactory.java:65)
at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.<init>(EnterpriseTerracottaClusteredInstanceFactory.java:15)
at net.sf.ehcache.terracotta.ExpressEnterpriseTerracottaClusteredInstanceFactory.<init>(ExpressEnterpriseTerracottaClusteredInstanceFactory.java:11)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at net.sf.ehcache.util.ClassLoaderUtil.createNewInstance(ClassLoaderUtil.java:73)
at net.sf.ehcache.terracotta.TerracottaClusteredInstanceHelper.newClusteredInstanceFactory(TerracottaClusteredInstanceHelper.java:157)
at net.sf.ehcache.terracotta.TerracottaClient.createNewClusteredInstanceFactory(TerracottaClient.java:180)
at net.sf.ehcache.terracotta.TerracottaClient.createClusteredInstanceFactory(TerracottaClient.java:129)
at net.sf.ehcache.CacheManager.doInit(CacheManager.java:466)
at net.sf.ehcache.CacheManager.init(CacheManager.java:398)
at net.sf.ehcache.CacheManager.<init>(CacheManager.java:269)
at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1108)
at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1084)
at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:374)
at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:428)
at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:33)
at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:13)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-07-02 16:53:06 BST [ISS.0017.0001E] (tid=94) An error was encountered parsing SoftwareAG-IS-WMN.xml: net.sf.ehcache.CacheException: Unable to load class net.sf.ehc
ache.terracotta.ExpressEnterpriseTerracottaClusteredInstanceFactory. Initial cause was org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException:
Unable to create toolkit.
at net.sf.ehcache.util.ClassLoaderUtil.createNewInstance(ClassLoaderUtil.java:91)
at net.sf.ehcache.terracotta.TerracottaClusteredInstanceHelper.newClusteredInstanceFactory(TerracottaClusteredInstanceHelper.java:157)
at net.sf.ehcache.terracotta.TerracottaClient.createNewClusteredInstanceFactory(TerracottaClient.java:180)
at net.sf.ehcache.terracotta.TerracottaClient.createClusteredInstanceFactory(TerracottaClient.java:129)
at net.sf.ehcache.CacheManager.doInit(CacheManager.java:466)
at net.sf.ehcache.CacheManager.init(CacheManager.java:398)
at net.sf.ehcache.CacheManager.<init>(CacheManager.java:269)
at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1108)
at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1084)
at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:374)
at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:428)
at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:33)
at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:13)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.createToolkit(TerracottaToolkitBuilder.java:65)
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.buildToolkit(TerracottaToolkitBuilder.java:58)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.createTerracottaToolkit(ToolkitInstanceFactoryImpl.java:184)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.<init>(ToolkitInstanceFactoryImpl.java:120)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.<init>(ToolkitInstanceFactoryImpl.java:130)
at org.terracotta.modules.ehcache.EnterpriseToolkitInstanceFactory.<init>(EnterpriseToolkitInstanceFactory.java:23)
at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.createToolkitInstanceFactory(EnterpriseTerracottaClusteredInstanceFactory.j
ava:21)
at org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.<init>(TerracottaClusteredInstanceFactory.java:65)
at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.<init>(EnterpriseTerracottaClusteredInstanceFactory.java:15)
at net.sf.ehcache.terracotta.ExpressEnterpriseTerracottaClusteredInstanceFactory.<init>(ExpressEnterpriseTerracottaClusteredInstanceFactory.java:11)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at net.sf.ehcache.util.ClassLoaderUtil.createNewInstance(ClassLoaderUtil.java:73)
... 16 more
Caused by: org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
at com.terracotta.toolkit.api.TerracottaToolkitFactoryService.createToolkit(TerracottaToolkitFactoryService.java:63)
at org.terracotta.toolkit.ToolkitFactory.create(ToolkitFactory.java:100)
at org.terracotta.toolkit.ToolkitFactory.createToolkit(ToolkitFactory.java:84)
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.createToolkit(TerracottaToolkitBuilder.java:63)
... 30 more
Caused by: java.lang.RuntimeException: Unable to create toolkit.
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createToolkit(TerracottaToolkitCreator.java:95)
at com.terracotta.toolkit.api.EnterpriseTerracottaToolkitFactoryService.createToolkit(EnterpriseTerracottaToolkitFactoryService.java:17)
at com.terracotta.toolkit.api.TerracottaToolkitFactoryService.createToolkit(TerracottaToolkitFactoryService.java:58)
... 33 more
Caused by: java.lang.RuntimeException: com.tc.config.schema.setup.ConfigurationSetupException:
*******************************************************************************
Could not fetch configuration data from 2 different configuration sources.  The sources we tried were: the server at 'devtlnx0515.stbc2.jstest2.net:9510', and the serve
r at 'devtlnx0516.stbc2.jstest2.net:9510'.  Fetch attempt duration: 60 seconds.

To correct this problem specify a valid configuration location using the -f/--config command-line options.

Have you installed latest fixes for the products

yes

As per existing blogs, i have tried and added the terracotta license key to ISpath/common/conf but its not working.

Any suggestions please

Cache Config - The error complains of “pre-formatted text”, in the SoftwareAG-IS-Core.xml file.
Was this file copied from somewhere else, or manually edited?

Can you review the content of this file, or attach the file here?
/yourInstallationPath/IntegrationServer/instances/instanceName/config/Caching/SoftwareAG-IS-Core.xml

License Key - In addition, double-check if you have the Terracotta license key at the right place, and if a restart was performed after updating the license.

KM

Hello Kasi,

Thanks for your response
I have placed the key at right location /instancename/common/conf .

Today i have reinstalled the IS but still im facing the same issue.Please find the aatched logs and required files

IS Logs:

2022-07-06 11:15:02 BST [ISS.0033.0168I] (tid=93) Cluster Node Name: devtlnx0515.stbc2.jstest2.net.
2022-07-06 11:15:02 BST [ISS.0033.0104I] (tid=93) Could not add server to the cluster. net.sf.ehcache.constructs.nonstop.NonStopCacheException: put timed out
2022-07-06 11:15:02 BST [ISS.0033.0103W] (tid=93) Could not open the session cache. net.sf.ehcache.constructs.nonstop.NonStopCacheException: put timed out
2022-07-06 11:15:02 BST [ISS.0045.9998E] (tid=93) Exception --> Exception Loading impl (org.apache.lucene.codecs.lucene87.Lucene87Codec) for service (org.apache.lucene.codecs.Codec) via java.net.URLClassLoader@dc1078b: org.apache.lucene.codecs.lucene87.Lucene87Codec

2022-07-06 11:15:02 BST [ISS.0017.0001E] (tid=93) An error was encountered parsing SoftwareAG-IS-WMN.xml: net.sf.ehcache.CacheException: Unable to load class net.sf.ehcache.terracotta.ExpressEnterpriseTerracottaClusteredInstanceFactory. Initial cause was org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
at net.sf.ehcache.util.ClassLoaderUtil.createNewInstance(ClassLoaderUtil.java:91)
at net.sf.ehcache.terracotta.TerracottaClusteredInstanceHelper.newClusteredInstanceFactory(TerracottaClusteredInstanceHelper.java:157)
at net.sf.ehcache.terracotta.TerracottaClient.createNewClusteredInstanceFactory(TerracottaClient.java:180)
at net.sf.ehcache.terracotta.TerracottaClient.createClusteredInstanceFactory(TerracottaClient.java:129)
at net.sf.ehcache.CacheManager.doInit(CacheManager.java:466)
at net.sf.ehcache.CacheManager.init(CacheManager.java:398)
at net.sf.ehcache.CacheManager.<init>(CacheManager.java:269)
at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1108)
at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1084)
at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:374)
at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:428)
at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:33)
at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:13)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.createToolkit(TerracottaToolkitBuilder.java:65)
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.buildToolkit(TerracottaToolkitBuilder.java:58)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.createTerracottaToolkit(ToolkitInstanceFactoryImpl.java:184)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.<init>(ToolkitInstanceFactoryImpl.java:120)
at org.terracotta.modules.ehcache.ToolkitInstanceFactoryImpl.<init>(ToolkitInstanceFactoryImpl.java:130)
at org.terracotta.modules.ehcache.EnterpriseToolkitInstanceFactory.<init>(EnterpriseToolkitInstanceFactory.java:23)
at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.createToolkitInstanceFactory(EnterpriseTerracottaClusteredInstanceFactory.java:21)
at org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.<init>(TerracottaClusteredInstanceFactory.java:65)
at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.<init>(EnterpriseTerracottaClusteredInstanceFactory.java:15)
at net.sf.ehcache.terracotta.ExpressEnterpriseTerracottaClusteredInstanceFactory.<init>(ExpressEnterpriseTerracottaClusteredInstanceFactory.java:11)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at net.sf.ehcache.util.ClassLoaderUtil.createNewInstance(ClassLoaderUtil.java:73)
... 16 more
Caused by: org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
at com.terracotta.toolkit.api.TerracottaToolkitFactoryService.createToolkit(TerracottaToolkitFactoryService.java:63)
at org.terracotta.toolkit.ToolkitFactory.create(ToolkitFactory.java:100)
at org.terracotta.toolkit.ToolkitFactory.createToolkit(ToolkitFactory.java:84)
at org.terracotta.modules.ehcache.TerracottaToolkitBuilder.createToolkit(TerracottaToolkitBuilder.java:63)
... 30 more
Caused by: java.lang.RuntimeException: Unable to create toolkit.
at com.terracotta.toolkit.client.TerracottaToolkitCreator.createToolkit(TerracottaToolkitCreator.java:95)
at com.terracotta.toolkit.api.EnterpriseTerracottaToolkitFactoryService.createToolkit(EnterpriseTerracottaToolkitFactoryService.java:17)
at com.terracotta.toolkit.api.TerracottaToolkitFactoryService.createToolkit(TerracottaToolkitFactoryService.java:58)
... 33 more
Caused by: java.lang.RuntimeException: com.tc.config.schema.setup.ConfigurationSetupException:
*******************************************************************************
Could not fetch configuration data from 2 different configuration sources.  The sources we tried were: the server at 'devtlnx0515.stbc2.jstest2.net:9510', and the server at 'devtlnx0516.stbc2.jstest2.net:9510'.  Fetch attempt duration: 60 seconds.

Terracotta logs:

2022-07-06 11:13:01,172 [WorkerThread(client_handshake_stage, 0)] INFO com.tc.objectserver.handshakemanager.ServerClientHandshakeManager - Sending handshake acknowledge
ment to ClientID[22]
2022-07-06 11:13:01,172 [WorkerThread(client_handshake_stage, 0)] INFO tc.operator.event - NODE : devtlnx0516 Subsystem: CLUSTER_TOPOLOGY EventType: TOPOLOGY_NODE_JOINE
D Message: Node ClientID[22] joined the cluster
2022-07-06 11:13:01,177 [WorkerThread(jmxremote_tunnel_stage, 0)] INFO com.tc.management.remote.protocol.terracotta.ClientTunnelingEventHandler - L1[ChannelID=[22]] not
ified us that their JMX server is now available
2022-07-06 11:13:01,177 [WorkerThread(jmxremote_connect_stage, 0)] INFO com.tc.management.remote.connect.ClientConnectEventHandler - addJmxConnection(ChannelID=[22])
2022-07-06 11:13:01,299 [WorkerThread(jmxremote_connect_stage, 0)] INFO com.tc.management.remote.connect.ClientConnectEventHandler - addJmxConnection(ChannelID=[22]) co
mpleted in 122 millis
2022-07-06 11:13:01,989 [WorkerThread(apply_changes_stage, 2, 2)] ERROR com.tc.server.TCServerMain - Thread:Thread[WorkerThread(apply_changes_stage, 2, 2),5,TC Thread G
roup] got an uncaught exception. calling CallbackOnExitDefaultHandlers.
com.tc.exception.TCRuntimeException: Uncaught exception in stage
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:211)
Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer
at com.tc.objectserver.managedobject.ConcurrentDistributedServerMapManagedObjectState.applyRegisterServerEventListener(ConcurrentDistributedServerMapManagedObje
ctState.java:593)
at com.tc.objectserver.managedobject.ConcurrentDistributedServerMapManagedObjectState.applyLogicalAction(ConcurrentDistributedServerMapManagedObjectState.java:3
03)
at com.tc.objectserver.managedobject.ConcurrentDistributedServerMapManagedObjectState.apply(ConcurrentDistributedServerMapManagedObjectState.java:187)
at com.tc.objectserver.managedobject.ManagedObjectImpl.apply(ManagedObjectImpl.java:199)
at com.tc.objectserver.tx.ServerTransactionManagerImpl.apply(ServerTransactionManagerImpl.java:375)
at com.tc.objectserver.handler.ApplyTransactionChangeHandler.handleEvent(ApplyTransactionChangeHandler.java:123)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:188)
2022-07-06 11:13:01,989 [WorkerThread(apply_changes_stage, 3, 3)] ERROR com.tc.server.TCServerMain - Thread:Thread[WorkerThread(apply_changes_stage, 3, 3),5,TC Thread G
roup] got an uncaught exception. calling CallbackOnExitDefaultHandlers.
com.tc.exception.TCRuntimeException: Uncaught exception in stage
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:211)
Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer
at com.tc.objectserver.managedobject.ConcurrentDistributedServerMapManagedObjectState.applyRegisterServerEventListener(ConcurrentDistributedServerMapManagedObje
ctState.java:593)
at com.tc.objectserver.managedobject.ConcurrentDistributedServerMapManagedObjectState.applyLogicalAction(ConcurrentDistributedServerMapManagedObjectState.java:3
03)
at com.tc.objectserver.managedobject.ConcurrentDistributedServerMapManagedObjectState.apply(ConcurrentDistributedServerMapManagedObjectState.java:187)
at com.tc.objectserver.managedobject.ManagedObjectImpl.apply(ManagedObjectImpl.java:199)
at com.tc.objectserver.tx.ServerTransactionManagerImpl.apply(ServerTransactionManagerImpl.java:375)
at com.tc.objectserver.handler.ApplyTransactionChangeHandler.handleEvent(ApplyTransactionChangeHandler.java:123)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:188)
2022-07-06 11:13:02,068 [WorkerThread(apply_changes_stage, 5, 5)] ERROR com.tc.server.TCServerMain - Thread:Thread[WorkerThread(apply_changes_stage, 5, 5),5,TC Thread G
roup] got an uncaught exception. calling CallbackOnExitDefaultHandlers.

SoftwareAG-IS-Core.xml (7.3 KB)
SoftwareAG-IS-WMN.xml (1.6 KB)

The error “Unable to create toolkit” tells me that you may have made the same mistake I had done a few years ago.

For Clustering, you must install “BigMemory” (and use the BigMemory license key) and not Terracotta

KM

Hello Kasi.

But i have installed the integration server and bring it to cluster manually via CC console without Bigmemory successfully and Is were working fine in cluster.This issue comes when i try to install the integration server via CC composite template.

I feel that there is some differences are there when we installing the integration server via CC gui and CC composite template

On a deeper look at the logs, I notice that the IS is perhaps unable to reach (or connect to) either BM node -

Could not fetch configuration data from 2 different configuration sources. The sources we tried were: the server at 'devtlnx0515.stbc2.jstest2.net:9510', and the server at 'devtlnx0516.stbc2.jstest2.net:9510'. Fetch attempt duration: 60 seconds.

1. Check if the DNS:IP values are correct
2. Check if both these targets are reachable from both your IS instances

If that doesn’t solve it, do post the tc-config.xml as well.

KM

There are threads on the forums that question the need for an IS cluster. I know this doesn’t address the issue you’ve posted about but perhaps one “solution” to the problem is to not use IS clustering. You may not need it. Review the docs about what it provides specifically. Chances are you can get by with a load-balancer, and shared DB and not need IS cluster.

Hi Saran,

Can you please copy the Bigmemory license file into /app/opt/sag/webMethods/SITISA/common/conf and restart it and check.

Regards,
Nayan

Hello Nayan,

we dont have Bigmemory licsense and clustering is working without this key when we provisioning the Integration server manually in CC GUI

Thanks you all for your suggestions.The issue here is the fix which we havent applied after the terracotta installation. we have updated the latest fix and restarted the server and the issue was resolved.