Hi experts,

We got to know our Integration server is down and we tried checking the servers logs, but there is no logs.
I restarted the server, in logs i can see config directory saved and process is running, But IS console is not coming up, in the logs i can see

2022-05-20 01:52:18 PDT [ISS.0014.0025W] Master password for outbound password encryption has expired.
2022-05-20 01:12:15 PDT [ISS.0025.0016I] Config File Directory Saved
2022-05-20 01:13:40 PDT [ISP.0061.0002I] Expired remote connection to ma_b2bt_lapp858662 for user Administrator
2022-05-20 01:13:40 PDT [ISP.0061.0002I] Expired remote connection to ma_b2bt_lapp858662 for user Administrator
2022-05-20 01:13:40 PDT [ISP.0061.0002I] Expired remote connection to ma_b2bt_lapp858662 for user Administrator

Could you please help me to bring back the IS ?

Best Regards
K M,Varun

Hi @varunkmv
If you see “Config File Directory Saved” , it means that the server is up,
you can check for the message [ISP.0046.0012I] Enabling HTTP Listener on port num
You can try connecting to the port from different clients like a browser / postman to confirm that the IS is indeed not up.

The logs that you see are either warnings or Informational messages 0025W and 0002I , you can check if there are more logs which clearly show an error. You could also check the /profiles/IS_default/logs wrapper.log files.

-NP

ENTRY org.apache.felix.scr 4 0 2022-05-20 02:38:40.130
INFO | jvm 1 | 2022/05/20 02:38:40 | !MESSAGE bundle org.apache.felix.scr:2.1.16 (250)Circular reference detected trying to get service {com.softwareag.platform.bundle.watchdog.BundleWatchdog}={service.id=224, directory=/ngs/app/b2bit/apps/wm105/profiles/IS_default/workspace/app-platform/deployer/bundles, service.bundleid=105, service.scope=bundle, TokenResolver.target=(service.pid=location.token.resolver), component.name=com.softwareag.platform.bundle.watchdog.internal.BundleWatchdogImpl, component.id=12, com.softwareag.config.loader.pid=file.property.loader, service.factoryPid=com.softwareag.bundle.watchdog.pid, com.softwareag.config.source=file:/ngs/app/b2bit/apps/wm105/profiles/IS_default/configuration/com.softwareag.platform.config.propsloader/com.softwareag.bundle.watchdog.pid-AppPlatform.properties, com.softwareag.config.timestamp=1566229524000, service.pid=com.softwareag.bundle.watchdog.pid-1653036522811-10, bundle.watchdog.name=AppPlatform}
INFO | jvm 1 | 2022/05/20 02:38:40 | stack of references: ServiceReference: {com.softwareag.platform.bundle.watchdog.BundleWatchdog}={service.id=224, directory=/ngs/app/b2bit/apps/wm105/profiles/IS_default/workspace/app-platform/deployer/bundles, service.bundleid=105, service.scope=bundle, TokenResolver.target=(service.pid=location.token.resolver), component.name=com.softwareag.platform.bundle.watchdog.internal.BundleWatchdogImpl, component.id=12, com.softwareag.config.loader.pid=file.property.loader, service.factoryPid=com.softwareag.bundle.watchdog.pid, com.softwareag.config.source=file:/ngs/app/b2bit/apps/wm105/profiles/IS_default/configuration/com.softwareag.platform.config.propsloader/com.softwareag.bundle.watchdog.pid-AppPlatform.properties, com.softwareag.config.timestamp=1566229524000, service.pid=com.softwareag.bundle.watchdog.pid-1653036522811-10, bundle.watchdog.name=AppPlatform}
INFO | jvm 1 | 2022/05/20 02:38:40 | ServiceReference: {com.softwareag.platform.bundle.watchdog.internal.FrameworkRefresher}={service.id=223, service.bundleid=105, service.scope=bundle, component.name=com.softwareag.platform.bundle.watchdog.internal.FrameworkRefresher, component.id=14}
INFO | jvm 1 | 2022/05/20 02:38:40 |
INFO | jvm 1 | 2022/05/20 02:38:40 | !STACK 0
INFO | jvm 1 | 2022/05/20 02:38:40 | java.lang.Exception: stack trace
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.ComponentRegistry.enterCreate(ComponentRegistry.java:485)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.BundleComponentActivator.enterCreate(BundleComponentActivator.java:734)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:881)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse$1.run(ServiceFactoryUse.java:215)
INFO | jvm 1 | 2022/05/20 02:38:40 | at java.security.AccessController.doPrivileged(Native Method)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse.factoryGetService(ServiceFactoryUse.java:213)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse.getService(ServiceFactoryUse.java:114)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceConsumer$2.getService(ServiceConsumer.java:48)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.getService(ServiceRegistrationImpl.java:527)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.getService(ServiceRegistry.java:464)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.BundleContextImpl.getService(BundleContextImpl.java:661)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.SingleRefPair.getServiceObject(SingleRefPair.java:86)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.inject.BindParameters.getServiceObject(BindParameters.java:47)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.inject.methods.BindMethod.getServiceObject(BindMethod.java:664)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.DependencyManager.getServiceObject(DependencyManager.java:2308)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.DependencyManager$MultipleDynamicCustomizer.prebind(DependencyManager.java:411)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.DependencyManager.prebind(DependencyManager.java:1568)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager.collectDependencies(AbstractComponentManager.java:1029)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:935)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:900)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse$1.run(ServiceFactoryUse.java:215)
INFO | jvm 1 | 2022/05/20 02:38:40 | at java.security.AccessController.doPrivileged(Native Method)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse.factoryGetService(ServiceFactoryUse.java:213)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse.getService(ServiceFactoryUse.java:114)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceConsumer$2.getService(ServiceConsumer.java:48)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.getService(ServiceRegistrationImpl.java:527)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.getService(ServiceRegistry.java:464)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.BundleContextImpl.getService(BundleContextImpl.java:661)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.SingleRefPair.getServiceObject(SingleRefPair.java:86)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.inject.BindParameters.getServiceObject(BindParameters.java:47)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.inject.methods.BindMethod.getServiceObject(BindMethod.java:664)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.DependencyManager.getServiceObject(DependencyManager.java:2308)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.prebind(DependencyManager.java:1154)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.DependencyManager.prebind(DependencyManager.java:1568)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager.collectDependencies(AbstractComponentManager.java:1029)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:935)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:756)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.addedService(DependencyManager.java:1045)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.addedService(DependencyManager.java:999)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1216)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.customizerAdded(ServiceTracker.java:1137)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.trackAdding(ServiceTracker.java:944)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.ServiceTracker$AbstractTracked.track(ServiceTracker.java:880)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:1168)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.BundleComponentActivator$ListenerInfo.serviceChanged(BundleComponentActivator.java:125)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:112)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:956)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:233)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:151)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:865)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:804)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:228)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:506)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.register(AbstractComponentManager.java:906)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.register(AbstractComponentManager.java:892)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.RegistrationManager.changeRegistration(RegistrationManager.java:128)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:959)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:732)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:666)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:432)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:665)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:338)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:382)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.Activator.access$200(Activator.java:49)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:264)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.AbstractExtender.createExtension(AbstractExtender.java:196)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:169)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.AbstractExtender.addingBundle(AbstractExtender.java:139)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.AbstractExtender.addingBundle(AbstractExtender.java:49)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.osgi.util.tracker.BundleTracker$Tracked.customizerAdding(BundleTracker.java:475)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.osgi.util.tracker.BundleTracker$Tracked.customizerAdding(BundleTracker.java:420)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.osgi.util.tracker.AbstractTracked.trackInitial(AbstractTracked.java:183)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.osgi.util.tracker.BundleTracker.open(BundleTracker.java:159)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.AbstractExtender.startTracking(AbstractExtender.java:100)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.AbstractExtender.doStart(AbstractExtender.java:92)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.Activator.doStart(Activator.java:172)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.AbstractExtender.start(AbstractExtender.java:72)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.Activator.restart(Activator.java:139)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.config.ScrConfigurationImpl.configure(ScrConfigurationImpl.java:217)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.config.ScrConfigurationImpl.start(ScrConfigurationImpl.java:120)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.apache.felix.scr.impl.Activator.start(Activator.java:100)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:815)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.BundleContextImpl$3.run(BundleContextImpl.java:808)
INFO | jvm 1 | 2022/05/20 02:38:40 | at java.security.AccessController.doPrivileged(Native Method)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.BundleContextImpl.startActivator(BundleContextImpl.java:808)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.BundleContextImpl.start(BundleContextImpl.java:765)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.EquinoxBundle.startWorker0(EquinoxBundle.java:1005)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.internal.framework.EquinoxBundle$EquinoxModule.startWorker(EquinoxBundle.java:357)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.container.Module.doStart(Module.java:589)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.container.Module.start(Module.java:457)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1784)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1764)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.doContainerStartLevel(ModuleContainer.java:1726)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1657)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1558)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:233)
INFO | jvm 1 | 2022/05/20 02:38:40 | at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:343)
INFO | jvm 1 | 2022/05/20 02:39:00 | [Fri May 20 02:39:00.675 PDT 2022] [webMethods Integration Server] Setting log level to ERROR
INFO | jvm 1 | 2022/05/20 02:39:00 | --------- Log File Closed ---------

I can see only this one as suspicious log from wrapper.log … remaining all logs are relevant to the insert SQL Queries.

Also i tried from the different browsers , IS is not reachable

You’ve mentioned that the IS process is running, is restarting an option for you?

Do you see this message in the server logs?
And when you try to reach IS , are there any messages being generated in the server logs?
-NP

yes i can see this message Enabling HTTP Listener on port 8662 … But here 8662 is not our primary port number.

Our Primary port is 8668 , So in the logs it should be right …
But in logs i dont see this port … only i can see for 8662
2022-05-20 02:41:58 PDT [TNS.0005.1042I] Warning: Trying to register delivery service FTP AMEX to BOA that is already registered
2022-05-20 02:41:58 PDT [ISS.0028.0012I] AppleTNInfrastructure: Startup service (com.apple.b2b.tninfrastructure.sap:registerRouteOnly)
2022-05-20 02:41:58 PDT [ISS.0028.0013E] Startup service exited with the following error: java.lang.NullPointerException
2022-05-20 02:41:58 PDT [ISS.0028.0012I] AppleTNInfrastructure: Startup service (com.apple.b2b.tninfrastructure.webService.soapProcessor:registerCustomSOAPmsgProcessor)
2022-05-20 02:41:58 PDT [ISP.0046.0012I] Enabling HTTP Listener on port 8662
2022-05-20 02:41:58 PDT [ISS.0098.0021I] Persistent Trigger Output Dispatcher started
2022-05-20 02:41:58 PDT [ISS.0098.0021I] Volatile Trigger Output Dispatcher started
2022-05-20 02:41:58 PDT [ISS.0098.0027I] PersistenceManager started all Stores
2022-05-20 02:41:58 PDT [ISS.0025.0036I] Dispatcher started
2022-05-20 02:41:58 PDT [ISS.0025.0013I] Cache Sweeper started
2022-05-20 02:41:58 PDT [ISS.0035.0003I] Log rotation interval set to 1440 minutes.
2022-05-20 02:41:58 PDT [ISS.0014.0025W] Master password for outbound password encryption has expired. Password was last changed on 2020-03-27 11:21:01 PDT. The current expiration interval is 90 days
2022-05-20 02:41:58 PDT [ISS.0025.0057I] Reading system level information
2022-05-20 02:41:58 PDT [ISS.0014.0002I] Initialization completed in 192 seconds.
2022-05-20 02:41:58 PDT [SAP.0118.0001W] Adapter is not using Centralized Transaction Store.
2022-05-20 02:41:58 PDT [ISS.0025.0016I] Config File Directory Saved

You can connect to the port that is enabled and check the Security->Ports screen to check for the primary port that doesn’t seem to be enabled.
-NP

Resolution: listeners.cnf file is corrupted, ports are changed. We have corrected that file and restarted the server.
It works