Product/components used and version/fix level:

webMethods Integration Server 10.3

Detailed explanation of the problem:

Hello.

Our IS stopped running the scheduled tasks after the last restart in order to import the MySQL Connector JAR into the IS’s classpath.

Scheduled tasks in IS doesn’t run - it stucks “Next Run” on “0.0” without logging to server.log

image723×38 2.99 KB

Please, is there something that I must do in other to check how to fix it?

Also, it started to echo the following log (it wasn’t happening before):

Error messages / full error message screenshot / log file:

2023-06-21 15:20:31 GMT-03:00 [ISS.0137.0000I] (tid=833) [SCHED] IS 10.3, scheduler version: 10.3.2
2023-06-21 15:20:31 GMT-03:00 [ISS.0137.0000I] (tid=833) [SCHED] checking DB level
2023-06-21 15:21:31 GMT-03:00 [ISS.0137.0000I] (tid=833) [SCHED] IS 10.3, scheduler version: 10.3.2
2023-06-21 15:21:31 GMT-03:00 [ISS.0137.0000I] (tid=833) [SCHED] checking DB level
2023-06-21 15:22:31 GMT-03:00 [ISS.0137.0000I] (tid=833) [SCHED] IS 10.3, scheduler version: 10.3.2
2023-06-21 15:22:31 GMT-03:00 [ISS.0137.0000I] (tid=833) [SCHED] checking DB level

Question related to a free trial, or to a production (customer) instance?

Customer Development IS

please check if IS_10.3_Core_Fix12 is installed or not. Installing the latest fixes should resolve this problem. Check this empower ticket for referance.

Hello Engin thanks for the response.

I tried to install the Fix12 but I’m receiving the following error in UpdateManager.sh:

2023/06/21 16:50:52  INFO    | Connecting to sdc.softwareag.com
2023/06/21 16:51:23  INFO    |
2023/06/21 16:51:23  SEVERE    o-------------------LAUNCHER_ERROR-------------------o
2023/06/21 16:51:23  SEVERE    o Message: Network issue. Please, check your parameters and retry again. The connection cannot be established to https://sdc.softwareag.com/sumv2/api/core/update:
2023/06/21 16:51:23  SEVERE    o Exception: java.io.IOException
2023/06/21 16:51:23  SEVERE    o Stacktrace:
2023/06/21 16:51:23  SEVERE    o com.softwareag.plm.sum.client.launcher.utils.Downloader.getInformation(Downloader.java:471)
2023/06/21 16:51:23  SEVERE    o com.softwareag.plm.sum.client.launcher.steps.AbstractConnectionAction.connect(AbstractConnectionAction.java:62)
2023/06/21 16:51:23  SEVERE    o com.softwareag.plm.sum.client.launcher.steps.AbstractConnectionAction.run(AbstractConnectionAction.java:108)
2023/06/21 16:51:23  SEVERE    o com.softwareag.plm.sum.client.launcher.steps.AbstractAction.begin(AbstractAction.java:85)
2023/06/21 16:51:23  SEVERE    o com.softwareag.plm.sum.client.launcher.steps.StepManager.executeSteps(StepManager.java:83)
2023/06/21 16:51:23  SEVERE    o com.softwareag.plm.sum.client.launcher.Launcher.main(Launcher.java:53)
2023/06/21 16:51:23  SEVERE    o----------------------------------------------------o
2023/06/21 16:51:23  SEVERE    o====================================================o
2023/06/21 16:51:23  SEVERE    Exit code: 171 Error in communicating with the server.
2023/06/21 16:51:23  INFO    |
2023/06/21 16:51:23  INFO    | Exit from Launcher: 171
2023/06/21 16:51:23  INFO    |

Did something change with “sdc.software.com”?

It was working some months ago.

Thanks in advance!

If you company uses proxy add that configuration to SUM or ask your IT to white list sdc.softwareag.com. There might be some redirects from that server, you need to whitelist them all. Currently we have the same version and SUM 10.1 has some weird issue on my computer. Support couldn’t fix it in past 6 months. You may need to login using a local admin account and run SUM as an admin, it may fix your problem.

Also fixes are cumulative, I recommend installing the latest fixes to every product not just an old fix. Some of the fixes are security related so its always a good idea to keep your environment up to date. Also 10.3 support has ended, You need to upgrade to a later version (10.15 is preferred or a newer version if there is one when you start your upgrade).

use this url to test your connection to https://sdc.softwareag.com/sumv2/api/version

Thanks again for your return Engin!

I did the test with the https://sdc.softwareag.com/sumv2/api/core/update but it returned that “Full authentication required” when doing with curl.

Isn’t this authentication causing the above exception?
Can I pass manually our empower credentials when starting UpdateManagerCMD? Via the -empowerUser user_name -empowerPass parameters?

I’ve also asked to one of our infra guys and he said that the sdc.softwareag.com isn’t blocked in our proxy.

Here is more logs from UpdateManager:

2023/06/22 13:22:18  INFO    | Connecting to sdc.softwareag.com
2023/06/22 13:22:18  FINEST  | JSON with given arguments: { "java": "java-LNXAMD64_10.1.0.0000-0033" ,"equinox": "equinox-LNXAMD64_10.0.0.0000-0053" ,"core": "core_10.0.0.0000-0054" ,"launcher": "launcher_10.1.0.0000-0308" ,"platform": "LNXAMD64" ,"version": "10.1.0.0000-0021"  }
2023/06/22 13:22:18  FINEST  | The following URL is used: https://sdc.softwareag.com/sumv2/api/core/update
2023/06/22 13:22:18  FINEST  | followRedirects url = https://sdc.softwareag.com/sumv2/api/core/update
2023/06/22 13:22:18  FINEST  | followRedirects CURRENT url = https://sdc.softwareag.com/sumv2/api/core/update
2023/06/22 13:22:18  FINEST  | Sending JSON as input: { "java": "java-LNXAMD64_10.1.0.0000-0033" ,"equinox": "equinox-LNXAMD64_10.0.0.0000-0053" ,"core": "core_10.0.0.0000-0054" ,"launcher": "launcher_10.1.0.0000-0308" ,"platform": "LNXAMD64" ,"version": "10.1.0.0000-0021"  }
2023/06/22 13:22:51  FINEST  |
2023/06/22 13:22:51  FINEST  | java.net.SocketException was caught while trying to connect to the server!
2023/06/22 13:22:51  FINEST  | Message: java.net.SocketException: Unexpected end of file from server
2023/06/22 13:22:51  INFO    |
2023/06/22 13:22:51  SEVERE    o-------------------LAUNCHER_ERROR-------------------o
2023/06/22 13:22:51  SEVERE    o Message: Network issue. Please, check your parameters and retry again. The connection cannot be established to https://sdc.softwareag.com/sumv2/api/core/update:
2023/06/22 13:22:51  SEVERE    o Exception: java.io.IOException
2023/06/22 13:22:51  SEVERE    o Stacktrace:
2023/06/22 13:22:51  SEVERE    o com.softwareag.plm.sum.client.launcher.utils.Downloader.getInformation(Downloader.java:471)
2023/06/22 13:22:51  SEVERE    o com.softwareag.plm.sum.client.launcher.steps.AbstractConnectionAction.connect(AbstractConnectionAction.java:62)
2023/06/22 13:22:51  SEVERE    o com.softwareag.plm.sum.client.launcher.steps.AbstractConnectionAction.run(AbstractConnectionAction.java:108)
2023/06/22 13:22:51  SEVERE    o com.softwareag.plm.sum.client.launcher.steps.AbstractAction.begin(AbstractAction.java:85)
2023/06/22 13:22:51  SEVERE    o com.softwareag.plm.sum.client.launcher.steps.StepManager.executeSteps(StepManager.java:83)
2023/06/22 13:22:51  SEVERE    o com.softwareag.plm.sum.client.launcher.Launcher.main(Launcher.java:53)
2023/06/22 13:22:51  SEVERE    o----------------------------------------------------o
2023/06/22 13:22:51  FINEST EXIT SendVersions < run(

Tbh, easier and recommended way of installing fixes is to create a fix image and use that image to install to the target servers. If you want to install them without an image then you need to do it monthly for the latest fixes available for all servers, even then its better to do it with an image especially if the target servers are UNIX because sometimes you can’t use x-window in production servers and the other reason is you want every server has the same fix level. Therefore I will recommend downloading the fixes on to your local machine and use that image to install fixes.

I forgot to tell you, that service response looks OK. If you have firefox installed on that server and if that server has x-window installed, you can forward firefox to your local terminal and check that url from firefox. If you don’t know how to forward display port to your local terminal you can use MobaXterm instead of putty. It will automatically configure it and it will tell you it’s not supported on the login page if it is not installed.

Our IS is installed on a Red Hat 7.x - unfortunately, we don’t have the graphical interfaces installed for it.

I’m still talking to our Infra team to try to resolve the above network error.

This is what is described in “About” page of our IS Admin:

Software
Product webMethods Integration Server
Version 10.3.0.0
Updates IS_10.3_Core_Fix11
IS_10.3_SPM_Fix6
Build Number 118

So, I installed the Update Manager to try to create the image with the following:
went to “Advanced Options”
“Images” > “Use installation Image” > “Create Image”
Checked “Include product fixes”
Set the file path and name to save the image
Set Image platform to “Linux RHEL and SLES x86-64”
In the next steps, I checked to download the IS 10.3 Core Fix #19
The image was created succesfully…

With the above, the next step now is to use the UpdateManagerCMD.sh and point to the generated image file in our QAS Integration Server, is this correct?

Please, correct me if I’m wrong.

Correct.

You can create a SUM update script to make it easier and less painful. UpdateManager can create the script for you but the directory structure must be same for every server you are using the script on.

Engin,

Our infra team managed to solve the network issue.

But, when selecting the updates to be installed, some of them are already checked and I can’t uncheck them, even without checking the update for IS Core, as you can see below:

Update Manager is about to install the fixes you selected and their related fixes.
The fixes marked with an asterisk (*) have steps you must perform before installing.
Install Directory:/WebM/IS01

1)    Common Library
2)      WGL 10.3 OSGI Glue Fix 3(Reinstall)
3)    Database Configuration
4)      DC_10.3_DCC_Fix9(related)
5)    Database Driver Bundles
6)      Shared Bundles DataDirect 10.3 Fix 4
7)    Database Driver Libraries
8)      Shared Libraries DataDirect 10.3 Fix 4(related)
9)    Infrastructure
10)     Shared Platform
11)       Security Infrastructure 10.3 Fix 8
12)       Platform 10.3.0 Fix 0013
13)       Bundles
14)         * Broker Bundles 9.6 Fix 16 *
15)         Elasticsearch Bundles 5.6.4 10.3 Fix 2
16)         Terracotta Bundles 4.3.6 Fix 15 (Upgrades to 4.3.9)
17)         Digital Event Services
18)           Digital Event Services Runtime Bundles 10.3 Fix 1(Reinstall)
19)           Digital Event Services Shared Bundles 10.3 Fix 1(Reinstall)
20)         Event Routing
21)           Event Routing Runtime Bundles 10.3 Fix 1(Reinstall)
22)       Web Services Stack
23)         Web Services Stack Core 10.3.0 Fix 14(related)
24)         Web Services Stack OSGi 10.3.0 Fix 14
25)   My webMethods Server
26)     My webMethods Server 10.3 Fix 20(related)
27)   Shared Bundles
28)     Universal Messaging Shared Bundles 10.3 Fix 23
29)     Shared Bundles ApacheCommons 10.3 Fix 3
30)     Shared Bundles Commons 10.3 Fix 2
31)     Shared Bundles 10.3 XMLTools Fix 3(Reinstall)

I wouldn’t like to install the packages above - only the IS Core fix.

The reason for that is that some fixes may require further checking, and we would like to solve only this issue.

Please, do you know if there is a way to do it?

Thanks again in advance!

@Renan_Lopes1

First we need to talk about what (REINSTALL) and (RELATED) mean for you.

The later (RELATED) is a depending fix that is missing in order to properly run the fix you have chose to be added. Installing them is unavoidable.
For REINSTALL this is even easier, the fix is already installed on your server, and the SUM tools knows that when installing the new fixes you wanted relevant fixes would be lost. Therefore once the new fixes (and dependencies) are installed system will apply the changes one more.
The main reason for even seeing all this is an old installation where you missed to regularly install all available fixes (recommendation by SAG).

Some fixes depend on other fixes on other products and sometimes on to db scripts so you may need to run db scripts again.

Fixing only this issue doesn’t mean much. Some of these fixes are security related and you should always keep the latest available fixes installed on your servers and also fixes are cumulative. It means you can’t skip fix 3 to install fix 5. fix 5 will install fixes from 1 to 5.

If you insist on installing only this fix ignore that message, select that fix, let it include the dependent fixes as well. Read and take a copy of the warning message, it will also pop up after you install. Take note of the red parts and read carefully. Apply the required fixes to other products as told.

You don’t need to be too cautious when installing fixes. You don’t need to test it like you test a new environment. Just patch your test servers, let it run for awhile, and then patch production.

The only important thing to be careful while installing fixes is to read the fix recommendations carefully and apply them as necessary.

Hello guys!

We successfully updated our IS to fix #19 provided by Update Manager and the fix did solve the issue with the scheduled tasks in our QAS Integration Server.

Thank you for all advices provided !

We will create a image with this fix to be deployed to production in 2 weeks.

But, some errors occurred during the fix installation, I’m not sure if this is related:

https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files3/4fffb7b4ff98f1d40e5d9ae52d3203681a2d76f0_2_1035x400.png 1.5x, https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files3/4fffb7b4ff98f1d40e5d9ae52d3203681a2d76f0_2_1380x534.png 2x" data-dominant-color="090909">

error_updt_11651×640 57.1 KB

error_updt_21641×612 20.4 KB

Should I open an ticket directly with support@empower.com or just ignore the above errors?

Thank you in advance!

If you read those messages carefully it is telling you that required fixes for fix19 aren’t installed. Don’t skip fixes, always install the latest and patch every product to the latest at the same time and create an image before installing any fix. This will prevent version miss match; micromanagement wont do it, in fact it will cause it. After running the fixes in test for a week you are good to install the same to production servers. You shouldn’t micro manage fixes. They can be security related. Skipping some of them is never a good idea!

https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files3/3ec50ed26396295722c50641199d12afffcc834b_2_1035x106.png 1.5x, https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files/3ec50ed26396295722c50641199d12afffcc834b.png 2x" data-dominant-color="0C0C0C">

image1203×124 59.2 KB

Thanks for the advices Engin!

Me and my team will develop a maintenance schedule for our Integration Servers in order to maintain the last fixes installed in our environment.

Have a great day!