Java

Java

Java

Topics on Semeru (Java) on IBM Z

 View Only
Back to discussions
Expand all | Collapse all

Jenkins ssh agent on USS hangs with Java21

  • 1.  Jenkins ssh agent on USS hangs with Java21

    Posted Fri January 23, 2026 10:41 AM

    Hello,

      I have a problem with Jenkins ssh remote agent on USS with Java 21 (J21.0_64.0.9.1). i guess, this is because of changes described in https://www.ibm.com/docs/en/semeru-runtime-ce-z/21.0.0?topic=guide-file-encoding-utf-8-as-default-charset but i can't find the correct parameters update.

    I use 2.534 Jenkins server, which can run with both, java21 and java17. Server run on Linux (docker, java21), agent on USS. JVM Options is " -Dfile.encoding=ISO8859-1 -Xnoargsconversion ", Suffix Start Agent Command is " -text "

    Server:

    docker run --restart unless-stopped --name jenkins -p 8080:8080 -p 50000:50000 jenkins/jenkins:2.534-jdk21

    Here is the log (tweaked to show FINE level)  for Java 17 agent:


    Verified agent jar. No update is necessary.
    Expanded the channel window size to 4MB
    [01/22/26 11:09:53] [SSH] Starting agent process: export _BPXK_AUTOCVT=ON && export _CEE_RUNOPTS='FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)' && export _TAG_REDIR_ERR=txt && export _TAG_REDIR_IN=txt && export _TAG_REDIR_OUT=txt && export JAVA_HOME=/rsusr/java/IBM/J17.0_64 && export IBM_JAVA_ENABLE_ASCII_FILETAG=ON && export PATH=/rsusr/ported/bin:$PATH && env && cd "/u/ssuser1/jenkins" && /rsusr/java/IBM/J17.0_64/bin/java  -Dfile.encoding=ISO8859-1 -Xnoargsconversion  -jar remoting.jar -workDir /u/ssuser1/jenkins -jar-cache /u/ssuser1/jenkins/remoting/jarCache -text -loggingConfig /u/ssuser1/jenkins/logging.properties
    MAIL=/usr/mail/SSUSER1
    PATH=/rsusr/ported/bin:/bin
    _TAG_REDIR_IN=txt
    SSH_CLIENT= xxx
    _BPXK_AUTOCVT=ON
    SHELL=/bin/sh
    IBM_JAVA_ENABLE_ASCII_FILETAG=ON
    _CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)
    _=/bin/env
    LOGNAME=SSUSER1
    _TAG_REDIR_OUT=txt
    USER=SSUSER1
    HOME=/u/ssuser1
    SSH_CONNECTION=xxx 50438 yyyy
    JAVA_HOME=/rsusr/java/IBM/J17.0_64
    TZ=EST5EDT
    _TAG_REDIR_ERR=txt
    _ZOS_SMF_FD=3
    Running in text mode
    Jan 22, 2026 6:09:54 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
    INFO: Using /u/ssuser1/jenkins/remoting as a remoting work directory
    Jan 22, 2026 6:09:54 AM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
    INFO: Both error and output logs will be printed to /u/ssuser1/jenkins/remoting
    Jan 22, 2026 6:09:54 AM hudson.remoting.ClassFilter createDefaultInstance
    FINE: Using default in built class blacklisting
    Jan 22, 2026 6:09:54 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    <===[JENKINS REMOTING CAPACITY]===Jan 22, 2026 6:09:54 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending mode preamble: TEXT
    ><===[HUDSON TRANSMISSION BEGINS]===Jan 22, 2026 6:09:54 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    Jan 22, 2026 6:09:54 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Received capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    Jan 22, 2026 6:09:54 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Received mode preamble: TEXT
    Jan 22, 2026 6:09:54 AM hudson.remoting.ChannelBuilder negotiate
    FINE: Channel name channel negotiated mode TEXT with capability Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    channel started
    Jan 22, 2026 6:09:54 AM hudson.remoting.Launcher main
    FINE: performPing=true timeout=240,000 interval=0
    Jan 22, 2026 6:09:55 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    Jan 22, 2026 6:09:55 AM hudson.remoting.Channel$1 handle
    FINE: Received UserRequest:hudson.slaves.SlaveComputer$SlaveVersion@1b26cfd9
    Jan 22, 2026 6:09:55 AM hudson.remoting.Channel$1 handle
    FINE: Completed command UserRequest:hudson.slaves.SlaveComputer$SlaveVersion@1b26cfd9. It took 2ms
    Jan 22, 2026 6:09:55 AM hudson.remoting.Channel send
    FINE: Send RPCRequest:hudson.remoting.RemoteClassLoader$IClassLoader.getName[](2)
    Jan 22, 2026 6:09:55 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    Jan 22, 2026 6:09:55 AM hudson.remoting.Channel$1 handle
    FINE: Received Response(java.lang.String)
    Jan 22, 2026 6:09:55 AM hudson.remoting.Channel$1 handle
    FINE: Completed command Response:RPCRequest:hudson.remoting.RemoteClassLoader$IClassLoader.getName[](2)(java.lang.String). It took 2ms
    Jan 22, 2026 6:09:55 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null

    Here is a Java 21 log (last couple of lines which are different)

    FINE: Using default in built class blacklisting
    Jan 22, 2026 6:58:56 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    $��+.�+���(|��+��&�����)��|����+`���?���:����_��“���_����������%��頠��������Š��Š����:Jan 22, 2026 6:58:56 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending mode preamble: TEXT
    /��Ϡ��������Ô$����|+���+�(����|+�����+�)�Jan 22, 2026 6:58:56 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    Jan 22, 2026 6:58:56 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Received capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}

    This is the last line and agent hangs in "launching" state.

    Any ideas?

    Thanks.



    ------------------------------
    Sergey Korzhevsky
    ------------------------------


  • 2.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Wed January 28, 2026 06:48 AM

    Hi Sergey,


    Thanks for reporting this issue. The text that is garbled in JDK21 is probably coming from 

    https://github.com/jglick/remoting/blob/77a22b39f14a1b9d0bffdff9f431e02f5cc26967/src/main/java/hudson/remoting/Capability.java#L280


    1) Is it possible for you to run this scenario with  -Dfile.encoding=UTF-8 and post the output as before?

    2) Can you please also run with -Dcom.ibm.autocvt.trace=true and post the output. 


    Both 1) and 2) can be done independently.

    Thanks
    Vikas





    ------------------------------
    VIKAS CHANDRA
    ------------------------------

    Original Message


  • 3.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Wed January 28, 2026 09:22 AM

    Hi Vikas,

    Here are additional logs:

    -Dfile.encoding=ISO8859-1 -Dcom.ibm.autocvt.trace=true 

    SSHLauncher{host='yyy', port=22, credentialsId='ssuser1', jvmOptions=' -Dfile.encoding=ISO8859-1 -Dcom.ibm.autocvt.trace=true -Xnoargsconversion ', javaPath='/rsusr/java/IBM/J21.0_64/bin/java', prefixStartSlaveCmd='export _BPXK_AUTOCVT=ON && export _CEE_RUNOPTS='FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)' && export _TAG_REDIR_ERR=txt && export _TAG_REDIR_IN=txt && export _TAG_REDIR_OUT=txt && export JAVA_HOME=/rsusr/java/IBM/J21.0_64 && export IBM_JAVA_ENABLE_ASCII_FILETAG=ON && export PATH=/rsusr/ported/bin:$PATH && env && ', suffixStartSlaveCmd=' -text -loggingConfig /u/ssuser1/jenkins/logging.properties', launchTimeoutSeconds=60, maxNumRetries=10, retryWaitTime=15, sshHostKeyVerificationStrategy=hudson.plugins.sshslaves.verifiers.ManuallyTrustedKeyVerificationStrategy, tcpNoDelay=true, trackCredentials=true}
    [01/28/26 11:56:44] [SSH] Opening SSH connection to xxx:22.
    [01/28/26 11:56:45] [SSH] SSH host key matches key seen previously for this host. Connection will be allowed.
    [01/28/26 11:56:45] [SSH] Authentication successful.
    [01/28/26 11:56:46] [SSH] The remote user's environment is:
    @="sh"
    ERRNO="0"
    HOME="/u/ssuser1"
    IFS="     
    "
    LINENO="0"
    LOGNAME="SSUSER1"
    MAIL="/usr/mail/SSUSER1"
    MAILCHECK="600"
    OPTIND="1"
    PATH="/bin"
    PPID="33620043"
    PS1="\$ "
    PS2="> "
    PS3="#? "
    PS4="+ "
    PWD="/u/ssuser1"
    RANDOM="2151"
    SECONDS="0"
    SHELL="/bin/sh"
    SSH_CLIENT="xxx 60922 22"
    SSH_CONNECTION="xxx 60922 xxx 22"
    TZ="EST5EDT"
    USER="SSUSER1"
    _ZOS_SMF_FD="3"
    [01/28/26 11:56:46] [SSH] Starting sftp client.
    [01/28/26 11:56:47] [SSH] Copying latest remoting.jar...
    Source agent hash is A9B5046C1B63AB317922418BCA5317E8. Installed agent hash is A9B5046C1B63AB317922418BCA5317E8
    Verified agent jar. No update is necessary.
    Expanded the channel window size to 4MB
    [01/28/26 11:57:13] [SSH] Starting agent process: export _BPXK_AUTOCVT=ON && export _CEE_RUNOPTS='FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)' && export _TAG_REDIR_ERR=txt && export _TAG_REDIR_IN=txt && export _TAG_REDIR_OUT=txt && export JAVA_HOME=/rsusr/java/IBM/J21.0_64 && export IBM_JAVA_ENABLE_ASCII_FILETAG=ON && export PATH=/rsusr/ported/bin:$PATH && env && cd "/u/ssuser1/jenkins" && /rsusr/java/IBM/J21.0_64/bin/java  -Dfile.encoding=ISO8859-1 -Dcom.ibm.autocvt.trace=true -Xnoargsconversion  -jar remoting.jar -workDir /u/ssuser1/jenkins -jar-cache /u/ssuser1/jenkins/remoting/jarCache -text -loggingConfig /u/ssuser1/jenkins/logging.properties
    MAIL=/usr/mail/SSUSER1
    PATH=/rsusr/ported/bin:/bin
    _TAG_REDIR_IN=txt
    SSH_CLIENT=xxx 60922 22
    _BPXK_AUTOCVT=ON
    SHELL=/bin/sh
    IBM_JAVA_ENABLE_ASCII_FILETAG=ON
    _CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)
    _=/bin/env
    LOGNAME=SSUSER1
    _TAG_REDIR_OUT=txt
    USER=SSUSER1
    HOME=/u/ssuser1
    SSH_CONNECTION=xxx 60922 xxx 22
    JAVA_HOME=/rsusr/java/IBM/J21.0_64
    TZ=EST5EDT
    _TAG_REDIR_ERR=txt
    _ZOS_SMF_FD=3
    [AUTOCVT.INFO] [ThreadID:2] [jdk.internal.util.AutoConversion.<clinit>] Autoconversion is false
    Running in text mode
    Jan 28, 2026 6:57:13 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
    INFO: Using /u/ssuser1/jenkins/remoting as a remoting work directory
    Jan 28, 2026 6:57:13 AM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
    INFO: Both error and output logs will be printed to /u/ssuser1/jenkins/remoting
    Jan 28, 2026 6:57:13 AM hudson.remoting.ClassFilter createDefaultInstance
    FINE: Using default in built class blacklisting
    Jan 28, 2026 6:57:14 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    $��+.�+���(|��+��&�����)��|����+`���?���:����_��“���_����������%��頠��������Š��Š����:/��Ϡ��������ÔJan 28, 2026 6:57:14 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending mode preamble: TEXT
    $����|+���+�(����|+�����+�)�Jan 28, 2026 6:57:14 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    Jan 28, 2026 6:57:14 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Received capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}

    -Dfile.encoding=UTF-8 -Dcom.ibm.autocvt.trace=true 

    SSHLauncher{host='xxxx', port=22, credentialsId='ssuser1', jvmOptions=' -Dfile.encoding=UTF-8 -Dcom.ibm.autocvt.trace=true -Xnoargsconversion ', javaPath='/rsusr/java/IBM/J21.0_64/bin/java', prefixStartSlaveCmd='export _BPXK_AUTOCVT=ON && export _CEE_RUNOPTS='FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)' && export _TAG_REDIR_ERR=txt && export _TAG_REDIR_IN=txt && export _TAG_REDIR_OUT=txt && export JAVA_HOME=/rsusr/java/IBM/J21.0_64 && export IBM_JAVA_ENABLE_ASCII_FILETAG=ON && export PATH=/rsusr/ported/bin:$PATH && env && ', suffixStartSlaveCmd=' -text -loggingConfig /u/ssuser1/jenkins/logging.properties', launchTimeoutSeconds=60, maxNumRetries=10, retryWaitTime=15, sshHostKeyVerificationStrategy=hudson.plugins.sshslaves.verifiers.ManuallyTrustedKeyVerificationStrategy, tcpNoDelay=true, trackCredentials=true}
    [01/28/26 12:00:48] [SSH] Opening SSH connection to xxxx:22.
    [01/28/26 12:00:49] [SSH] SSH host key matches key seen previously for this host. Connection will be allowed.
    [01/28/26 12:00:50] [SSH] Authentication successful.
    [01/28/26 12:00:50] [SSH] The remote user's environment is:
    @="sh"
    ERRNO="0"
    HOME="/u/ssuser1"
    IFS="     
    "
    LINENO="0"
    LOGNAME="SSUSER1"
    MAIL="/usr/mail/SSUSER1"
    MAILCHECK="600"
    OPTIND="1"
    PATH="/bin"
    PPID="16842829"
    PS1="\$ "
    PS2="> "
    PS3="#? "
    PS4="+ "
    PWD="/u/ssuser1"
    RANDOM="14698"
    SECONDS="0"
    SHELL="/bin/sh"
    SSH_CLIENT="xxx 60982 22"
    SSH_CONNECTION="xxx 60982 xxxx 22"
    TZ="EST5EDT"
    USER="SSUSER1"
    _ZOS_SMF_FD="3"
    [01/28/26 12:00:50] [SSH] Starting sftp client.
    [01/28/26 12:00:52] [SSH] Copying latest remoting.jar...
    Source agent hash is A9B5046C1B63AB317922418BCA5317E8. Installed agent hash is A9B5046C1B63AB317922418BCA5317E8
    Verified agent jar. No update is necessary.
    Expanded the channel window size to 4MB
    [01/28/26 12:01:15] [SSH] Starting agent process: export _BPXK_AUTOCVT=ON && export _CEE_RUNOPTS='FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)' && export _TAG_REDIR_ERR=txt && export _TAG_REDIR_IN=txt && export _TAG_REDIR_OUT=txt && export JAVA_HOME=/rsusr/java/IBM/J21.0_64 && export IBM_JAVA_ENABLE_ASCII_FILETAG=ON && export PATH=/rsusr/ported/bin:$PATH && env && cd "/u/ssuser1/jenkins" && /rsusr/java/IBM/J21.0_64/bin/java  -Dfile.encoding=UTF-8 -Dcom.ibm.autocvt.trace=true -Xnoargsconversion  -jar remoting.jar -workDir /u/ssuser1/jenkins -jar-cache /u/ssuser1/jenkins/remoting/jarCache -text -loggingConfig /u/ssuser1/jenkins/logging.properties
    MAIL=/usr/mail/SSUSER1
    PATH=/rsusr/ported/bin:/bin
    _TAG_REDIR_IN=txt
    SSH_CLIENT=xxxx 60982 22
    _BPXK_AUTOCVT=ON
    SHELL=/bin/sh
    IBM_JAVA_ENABLE_ASCII_FILETAG=ON
    _CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)
    _=/bin/env
    LOGNAME=SSUSER1
    _TAG_REDIR_OUT=txt
    USER=SSUSER1
    HOME=/u/ssuser1
    SSH_CONNECTION=xxx 60982 xxxx 22
    JAVA_HOME=/rsusr/java/IBM/J21.0_64
    TZ=EST5EDT
    _TAG_REDIR_ERR=txt
    _ZOS_SMF_FD=3
    [AUTOCVT.INFO] [ThreadID:2] [jdk.internal.util.AutoConversion.<clinit>] Autoconversion is false
    Running in text mode
    Jan 28, 2026 7:01:16 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
    INFO: Using /u/ssuser1/jenkins/remoting as a remoting work directory
    Jan 28, 2026 7:01:16 AM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
    INFO: Both error and output logs will be printed to /u/ssuser1/jenkins/remoting
    Jan 28, 2026 7:01:16 AM hudson.remoting.ClassFilter createDefaultInstance
    FINE: Using default in built class blacklisting
    Jan 28, 2026 7:01:16 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    $��+.�+���(|��+��&�����)��|����+`���?���:����_��“���_����������%��頠��������Š��Š����:/��Ϡ������Jan 28, 2026 7:01:16 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending mode preamble: TEXT
    ��Ô$����|+���+�(����|+�����+�)�Jan 28, 2026 7:01:16 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    Jan 28, 2026 7:01:16 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Received capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}



    ------------------------------
    Sergey Korzhevsky
    ------------------------------

    Original Message


  • 4.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Thu January 29, 2026 08:00 AM

    Hi Sergey,

    Thanks for sending the logs!

    As per the logs, for some reason, autoconversion is disabled with -Dfile.encoding=UTF-8 -Dcom.ibm.autocvt.trace=true but by default it should have been enabled 

    Can we try with these 3 options together -Dfile.encoding=UTF-8 -Dcom.ibm.autocvt=true -Dcom.ibm.autocvt.trace=true 

    Note : It has one more option in the middle.

    Thanks,
    Vikas



    ------------------------------
    VIKAS CHANDRA
    ------------------------------

    Original Message


  • 5.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Thu January 29, 2026 09:16 AM

    Hi Vikas,

      Unfortunately, same result.

    <skip...>

    JAVA_HOME=/rsusr/java/IBM/J21.0_64
    TZ=EST5EDT
    _TAG_REDIR_ERR=txt
    _ZOS_SMF_FD=3
    [AUTOCVT.INFO] [ThreadID:2] [jdk.internal.util.AutoConversion.<clinit>] Autoconversion is true
    [AUTOCVT.INFO] [ThreadID:2] [jdk.internal.util.AutoConversion.enable] Auto-conversion not enabled for file (not tagged as text): /rsusr/java/IBM/J21.0_64.0.9.1/conf/logging.properties
    Running in text mode
    [AUTOCVT.INFO] [ThreadID:2] [jdk.internal.util.AutoConversion.enable] Auto-conversion not enabled for file (not tagged as text): /rsusr/java/IBM/J21.0_64.0.9.1/lib/tzdb.dat
    [AUTOCVT.INFO] [ThreadID:2] [jdk.internal.util.AutoConversion.enable] Auto-conversion not enabled for file (not tagged as text): /rsusr/java/IBM/J21.0_64.0.9.1/lib/tzdb.dat
    Jan 29, 2026 9:10:48 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
    INFO: Using /u/ssuser1/jenkins/remoting as a remoting work directory
    [AUTOCVT.INFO] [ThreadID:2] [jdk.internal.util.AutoConversion.enable] Auto-conversion not enabled for file (not tagged as text): /u/ssuser1/jenkins/logging.properties
    Jan 29, 2026 9:10:48 AM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
    INFO: Both error and output logs will be printed to /u/ssuser1/jenkins/remoting
    Jan 29, 2026 9:10:48 AM hudson.remoting.ClassFilter createDefaultInstance
    FINE: Using default in built class blacklisting
    Jan 29, 2026 9:10:48 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    $��+.�+���(|��+��&�����)��|���Jan 29, 2026 9:10:48 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending mode preamble: TEXT
    [AUTOCVT.INFO] [ThreadID:2] [jdk.internal.util.AutoConversion.enable] �Auto-conversion not enabled for file (not tagged as text): /rsusr/java/IBM/J21.0_64.0.9.1/conf/security/java.security+`
    �Jan 29, 2026 9:10:48 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    �Jan 29, 2026 9:10:48 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Received capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}



    ------------------------------
    Sergey Korzhevsky
    ------------------------------

    Original Message


  • 6.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Mon February 02, 2026 07:37 AM

    Hi Sergey

    A case is opened for this and it is being tracked through that. We are investigating

    Thanks
    Vikas



    ------------------------------
    VIKAS CHANDRA
    ------------------------------

    Original Message


  • 7.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Tue February 03, 2026 04:35 AM
    Edited by VIKAS CHANDRA Tue February 03, 2026 04:36 AM

    Hi Sergey,

    For our diagnosis, can you please try with this option and see if hang happens.
    Also please provide traces.

    -Dfile.encoding=UTF-8 -Dconsole.encoding=UTF-8 -Dcom.ibm.autocvt=true 

    You could also add -Dcom.ibm.autocvt.trace=true (4th option for tracing)
    Thanks,

    Vikas



    ------------------------------
    VIKAS CHANDRA
    ------------------------------

    Original Message


  • 8.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Tue February 03, 2026 09:51 AM

    Hi Vikas,

      In this case, it hangs even earlier, this is a full log:

    SSHLauncher{host='xxx.com', port=22, credentialsId='ssuser1', jvmOptions=' -Dfile.encoding=UTF-8 -Dconsole.encoding=UTF-8 -Dcom.ibm.autocvt=true -Dcom.ibm.autocvt.trace=true -Xnoargsconversion ', javaPath='/rsusr/java/IBM/J21.0_64/bin/java', prefixStartSlaveCmd='export _BPXK_AUTOCVT=ON && export _CEE_RUNOPTS='FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)' && export _TAG_REDIR_ERR=txt && export _TAG_REDIR_IN=txt && export _TAG_REDIR_OUT=txt && export JAVA_HOME=/rsusr/java/IBM/J21.0_64 && export IBM_JAVA_ENABLE_ASCII_FILETAG=ON && export PATH=/rsusr/ported/bin:$PATH && env && ', suffixStartSlaveCmd=' -text -loggingConfig /u/ssuser1/jenkins/logging.properties', launchTimeoutSeconds=60, maxNumRetries=10, retryWaitTime=15, sshHostKeyVerificationStrategy=hudson.plugins.sshslaves.verifiers.ManuallyTrustedKeyVerificationStrategy, tcpNoDelay=true, trackCredentials=true}
    [02/03/26 10:45:21] [SSH] Opening SSH connection to xxxxx:22.
    [02/03/26 10:45:22] [SSH] SSH host key matches key seen previously for this host. Connection will be allowed.
    [02/03/26 10:45:22] [SSH] Authentication successful.
    [02/03/26 10:45:22] [SSH] The remote user's environment is:
    @="sh"
    ERRNO="0"
    HOME="/u/ssuser1"
    IFS="     
    "
    LINENO="0"
    LOGNAME="SSUSER1"
    MAIL="/usr/mail/SSUSER1"
    MAILCHECK="600"
    OPTIND="1"
    PATH="/bin"
    PPID="16842930"
    PS1="\$ "
    PS2="> "
    PS3="#? "
    PS4="+ "
    PWD="/u/ssuser1"
    RANDOM="7097"
    SECONDS="0"
    SHELL="/bin/sh"
    SSH_CLIENT="xxx 51296 22"
    SSH_CONNECTION="xxx 51296 xxxx 22"
    TZ="EST5EDT"
    USER="SSUSER1"
    _ZOS_SMF_FD="3"
    [02/03/26 10:45:23] [SSH] Starting sftp client.
    [02/03/26 10:45:24] [SSH] Copying latest remoting.jar...
    Source agent hash is A9B5046C1B63AB317922418BCA5317E8. Installed agent hash is A9B5046C1B63AB317922418BCA5317E8
    Verified agent jar. No update is necessary.
    Expanded the channel window size to 4MB
    [02/03/26 10:45:48] [SSH] Starting agent process: export _BPXK_AUTOCVT=ON && export _CEE_RUNOPTS='FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)' && export _TAG_REDIR_ERR=txt && export _TAG_REDIR_IN=txt && export _TAG_REDIR_OUT=txt && export JAVA_HOME=/rsusr/java/IBM/J21.0_64 && export IBM_JAVA_ENABLE_ASCII_FILETAG=ON && export PATH=/rsusr/ported/bin:$PATH && env && cd "/u/ssuser1/jenkins" && /rsusr/java/IBM/J21.0_64/bin/java  -Dfile.encoding=UTF-8 -Dconsole.encoding=UTF-8 -Dcom.ibm.autocvt=true -Dcom.ibm.autocvt.trace=true -Xnoargsconversion  -jar remoting.jar -workDir /u/ssuser1/jenkins -jar-cache /u/ssuser1/jenkins/remoting/jarCache -text -loggingConfig /u/ssuser1/jenkins/logging.properties
    MAIL=/usr/mail/SSUSER1
    PATH=/rsusr/ported/bin:/bin
    _TAG_REDIR_IN=txt
    SSH_CLIENT=xxx 51296 22
    _BPXK_AUTOCVT=ON
    SHELL=/bin/sh
    IBM_JAVA_ENABLE_ASCII_FILETAG=ON
    _CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)
    _=/bin/env
    LOGNAME=SSUSER1
    _TAG_REDIR_OUT=txt
    USER=SSUSER1
    HOME=/u/ssuser1
    SSH_CONNECTION=xxx 51296 xxxx 22
    JAVA_HOME=/rsusr/java/IBM/J21.0_64
    TZ=EST5EDT
    _TAG_REDIR_ERR=txt
    _ZOS_SMF_FD=3



    ------------------------------
    Sergey Korzhevsky
    ------------------------------

    Original Message


  • 9.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Wed February 04, 2026 12:14 PM

    Hi Sergey,

    As Vikas posted, this issue has been reported to us and we are working it.

    At the moment I don't need anything from you.

    I will get back to you when we have a resolution or if we need anything from you.

    Thanks

    Stephen Flavell
    Java and Semeru on z/OS support



    ------------------------------
    Stephen Flavell
    ------------------------------

    Original Message


  • 10.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Thu February 05, 2026 07:00 AM

    Hi Sergey,

    I've discussed this issue with Vikas.

    Can you create a support Case with IBM for this issue please.

    Thanks

    Stephen Flavell
    Java and Semeru on z/OS support



    ------------------------------
    Stephen Flavell
    ------------------------------

    Original Message


  • 11.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Thu February 05, 2026 08:11 AM

    Hi Stephen, 

    There is a message above from Vikas, that "A case is opened for this and it is being tracked through that. We are investigating". What else do you want from me?

    Thanks.



    ------------------------------
    Sergey Korzhevsky
    Senior Software Engineer in Test
    Rocket Software
    ------------------------------

    Original Message


  • 12.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Thu February 05, 2026 10:44 AM
    Hi Sergey,
     
    >> There is a message above from Vikas, that "A case is opened for this and it is being tracked through that. 
    >> We are investigating". What else do you want from me?
     
    The Case that Vikas refers to is one that was created by an IBM team when they saw these symptoms.
    After discussion with Vikas, whilst your symptoms are the same, we can't be sure that you have exactly the same issue that was reported by the IBM team.
    For this reason we would like you to open a Support Case on CSP so that we are able to pass documents using that secure service. 
     
     
    Thanks


    ------------------------------
    Stephen Flavell
    ------------------------------

    Original Message


  • 13.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Tue February 10, 2026 04:31 AM

    Hi Sergey

    Can you please try with

    -Dfile.encoding=ISO8859-1 -Dconsole.encoding=IBM-1047

    -Dfile.encoding=UTF-8 -Dconsole.encoding=IBM-1047

    Thanks
    Vikas



    ------------------------------
    VIKAS CHANDRA
    ------------------------------

    Original Message


  • 14.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Tue February 10, 2026 06:18 AM

    Hi All,

      I have no idea what is CSP and i'm not sure i want to go to that rabbit hole...
      
    For the 
    -Dfile.encoding=ISO8859-1 -Dconsole.encoding=IBM-1047


    FINER: Sending capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    $��+.�+���(|��+��&�����)��|����+`���?���:����_��“���_����������%��頠��������Š��Š����:/Feb 10, 2026 6:09:32 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending mode preamble: TEXT
    ��Ϡ��������Ô$����|+���+�(����|+�����+�)�Feb 10, 2026 6:09:32 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    Feb 10, 2026 6:09:33 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Received capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}


    For the
    -Dfile.encoding=UTF-8 -Dconsole.encoding=IBM-1047


    FINER: Sending capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}
    $��+.�+���(|��+��&�����)��|����+`���?���:����_��“���_����������%��頠��������Š��Š����:Feb 10, 2026 6:12:19 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Sending mode preamble: TEXT
    /��Ϡ��������Ô$����|+���+�(����|+�����+�)�Feb 10, 2026 6:12:19 AM java.io.ObjectInputFilter$Config traceFilter
    FINER: Builtin factory: null -> new: null
    Feb 10, 2026 6:12:19 AM hudson.remoting.ChannelBuilder negotiate
    FINER: Received capability preamble: Capability{Multi-ClassLoader RPC, Pipe throttling, Mimic Exception, Prefetch, Greedy RemoteInputStream, Proxy writer 2.35, Chunked encoding, ProxyException fallback}


    Basically, the same.



    ------------------------------
    Sergey Korzhevsky
    Senior Software Engineer in Test
    Rocket Software
    ------------------------------

    Original Message


  • 15.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Tue February 10, 2026 10:47 AM
    Hi Sergey,
     
    >> I have no idea what is CSP and i'm not sure i want to go to that rabbit hole...
     
    That's OK.
    For your information, CSP is IBM's Cognitive Support Platform
    CSP is the way in which Customers report issues to IBM.


    ------------------------------
    Stephen Flavell
    ------------------------------

    Original Message


  • 16.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Wed February 25, 2026 05:52 AM

    Hi,

     Opened a case, it was really easy, thanks

    Case number

    TS021537689



    ------------------------------
    Sergey Korzhevsky
    Senior Software Engineer in Test
    Rocket Software
    ------------------------------

    Original Message


  • 17.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Thu February 26, 2026 08:14 AM

    Where might others with the same problem follow progress if the issue is now being worked on in a case?



    ------------------------------
    Arne Alppiranta
    ------------------------------

    Original Message


  • 18.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted Mon March 02, 2026 08:43 AM

    Unfortunately due to confidentiality and security requirements, to continue following you need to acquire access to IBM CSP where official support cases are opened and tracked.



    ------------------------------
    James Tang
    Java on IBM Z lead
    IBM
    ------------------------------

    Original Message


  • 19.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted 21 days ago
    Edited by Joran Siu 15 days ago

    Hi Sergey,

    Semeru 21.0.10.1 PTF has been released to address this Jenkins SSH agent hang issue. Additional details are available in the following blog post: https://community.ibm.com/community/user/blogs/paul-arockiam/2026/04/02/ibm-semeru-runtime-ce-for-zos-interim-210101. The corresponding APAR is IJ57789.

    Thanks,
    Joran

    -------------------------------------------

    Original Message


  • 20.  RE: Jenkins ssh agent on USS hangs with Java21

    Posted 16 days ago

    Hi Joran,

    Jenkins works fine now, thanks!

    I closed ticket.



    ------------------------------
    Sergey Korzhevsky
    Senior Software Engineer in Test
    Rocket Software
    ------------------------------

    Original Message


Related Content