Originally posted by: thorstenhirsch

We're trying to set up a build job for ITX 9 mappings with Jenkins 2.x. So we created a job in Jenkins that fetches all necessary sources (mms, mtt, xsd, ...) and calls mcompile.exe. Unfortunately the mcompile.exe call hangs for a lot of mms files. There's no output. Even -LOG <file> and -VERBOSE only create an empty log file.

• outside of Jenkins ALL mms files can be compiled with mcompile.exe
• we made sure that mcompile.exe has all necessary files (mtt, xsd, ...) in its workspace
• we have found no classification which mms files work and which don't in Jenkins
• all results are 100% reproducible: if a mms can be compiled, it can always be compiled / if a mms hangs, it always hangs

We also copied the workspace of some hanging Jenkins jobs and called mcompile.exe manually in a command shell - it worked successfully. What restrictions might exist within a Jenkins job that can cause mcompile.exe to hang?

Originally posted by: paul.brett

I would guess that the Jenkins environment does not have access to all the required 3rd party libraries that are required.  Are you able to create a Jenkins job that will give a list of all environment variables, and then replicate that environment?

Thank-you.

Paul.

Follow me on Twitter

Originally posted by: thorstenhirsch

Hi Paul.
We replicated the environment and could still compile a map successfully. We also checked permissions of the user that is running the jenkins job - he has write permissions on the whole itx installation directory.

There's a page in the jenkins wiki for "hanging jobs" that suggest to look into the thread dump. Here it is...

Thread Dump
cmd /c call C:\Windows\TEMP\hudson5456163115074949438.bat: stdout copier
"cmd /c call C:\Windows\TEMP\hudson5456163115074949438.bat: stdout copier" Id=3194 Group=main RUNNABLE (in native)
         at java.io.FileInputStream.readBytes(Native Method)
         at java.io.FileInputStream.read(Unknown Source)
         at java.io.BufferedInputStream.read1(Unknown Source)
         at java.io.BufferedInputStream.read(Unknown Source)
         -  locked java.io.BufferedInputStream@405484
         at java.io.FilterInputStream.read(Unknown Source)
         at hudson.util.StreamCopyThread.run(StreamCopyThread.java:60)

DestroyJavaVM
"DestroyJavaVM" Id=16 Group=main RUNNABLE

Executor #1 for master : executing SST #54
"Executor #1 for master : executing SST #54" Id=3182 Group=main RUNNABLE (in native)
         at java.lang.ProcessImpl.waitForInterruptibly(Native Method)
         at java.lang.ProcessImpl.waitFor(Unknown Source)
         at hudson.Proc$LocalProc.join(Proc.java:315)
         at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:135)
         at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:95)
         at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:64)
         at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
         at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
         at hudson.model.Build$BuildExecution.build(Build.java:205)
         at hudson.model.Build$BuildExecution.doRun(Build.java:162)
         at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
         at hudson.model.Run.execute(Run.java:1720)
         at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
         at hudson.model.ResourceController.execute(ResourceController.java:98)
         at hudson.model.Executor.run(Executor.java:410)

FilePath.localPool [#747]
"FilePath.localPool [#747]" Id=3390 Group=main RUNNABLE
         at sun.management.ThreadImpl.dumpThreads0(Native Method)
         at sun.management.ThreadImpl.dumpAllThreads(Unknown Source)
         at hudson.Functions.getThreadInfos(Functions.java:1196)
         at hudson.util.RemotingDiagnostics$GetThreadDump.call(RemotingDiagnostics.java:98)
         at hudson.util.RemotingDiagnostics$GetThreadDump.call(RemotingDiagnostics.java:95)
         at hudson.remoting.LocalChannel$1.call(LocalChannel.java:52)
         at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
         at java.util.concurrent.FutureTask.run(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

         Number of locked synchronizers = 1
         - java.util.concurrent.ThreadPoolExecutor$Worker@1c94bd4

Handling GET /threadDump from 10.241.30.58 : RequestHandlerThread[#88] Jenkins/threadDump.jelly
"Handling GET /threadDump from 10.241.30.58 : RequestHandlerThread[#88] Jenkins/threadDump.jelly" Id=3387 Group=main TIMED_WAITING on java.util.concurrent.FutureTask@17d5ebf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.FutureTask@17d5ebf
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.FutureTask.awaitDone(Unknown Source)
         at java.util.concurrent.FutureTask.get(Unknown Source)
         at hudson.remoting.LocalChannel$2.get(LocalChannel.java:81)
         at jenkins.model.Jenkins.getAllThreadDumps(Jenkins.java:3537)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
         at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
         at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
         at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
         at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
         at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
         at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
         at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
         at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
         at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
         at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
         at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
         at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
         at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
         at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
         at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
         at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
         at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
         at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95)
         at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
         at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
         at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
         at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
         at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
         at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135)
         at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126)
         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
         at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49)
         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
         at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
         at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
         at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
         at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
         at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
         at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
         at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
         at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
         at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
         at org.eclipse.jetty.server.Server.handle(Server.java:499)
         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
         at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

         Number of locked synchronizers = 1
         - java.util.concurrent.ThreadPoolExecutor$Worker@1c52f9e

Jenkins cron thread
"Jenkins cron thread" Id=17 Group=main WAITING on java.util.TaskQueue@143164b
         at java.lang.Object.wait(Native Method)
         -  waiting on java.util.TaskQueue@143164b
         at java.lang.Object.wait(Unknown Source)
         at java.util.TimerThread.mainLoop(Unknown Source)
         at java.util.TimerThread.run(Unknown Source)

Jenkins UDP 33848 monitoring thread
"Jenkins UDP 33848 monitoring thread" Id=38 Group=main RUNNABLE (in native)
         at java.net.TwoStacksPlainDatagramSocketImpl.receive0(Native Method)
         -  locked java.net.TwoStacksPlainDatagramSocketImpl@1096e14
         at java.net.TwoStacksPlainDatagramSocketImpl.receive(Unknown Source)
         -  locked java.net.TwoStacksPlainDatagramSocketImpl@1096e14
         at java.net.DatagramSocket.receive(Unknown Source)
         -  locked java.net.DatagramPacket@21b30d
         -  locked java.net.MulticastSocket@1a31c82
         at hudson.UDPBroadcastThread.run(UDPBroadcastThread.java:84)

jenkins.util.Timer [#10]
"jenkins.util.Timer [#10]" Id=34 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#1]
"jenkins.util.Timer [#1]" Id=18 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#2]
"jenkins.util.Timer [#2]" Id=23 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#3]
"jenkins.util.Timer [#3]" Id=24 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#4]
"jenkins.util.Timer [#4]" Id=25 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#5]
"jenkins.util.Timer [#5]" Id=26 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#6]
"jenkins.util.Timer [#6]" Id=30 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#7]
"jenkins.util.Timer [#7]" Id=31 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#8]
"jenkins.util.Timer [#8]" Id=32 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

jenkins.util.Timer [#9]
"jenkins.util.Timer [#9]" Id=33 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16b54cf
         at java.util.concurrent.locks.LockSupport.park(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

JmDNS(dfviaspwseu20.local.).State.Timer
"JmDNS(dfviaspwseu20.local.).State.Timer" Id=44 Group=main TIMED_WAITING on java.util.TaskQueue@18ef781
         at java.lang.Object.wait(Native Method)
         -  waiting on java.util.TaskQueue@18ef781
         at java.util.TimerThread.mainLoop(Unknown Source)
         at java.util.TimerThread.run(Unknown Source)

JmDNS(dfviaspwseu20.local.).Timer
"JmDNS(dfviaspwseu20.local.).Timer" Id=43 Group=main TIMED_WAITING on java.util.TaskQueue@d277a
         at java.lang.Object.wait(Native Method)
         -  waiting on java.util.TaskQueue@d277a
         at java.util.TimerThread.mainLoop(Unknown Source)
         at java.util.TimerThread.run(Unknown Source)

LauncherControlThread[ControlPort=-1]
"LauncherControlThread[ControlPort=-1]" Id=14 Group=main TIMED_WAITING
         at java.lang.Thread.sleep(Native Method)
         at winstone.Launcher.run(Launcher.java:243)
         at java.lang.Thread.run(Unknown Source)

NioSocketAcceptor-1
"NioSocketAcceptor-1" Id=36 Group=main RUNNABLE (in native)
         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source)
         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown Source)
         at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)
         at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
         -  locked sun.nio.ch.Util$2@5d2a3
         -  locked java.util.Collections$UnmodifiableSet@151a2b5
         -  locked sun.nio.ch.WindowsSelectorImpl@fde651
         at sun.nio.ch.SelectorImpl.select(Unknown Source)
         at sun.nio.ch.SelectorImpl.select(Unknown Source)
         at org.apache.mina.transport.socket.nio.NioSocketAcceptor.select(NioSocketAcceptor.java:238)
         at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:432)
         at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

         Number of locked synchronizers = 1
         - java.util.concurrent.ThreadPoolExecutor$Worker@80e81a

org.eclipse.jetty.server.session.HashSessionManager@1846619Timer
"org.eclipse.jetty.server.session.HashSessionManager@1846619Timer" Id=9 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4d2281
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4d2281
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

RequestHandlerThread[#1]-selector-ServerConnectorManager@19bad24/0
"RequestHandlerThread[#1]-selector-ServerConnectorManager@19bad24/0" Id=12 Group=main RUNNABLE (in native)
         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
         at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source)
         at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown Source)
         at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)
         at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
         -  locked sun.nio.ch.Util$2@41e71d
         -  locked java.util.Collections$UnmodifiableSet@1111836
         -  locked sun.nio.ch.WindowsSelectorImpl@87aa51
         at sun.nio.ch.SelectorImpl.select(Unknown Source)
         at sun.nio.ch.SelectorImpl.select(Unknown Source)
         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
         at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

         Number of locked synchronizers = 1
         - java.util.concurrent.ThreadPoolExecutor$Worker@86afc2

RequestHandlerThread[#2]-acceptor-0@127ac00-ServerConnector@145a2f5{HTTP/1.1}{0.0.0.0:8080}
"RequestHandlerThread[#2]-acceptor-0@127ac00-ServerConnector@145a2f5{HTTP/1.1}{0.0.0.0:8080}" Id=13 Group=main RUNNABLE (in native)
         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
         at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source)
         at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source)
         -  locked java.lang.Object@1f1571c
         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)
         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)
         at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

         Number of locked synchronizers = 1
         - java.util.concurrent.ThreadPoolExecutor$Worker@cee291

RequestHandlerThread[#80]
"RequestHandlerThread[#80]" Id=3197 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
         at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

RequestHandlerThread[#81]
"RequestHandlerThread[#81]" Id=3216 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
         at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

RequestHandlerThread[#83]
"RequestHandlerThread[#83]" Id=3218 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
         at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

RequestHandlerThread[#84]
"RequestHandlerThread[#84]" Id=3375 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
         at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

RequestHandlerThread[#85]
"RequestHandlerThread[#85]" Id=3376 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
         at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

RequestHandlerThread[#87]
"RequestHandlerThread[#87]" Id=3386 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.SynchronousQueue$TransferStack@142c2c8
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
         at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

Scheduler-21400537
"Scheduler-21400537" Id=200 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@29dadd
         at sun.misc.Unsafe.park(Native Method)
         -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@29dadd
         at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

SocketListener(dfviaspwseu20.local.)
"SocketListener(dfviaspwseu20.local.)" Id=42 Group=main RUNNABLE (in native)
         at java.net.TwoStacksPlainDatagramSocketImpl.receive0(Native Method)
         -  locked java.net.TwoStacksPlainDatagramSocketImpl@6fabcb
         at java.net.TwoStacksPlainDatagramSocketImpl.receive(Unknown Source)
         -  locked java.net.TwoStacksPlainDatagramSocketImpl@6fabcb
         at java.net.DatagramSocket.receive(Unknown Source)
         -  locked java.net.DatagramPacket@c46aa9
         -  locked java.net.MulticastSocket@b4a1cf
         at javax.jmdns.impl.SocketListener.run(SocketListener.java:41)

TCP agent listener port=0
"TCP agent listener port=0" Id=37 Group=main RUNNABLE (in native)
         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
         at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source)
         at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source)
         -  locked java.lang.Object@ce61e0
         at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:111)

Thread-35
"Thread-35" Id=3388 Group=main RUNNABLE (in native)
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.socketRead(Unknown Source)
         at java.net.SocketInputStream.read(Unknown Source)
         at java.net.SocketInputStream.read(Unknown Source)
         at java.io.BufferedInputStream.fill(Unknown Source)
         at java.io.BufferedInputStream.read1(Unknown Source)
         at java.io.BufferedInputStream.read(Unknown Source)
         -  locked java.io.BufferedInputStream@1baa9b8
         at com.sun.jndi.ldap.Connection.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

Attach Listener
"Attach Listener" Id=4 Group=system RUNNABLE

Finalizer
"Finalizer" Id=3 Group=system WAITING on java.lang.ref.ReferenceQueue$Lock@17e44d6
         at java.lang.Object.wait(Native Method)
         -  waiting on java.lang.ref.ReferenceQueue$Lock@17e44d6
         at java.lang.ref.ReferenceQueue.remove(Unknown Source)
         at java.lang.ref.ReferenceQueue.remove(Unknown Source)
         at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

Java2D Disposer
"Java2D Disposer" Id=10 Group=system WAITING on java.lang.ref.ReferenceQueue$Lock@1cd556a
         at java.lang.Object.wait(Native Method)
         -  waiting on java.lang.ref.ReferenceQueue$Lock@1cd556a
         at java.lang.ref.ReferenceQueue.remove(Unknown Source)
         at java.lang.ref.ReferenceQueue.remove(Unknown Source)
         at sun.java2d.Disposer.run(Unknown Source)
         at java.lang.Thread.run(Unknown Source)

Reference Handler
"Reference Handler" Id=2 Group=system WAITING on java.lang.ref.Reference$Lock@9ab10a
         at java.lang.Object.wait(Native Method)
         -  waiting on java.lang.ref.Reference$Lock@9ab10a
         at java.lang.Object.wait(Unknown Source)
         at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)

So far I could only see some PIDs - one of them might be the hanging mcompile.exe process. I'll try to get a stack trace of that process...

Originally posted by: thorstenhirsch

Windows crashes (bluescreen) when trying to get a stack trace of the mcompile.exe process in Sysinternal's Process Explorer. Any ideas how to get a stack trace otherwise?

Originally posted by: thorstenhirsch

Alright, I've found DrMemory's drstrace.exe. It's an open source project with a dubious name. Well, my anti virus scanner says it's clean, so I gave it a try. The stack trace is attached.

As you can see, one of the last things it does is accessing an xsd file (it exists and is readable and does not contain german umlauts). This xsd file is the native xsd of input card #1. So maybe only mappings with native xsd are affected. What does mcompile.exe do when encountering an mms file with native xsd input cards?
 

Attachment(s)

drstrace.mcompile.exe.00544.0000.log.zip   7 KB 1 version

Originally posted by: thorstenhirsch

Solved.

The drstrace file of a successful mcompile process revealed, that the next step would be:

NtOpenSection
    arg 0: 0x0000000000a7e3d8 (type=HANDLE*, size=0x8)
    arg 1: 0xf (type=unsigned int, size=0x4)
    arg 2: len=0x30, root=0x8, name=24/26 "dtxjpi64.dll", att=0x40, sd=0x0000000000000000, sqos=0x0000000000000000 (type=OBJECT_ATTRIBUTES*, size=0x8)
    failed (error=0xc0000034) =>
    arg 0: 0x0000000000a7e3d8 => 0x0 (type=HANDLE*, size=0x8)
    retval: 0xc0000034 (type=NTSTATUS, size=0x4)

That's loading a 64bit DLL. So the question was: Why would my jenkins server not load a 64bit DLL? It's running on a 64bit machine (identical to the one where mcompile runs successfully) and the Windows installation is also 64bit. The answer: because of the 32bit Java installation, in which Jenkins is running. Actually there was a hint when comparing the environments between the "good" machine and jenkins:

PROCESSOR_ARCHITECTURE=x86 (jenkins)
PROCESSOR_ARCHITECTURE=AMD64 (good machine)

However I did not pay enough attention to that difference and only called "set PROCESSOR_ARCHITECTURE=x86" on the good machine. This did not prevent the good machine from loading the 64bit DLL, so mcompile still worked successfully with the environment of jenkins. Anyway, the solution is clear: we need a 64bit Java installation for jenkins, so that mcompile.exe can still load 64bit DLL files in my jenkins job.

tl;dr: Jenkins was running in a 32bit Java installation, thus mcompile.exe could not load 64bit DLL files, which are required for mappings using native xsd. Solution was to install Jenkins in a 64bit Java.