Originally posted by: LM_OR

Hi,

I have a question regarding the MPI distributed optimization capability of cplex.

I am trying to set up a simple example with two different OpenMPI CPLEX workers as suggested in the tutorial and https://www.ibm.com/developerworks/community/forums/html/topic?id=4bff0bbc-c442-46e5-b134-d659910e6f2b&ps=25. However, a symbol is missing and I am not sure how to solve the issue:

I use the following command:

opt/openmpi-1.4.3-gcc44/bin/mpirun \

 -x "LD_LIBRARY_PATH=..." \
 -tag-output \
 -host mch1 -host mch2 \
 /nethome/lmmc3/CPLEX1261/cplex/bin/x86-64_linux/cplex \
 -mpi \
  -libpath="/opt/openmpi-1.4.3-gcc44/lib" \
 -mpilib="/opt/openmpi-1.4.3-gcc44/lib/libmpi.so" \
 -mpiapi="openmpi"

I am pretty sure all the paths and LD_LIBRARY_PATH are set right. However, I get an error of a symbol not found:

[1,1]<stderr>:/nethome/lmmc3/CPLEX1261/cplex/bin/x86-64_linux/cplex: symbol lookup error: ../../../bin/x86-64_linux/libcplex1261mpiworker.so: undefined symbol: timer_create
[1,0]<stdout>:
[1,0]<stdout>:Welcome to IBM(R) ILOG(R) CPLEX(R) Interactive Optimizer 12.6.1.0
[1,0]<stdout>:  with Simplex, Mixed Integer & Barrier Optimizers
[1,0]<stdout>:Copyright IBM Corp. 1988, 2014.  All Rights Reserved.
[1,0]<stdout>:
[1,0]<stdout>:Type 'help' for a list of available commands.
[1,0]<stdout>:Type 'help' followed by a command name for more
[1,0]<stdout>:information on commands.
[1,0]<stdout>:
[1,0]<stdout>:CPLEX>
--------------------------------------------------------------------------
mpirun has exited due to process rank 1 with PID 21951 on
node mch2 exiting without calling "finalize". This may
have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------

Any help with the issue will be greatly appreciated.

Thank you!

Originally posted by: Laci Ladanyi

The timer_create() function is available in the linux kernel, and before that glibc provided a userspace version. What is the kernel and the glibc version on the remote machine (mch2)? 'ldd --version' will tell you the glibc version.

Originally posted by: LM_OR

Laci, thanks for answering.

I see the version is 2.12. I will see if the computer cluster administrators are able to update it. Should this solve the issue?

Originally posted by: Laci Ladanyi

Ah, OK. This was a bug in cplex 12.6.1 and it has been fixed in 12.6.2. The issue is that the mpiworker shared library was not linked with -lrt (the real-time library).

So your options are either upgrading or modifying the mpirun command: besides specifying LD_LIBRARY_PATH you need to specify LDPRELOAD as well to make sure that librt.so is loaded before the cplex executable is. On my ubuntu system it'd be LD_PRELOAD=/lib/x86_64-linux-gnu/librt.so.1; on a redhat system LD_PRELOAD=/lib64/librt.so.1 , you need to find the correct location on your machine.

Originally posted by: LM_OR

Thank you very much, Laci. This Solved my issue. I am happily optimizing.