Originally posted by: GlcBrero

Hi guys,

I am running a Java application with a Cplex dependency on my Mac, but I keep getting the following error:

Error encountered while trying to solve MIP with CPLEX:

The native libraries were not found in the java library path.

Installing CPLEX should have set the environment variables right. Did you install CPLEX correctly?

To fix it, let your PATH (-> Windows) or LD_LIBRARY_PATH (-> Unix) environment variable point to:

<cplex_install_directory>/cplex/bin/<your_platform>

In my .bash_profile I have:

export LD_LIBRARY_PATH=/Applications/CPLEX_Studio128/cplex/bin/x86-64_osx

export DYLD_LIBRARY_PATH=/Applications/CPLEX_Studio128/cplex/bin/x86-64_osx

export DYLD_FALLBACK_LIBRARY_PATH=/Applications/CPLEX_Studio128/cplex/bin/x86-64_osx

Any idea on what could cause the problem? Everything works properly when I set -Djava.library.path=/Applications/CPLEX_Studio128/cplex/bin/x86-64_osx in my IDE. 

Thank you!

How exactly do you invoke your application in case of failure? Are you sure you are running in an environment that has .bashrc sourced? Can you print LD_LIBRARY_PATH and DYLD_LIBRARY_PATH from within your Java program and double-check that the values are correct?

Also, which JAR do you use? cplex.jar or oplall.jar?

Originally posted by: GlcBrero

Hi Daniel,

Thanks a lot for your answer! Regarding your questions:

1. I simply type "java -cp app.jar" + path to class with the main method I want to run
2. No, I am not sure about that, but I don't quite know how to deal with it. If I print LD_LIBRARY_PATH and DYLD_LIBRARY_PATH in my application I see that they are both NULL
3. cplex.jar

Thanks again!

Gianluca

Ok, the paths being NULL shows that the environment variables are not set the way you expect them to be.

One easy workaround is to just specify -Djava.library.path=... on the command as well, just like you do in the IDE. This is cumbersome but guaranteed to work.

Another option is to setup the environment variables in the shell in which you run your application. Like so

$ export DYLD_LIBRARY_PATH=/Applications/CPLEX_Studio128/cplex/bin/x86-64_osx
$ echo $DYLD_LIBRARY_PATH
$ java -cp app.jar ...

The first line sets the variable, the second prints it (double check the output) and the last line should run the application now with the correct paths in place.

Finally, you can still try to get these environment variables set up correctly in your global shell configuration. First thing to check is: are you running bash at all? What does "echo $SHELL" give? If you are not running bash then your shell may use a different way to setup environment variables. If you are running bash, can you try putting the environment variable settings into .bashrc in your home directory as well? After doing that, start a new terminal/console and issue "echo $DYLD_LIBRARY_PATH". That should print the settings you specified.

Originally posted by: GlcBrero

Hi Daniel,

Thank you for your answer!

I was actually specifying -Djava.library.path in the command line as well, but, as you said, it is a bit cumbersome, and it doesn't allow me to install other java applications using cplex via maven (I get some failures when tests are run). 

I am setting DYLD_LIBRARY_PATH in my .bash_profile. This means that whenever I print it in the bash session that runs the app I get "/Applications/CPLEX_Studio128/cplex/bin/x86-64_osx". However, this is null when I print it from the java application. This should mean that .bash_profile doesn't get sourced when I run the java application. I have tried to create a .bashrc, but it also doesn't get sourced.

Printing $SHELL gives me "/bin/bash" from both terminal and java application.

I don't know why the cplex installation hasn't properly set all these variables.

Thanks a lot!

Gianluca

For .bashrc to be sourced, I think you have to add it to .profile. I'm on Linux, so things may be a bit different, but my .profile file contains the following:

# if running bash
if [ -n "$BASH_VERSION" ]; then
    # include .bashrc if it exists
    if [ -f "$HOME/.bashrc" ]; then
        . "$HOME/.bashrc"
    fi
fi

That plus the library path lines being in .bashrc (rather than .bash_profile) works for me.
 

Originally posted by: GlcBrero

Thank you, Paul! 

Something is fishy here. As far as I understand, doing "echo $DYLD_LIBRARY_PATH" in a shell prints the expected value. Then, starting your Java application from the same shell and printing the environment variable from the Java application then the environment variable shows up as null.

AFAIK the environment is read-only, so the Java app should have no way to clear that environment variable. How exactly are you printing the variable's value from Java? I can see these reasons for Java printing null:

1. Your code to print the environment variable is wrong.
2. In your Java application you are creating a subprocess (via ProcessBuilder or Runtime.exec() or similar) and don't correctly propagate the variables to the new process.
3. You have some sort of security manager installed that clears this environment on JVM startup (I am not sure such a thing exists)

By the way, your error message

Error encountered while trying to solve MIP with CPLEX:

The native libraries were not found in the java library path.

Installing CPLEX should have set the environment variables right. Did you install CPLEX correctly?

To fix it, let your PATH (-> Windows) or LD_LIBRARY_PATH (-> Unix) environment variable point to:

<cplex_install_directory>/cplex/bin/<your_platform>

Does not look like a message that CPLEX would write. Does this message come from your code? Or are you using CPLEX through some 3rd party library and that issues this error? In that case maybe there is a problem with the way this library loads CPLEX.

Do things work correctly if you invoke your application like this:

java -Djava.library.path=$DYLD_LIBRARY_PATH ...

?

Originally posted by: GlcBrero

Ok, it is all very fishy.

Yes, "echo $DYLD_LIBRARY_PATH" prints the expected value, and starting Java from the same shell prints this environment variable as null. Also, everything works under "java -Djava.library.path=$DYLD_LIBRARY_PATH ..."

To print the environment variables from java, I simply do 

        Map<String, String> envVars = System.getenv();

        System.out.println("DYLD_LIBRARY_PATH = "+envVars.get("DYLD_LIBRARY_PATH"));

Last weird thing: if I modify the environment variable $PATH in the same shell that runs java, and I print the PATH variable from java, I get the modified version. And the same happens if, instead of java, I just run a bash script that prints variables: $PATH gets modified, "DYLD_LIBRARY_PATH" is null. Thus, the environment variable DYLD_LIBRARY_PATH gets cleared, and this is not even a java issue. 

Finally, yes, this error comes from a third party library. Thus, I guess it is all unrelated to cplex! I'll mark this as solved and try to find some workaround.

Many many thanks for your help!!!

Gianluca

It is odd that PATH and DYLD_LIBRARY_PATH behave differently.

The only reason I can imagine for that is that for DYLD_LIBRARY_PATH you are missing an "export" for DYLD_LIBRARY_PATH (although your .bash_profile seems to indicate that it is there).

Last thing to try: before you start java, say "export DYLD_LIBRARY_PATH":

$ echo $DYLD_LIBRARY_PATH
$ export DYLD_LIBRARY_PATH
$ java ...

If that does not fix the problem then I am really at my wits end. Unless you have some weird typo somewhere, I never saw anything like that before.

Originally posted by: GlcBrero

Doesn't work. I'll let you know if I ever find a solution. Thanks for your support!

This is interesting: https://gitlab.haskell.org/ghc/ghc/issues/11617 (although it is three years old).

I am not an expert on MacOS but it seems under some circumstances, DYLD_LIBRARY_PATH can be removed from a processes environment on startup.

Similar discussions that indicate that what you observe may be expected on recent MacOS:

https://www.mathworks.com/matlabcentral/answers/374930-append-library-path-to-dyld_library_path-in-mac

https://github.com/nasa/europa/issues/181

https://support.mulesoft.com/s/article/Variables-LD-LIBRARY-PATH-DYLD-LIBRARY-PATH-are-ignored-on-MAC-OS-if-System-Integrity-Protect-SIP-is-enable

So I have a feeling this is indeed completely unrelated to CPLEX. You will probably have the same issue for any 3rd party library. I am not sure this is the best solution but you could create a symlink to the CPLEX libraries from a place that is in the default library search path. Or maybe there is a way to tell MacOS to also check the CPLEX directories when looking for a library.

Originally posted by: GlcBrero

Thank you very much, Daniel!

I solved the problem by disabling the  MAC OS System Integrity Protect (SIP) as described in one of the links you sent me:

 https://support.mulesoft.com/s/article/Variables-LD-LIBRARY-PATH-DYLD-LIBRARY-PATH-are-ignored-on-MAC-OS-if-System-Integrity-Protect-SIP-is-enable

This was completely unrelated to CPLEX, and thanks for helping me anyway!

Best,

Gianluca