Decision Optimization

Expand all | Collapse all

Using Rcplex on CPLEX (studio) 129 or later

  • 1.  Using Rcplex on CPLEX (studio) 129 or later

    Posted Tue July 21, 2020 02:27 PM
    Dear IBM Decision Optimization Community,


    We need to use Rcplex on CPLEX Studio 12.9 or later. However, using linux Rcplex install only in CPLEX 12.6.3 (Error: checking for library containing CPXversion... no).


    Rcplex interface: Rcplex(cvec, Amat, bvec, Qmat = NULL, lb = 0, ub = Inf, control = list(), objsense = c("min", "max"), sense = "L", vtype = NULL, n = 1)

    How to use Rcplex on CPLEX12.9 or later?



    Links:
    Rcplex - https://cran.r-project.org/web/packages/Rcplex/index.html
    Rcplex test - https://cran.r-project.org/web/packages/Rcplex/NEWS (….Tested with
    the IBM ILOG CPLEX Community Edition 12.6.3)

    ------------------------------
    Sérgio Dias
    ------------------------------


  • 2.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Wed July 22, 2020 01:24 AM
    Edited by ALEX FLEISCHER Wed July 22, 2020 02:10 AM
    Hi

    you could use Cplexapi which is a more recent r package 

    https://cran.r-project.org/web/packages/cplexAPI/index.html

    regards

    PS:

    You could also have a look at https://developer.ibm.com/docloud/blog/2018/10/26/use-opl-r-framework/

    and

    https://community.ibm.com/community/user/datascience/communities/community-home/digestviewer/viewthread?MessageKey=ff87cd26-1963-41a4-b405-f025069f929b&CommunityKey=ab7de0fd-6f43-47a9-8261-33578a231bb7&tab=digestviewer#bmff87cd26-1963-41a4-b405-f025069f929b


    ------------------------------
    ALEX FLEISCHER
    ------------------------------



  • 3.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Thu July 30, 2020 05:14 PM
    Actually, I have the same installation problems with cplexAPI that I have with Rcplex (with CPLEX 12.10). In both cases, the make file seems to have trouble finding the right bits of CPLEX, no matter which options I specify. I wonder if, between CPLEX 12.6 and 129, something was reorganized in the CPLEX Studio directory hierarchy that is breaking both installers?

    Paul

    ------------------------------
    Paul Rubin
    Professor Emeritus
    Michigan State University
    ------------------------------



  • 4.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Fri July 31, 2020 12:22 AM
    Support for 32bit was dropped, that may be the reason? In order to track this down, it would be good to know what exactly the installation script is checking and how excatly that fails.

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 5.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Fri July 31, 2020 06:36 PM
    I don't think it's an issue with 32 bit support, although I can't be sure. What I know about compiling C code is pretty much limited to how you spell 'C'. I just retried the cplexAPI installer, giving it the path to the 'cplex' directory in the CPLEX_Studio installation as an argument. I think the compilation succeeded. The installation script said it found ilcplex/cplex.h, compiled with a couple of warnings that meant nothing to me, then installed to a temporary location. The script then tried to load cplexAPI from the temporary directory and failed due to "undefined symbol: CPXgetparamhiername" (which is defined in cplex.h). At this point, I am officially lost.

    ------------------------------
    Paul Rubin
    Professor Emeritus
    Michigan State University
    ------------------------------



  • 6.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Mon August 03, 2020 12:48 AM
    Edited by Daniel Junglas Mon August 03, 2020 01:11 AM
    This is very weird. I just checked the 12.10 libraries and the CPXparamhiername() function in the static as well as the dynamic library.

    I cannot test this myself here since I don't have R installed. I am not clear how R handles these additional packages like the cplexAPI connector. Is this a shared library?

    In any case, given that you could build the thing and then only execution failed, it seems that you already got one step further than the OP?

    I also took a quick look at the files in cplexAPI_1.3.6.tar.gz. In src/Makevars.win you have this:
    PKG_CPPFLAGS=-g -D_R_=1 -DUSE_R=1 -I"${CPLEX_STUDIO_DIR}\cplex\include"
    PKG_LIBS=-L"${CPLEX_STUDIO_LIB}" -lm -lcplex1261
    So it seems that at least on Windows the thing is hard-coded to link with the 12.6.1 library, which would of course not work in case you have a different version. Maybe it helps to adjust the version in this file before running configure? Probably also double check the content of environment variable $CPLEX_STUDIO_DIR to make sure it points to the right installation location.

    I guess the only way we can make progress here is if you send the full log of configure and make.

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 7.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Mon August 03, 2020 10:02 AM
    I thought I was the only one with this problem.
    So how can I solve this?

    ------------------------------
    Angela Fox
    DS Diamond
    ------------------------------



  • 8.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Mon August 03, 2020 10:14 AM
    The best way to investigate would be if someone could post a full log of the configure and make output. Both packages are not officially supported by IBM, so we know as much about this software as you. We may however be able to figure something out by staring at that output.

    Moreover, there have been two errors reported in this thread so far:
    1. Failure to find CPLEX libraries during configure
    2. Undefined symbols when trying to link the CPLEX library
    What exactly is your problem?

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 9.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Mon August 03, 2020 11:55 AM
    I've attached three text files. The configuration output is in log.txt. This is the result of attempting installation (on Linux Mint, with CPLEX Studio 12.10) using the command
       R CMD INSTALL --configure-args=" 
       --with-cplex-dir='<cplex_dir>/cplex'" \
       cplexAPI_x.x.x.tar.gz​

    which is one of the installation options suggested in the package documentation. I assume the configure and (hopefully) make output are both in the log. I've also attached the configuration script itself (in configure.txt -- it's actually a shell script). Finally, the NAMESPACE.txt file is part of the package. At the end of the file there's a section about changes in CPLEX 12.9, which would seem to imply that the package authors had updated it to work with 12.9.

    I also attempted installing it against CPLEX 12.9 and 12.8, using the same installation command (changing only the CPLEX directory). Installation against 12.9 failed with exactly the same error experienced with 12.10. Installation with 12.8 succeeded, however. The attached file "log3.txt" is the installer output from the successful attempt.

    Paul



    ------------------------------
    Paul Rubin
    Professor Emeritus
    Michigan State University
    ------------------------------

    Attachment(s)

    txt
    configure.txt   142K 1 version
    txt
    NAMESPACE.txt   19K 1 version
    txt
    log.txt   4K 1 version
    txt
    log3.txt   3K 1 version


  • 10.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Tue August 04, 2020 03:12 AM
    The error seems pretty obvious:
    In case the installation fails you have this command line to link the cplexAPI.so:

    gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o cplexAPI.so check.o cplexAPI.o cplexR.o cplex_checkAPI.o cplex_longparamAPI.o init.o -L/home/paul/ILOG/CPLEX_Studio1210/cplex/lib/x86-64_linux/static_pic -lm -lpthread -ldl -L/usr/lib/R/lib -lR

    In case the installation succeeds, you have this linker command:

    gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o cplexAPI.so check.o cplexAPI.o cplexR.o cplex_checkAPI.o cplex_longparamAPI.o init.o -L/home/paul/ILOG/CPLEX_Studio128/cplex/lib/x86-64_linux/static_pic -lcplex -lm -lpthread -ldl -L/usr/lib/R/lib -lR

    as you can see in the latter line, there is `-lcplex` which tells the linker to include the CPLEX library. This item is missing from the former and explains why you get undefined symbols.

    It seems the installation script attempts to extract the linker flags from CPLEX example Makefiles. Maybe this extraction fails for more recent versions of CPLEX. I will investigate ...

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 11.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Tue August 04, 2020 04:24 AM
      |   view attached
    Ok, between 12.8 and 12.9 the respective item in the CPLEX example Makefiles changed from -lcplex to -l$(CPLEXLIB). This makes the installation script fail to correctly extract the linker flags.
    I have attached a modified version of the configure script. Could you please test this? The important thing is that you need to have -lcplex on the linker command line.

    For sake of completeness, here is also the patch that turns the original configure script into one that should work:
    --- configure.txt.old	2020-08-04 10:04:20.822665817 +0200
    +++ configure.txt	2020-08-04 10:07:34.967563040 +0200
    @@ -3314,6 +3314,8 @@
     
         if test  "${MAKEFILEVERSION}" = "-lcplex"  ; then
             CPLEX_LIBS="-L${CPLEXLIBDIR} `${AWK} 'BEGIN {ORS = " "} /^CLNFLAGS/ { for (i=3;i<=NF;++i) print $i }' ${CPLEX_MAKEFILE}`"
    +    elif test  "${MAKEFILEVERSION}" = '-l$(CPLEXLIB)'  ; then
    +        CPLEX_LIBS="-L${CPLEXLIBDIR} -lcplex `${AWK} 'BEGIN {ORS = " "} /^CLNFLAGS/ { for (i=4;i<=NF;++i) print $i }' ${CPLEX_MAKEFILE}`"
         else
             CPLEX_LIBS="-L${CPLEXLIBDIR} `${AWK} 'BEGIN {ORS = " "} /^CLNFLAGS/ { for (i=4;i<=NF;++i) print $i }' ${CPLEX_MAKEFILE}`"
         fi​
    In case the update script works, you may want to share this patch with the maintainers of cplexAPI.

    ------------------------------
    Daniel Junglas
    ------------------------------

    Attachment(s)

    txt
    configure.txt   142K 1 version


  • 12.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Tue August 04, 2020 11:54 AM
      |   view attached
    Daniel,

    Thanks for the patched script. Unfortunately, it did not work. Again, it compiles, attempts to install to a temporary home, and then balks. This time it complains that CPXcopypartialbase is an undefined symbol. (This was with CPLEX 12.10.)

    Paul

    ------------------------------
    Paul Rubin
    Professor Emeritus
    Michigan State University
    ------------------------------

    Attachment(s)

    txt
    log.txt   4K 1 version


  • 13.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Wed August 05, 2020 12:25 AM
    Function `CPXcopypartialbase()` was removed in version 12.10. This was not announced in the release notes because the function was broken and so we figured nobody ever used it. We did not consider the case in which a connector like cplexAPI would just wrap all functions it finds.

    I guess you can just remove all mentions of copyPartBase and CPXcopypartialbase from the cplexAPI sources and try again. Or build the module against 12.9.

    On the bright side, the link line in your logfile now looks correct.

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 14.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Tue August 04, 2020 02:12 PM
    I've written to someone from each project (Rcplex and cplexAPI) regarding the installer issues. If I hear anything from either of them, I will pass it along.

    ------------------------------
    Paul Rubin
    Professor Emeritus
    Michigan State University
    ------------------------------



  • 15.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Mon August 10, 2020 04:27 PM
    Hi Dr.Rubin,

    Were you able to resolve this after speaking to the package maintainers for Rcplex / cplexAPI? I (and others in the O.R. community) are facing the same issue.

    ------------------------------
    Prashanth Sriram
    ------------------------------



  • 16.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Mon August 10, 2020 04:33 PM
    I'm working on it. One of the people I contacted is on vacation, and promised to look at it when he got back. The other replied today and suggested that I contact a different member of the project team (which I am trying to do). I'll post here when I have something to report (unless the developers beat me to it).

    ------------------------------
    Paul Rubin
    Professor Emeritus
    Michigan State University
    ------------------------------



  • 17.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Tue August 11, 2020 12:35 AM
      |   view attached
    I have generated a patch for the cplexAPI-1.3.6 sources. It is supposed to remove references to CPXcopypartialbase() from the sources. I have attached the patch. I am also attaching the patch for the configure script. Maybe you can try the two and see whether things work afterwards? As stated before, I cannot try compiling this myself since I cannot (easily) install R here.

    ------------------------------
    Daniel Junglas
    ------------------------------

    Attachment(s)

    zip
    patches.zip   3K 1 version


  • 18.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted Tue August 11, 2020 02:22 PM
    Daniel,

    Thanks for all your work on this. The source code patch was a little tricky to apply (file paths in the patch file differ from those in the package download, and I had to delete one file manually). Once patched, installation using the option where the user specifies just the path to the 'cplex' directory in CPLEX Studio worked! I tested the package but running the small LP test in the package help, and it ran as expected.

    I have taken the liberty of sending your patch file, along with my comments, to Dr. Mayo Röttger, the cplexAPI maintainer. There are three versions of the configure file (the one you patched, one with a .win extension that I assume is for Windows, and one with a '.ac' extension that I gather is for people running R on an air conditioner). He will need to update the other two configure files, but that should be easy.

    Again, thanks for doing this.

    ------------------------------
    Paul Rubin
    Professor Emeritus
    Michigan State University
    ------------------------------



  • 19.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted 27 days ago
    Dear All,

    I am able to install RCplex_0.3_3 and cplexAPI_1.3.6 packages using "CPLEX_Studio_Community129" in Windows-10 PC.

    I have installed "R" version 4.0.2 that has following specifications:
    • platform       x86_64-w64-mingw32
    • arch            x86_64
    • os               mingw32
    • system       x86_64, mingw32
    • status
    • major        4
    • minor        0.2
    • year         2020
    • month      06
    • day         22
    • svn rev    78730
    • language R
    • version.string R version 4.0.2 (2020-06-22
    Also, I installed "Rtools40" and accordingly modified "PATH" and "BINPREF" variables to recognize the rtools compilers.

    Then, I made the following changes in,
    Rcplex_0.3-3:
    • In src/Makevars.win file, I included the path for "CPLEX_Studio_Community129" as
               ifeq "$(WIN)" "64"
                   PKG_LIBS = -L"C:/Program Files/IBM/ILOG/CPLEX_Studio_Community129/cplex/bin/x64_win64" -lcplex1290 -lm
                   PKG_CPPFLAGS = -D_LP64 -I"C:/Program Files/IBM/ILOG/CPLEX_Studio_Community129/cplex/include"
              else
                   PKG_LIBS = -L"C:/Program Files/IBM/ILOG/CPLEX_Studio_Community129/cplex/bin/x86_win32" -lcplex1290 -lm
                   PKG_CPPFLAGS = -I"C:/Program Files/IBM/ILOG/CPLEX_Studio_Community129/cplex/include"
               endif
    • Then, I created the .tar.gz file using "R CMD build --no-build-vignettes --no-manual --md5 Rcplex_0.3-3
    • Finally, R CMD INSTALL --build --no-multiarch .\Rcplex_0.3-3.tar.gz 
    cplexAPI_1.3.6:
    • In src/Makevars.win file, I included the path for "CPLEX_Studio_Community129" as
                  PKG_CPPFLAGS=-g -D_R_=1 -DUSE_R=1 -I"C:\Program Files\IBM\ILOG\CPLEX_Studio_Community129\cplex\include"
                  PKG_LIBS=-L"C:\Program Files\IBM\ILOG\CPLEX_Studio_Community129\cplex\bin\x64_win64" -lm -lcplex1290

    • Then, I created the .tar.gz file using "R CMD build --no-build-vignettes --no-manual --md5 cplexAPI_1.3.6
    • Finally, R CMD INSTALL --build --no-multiarch .\cplexAPI_1.3.6.tar.gz

    By, doing this I successfully installed both the packages.


    Thanks & Regards
    -------------------------------------------------
    Ramachandra M Katabathula
    Research Associate
    Case Western Reserve University
    -----------------------------------------------

    ------------------------------
    Ramachandra Katabathula
    ------------------------------



  • 20.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted 27 days ago
    Edited by Prashanth Sriram 27 days ago
      |   view attached
    Edit: Nevermind, I figured it out and it works now. Thanks!

    For the sake of others stuck with the same problem. For installing Rcplex on Linux, running Cplex 12.9, I followed the following steps:
    1. wget https://cran.r-project.org/src/contrib/Rcplex_0.3-3.tar.gz
    2. tar xzvf Rcplex_0.3-3.tar.gz
    3. vi ./Rcplex/configure
    Go to line number 1548 where CPLEX_LIBS is defined.
    I didn't know how to fix the AWK expression, so I cheated and hard-coded it. I changed the line to:
    CPLEX_LIBS="-L${CPLEXLIBDIR} -lcplex -lm -lpthread -ldl"
    4. R CMD INSTALL Rcplex --configure-args="--with-cplex-dir=/opt/ibm/ILOG/CPLEX_Studio129/cplex"
    change the cplex path to whatever it is on your system.

    ---
    Hi Daniel,
    Due to a package dependency, I have to specifically install the package Rcplex: https://cran.r-project.org/src/contrib/Rcplex_0.3-3.tar.gz  (and not cplexAPI)
    Can you help me fix the config file for this?
    I am attaching the log file when the installation fails.

    Thanks.

    ------------------------------
    Prashanth Sriram
    ------------------------------

    Attachment(s)

    log
    config.log   20K 1 version


  • 21.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted 27 days ago
    The issue is the same as in the other packacge that we discussed here. According to your output you have -l$(CPLEXLIB) in LIBS and PKG_LIBS. This should be -lcplex for non-Windows platforms.
    The line to fix in the configure script is this one:
    ```
    CPLEX_LIBS="-L${CPLEXLIBDIR} `${AWK} 'BEGIN {FS = " = "} /^CLNFLAGS/ {print $2}' ${CPLEX_MAKEFILE}`"
    ```
    You can try to replace this with (I did not test that)
    ```
    CPLEX_LIBS="-L${CPLEXLIBDIR} `${AWK} 'BEGIN {FS = " = "} /^CLNFLAGS/ {print $2}' ${CPLEX_MAKEFILE} | sed -e 's,\$(CPLEXLIB),cplex,'`"
    ```
    and see if that helps.

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 22.  RE: Using Rcplex on CPLEX (studio) 129 or later

    Posted 25 days ago
    Thanks Daniel! This worked for me (Linux Mint, CPLEX 12.10, R 3.6.3, Rcplex 0.3-3).

    ------------------------------
    Paul Rubin
    Professor Emeritus
    Michigan State University
    ------------------------------