Decision Optimization

Expand all | Collapse all

Memory leakage using IloCP

  • 1.  Memory leakage using IloCP

    Posted Fri September 18, 2020 04:30 AM
    Edited by Sulivan Cheung Fri September 18, 2020 04:32 AM
    I am developing a CP optimization model using C++ and IloCP, and debugging with VLD (Visual Leak Detector) to catch memory leaks, however I cannot figure out where the leakages are from after numerous attempts. Finally I decided to use the following simple codes to give a test:
    #include <ilcp/cp.h>
     #include <vld.h>

     int main()
     {
         IloEnv env;
         IloModel model(env);
         IloCP cp(env);
         cp.extract(model);
         model.end();
         cp.end();
         env.end();
         return 0;
    }​

    Surprisingly, the above codes which create an empty model without solving it still give me a few memory leakages!
    Please help! Thanks a lot.

    ------------------------------
    Sulivan Cheung
    ------------------------------


  • 2.  RE: Memory leakage using IloCP

    Posted Fri September 18, 2020 04:57 AM
    Dear Sulivan,

    Could you please provide a log of the leaks.
    Also, what version of CPLEX Optimization Studio are you using?
    Cheers,

    ------------------------------
    Renaud Dumeur
    ------------------------------



  • 3.  RE: Memory leakage using IloCP

    Posted Fri September 18, 2020 08:14 AM
    Edited by Sulivan Cheung Fri September 18, 2020 08:19 AM
    Dear Renaud,
    Thank you for your reply. I am using version 12.10.0
    For the above test program the following information is generated by VLD:

    WARNING: Visual Leak Detector detected memory leaks!
    ---------- Block 204 at 0x00000000DE8765C0: 88 bytes ----------
    Leak Hash: 0x7C62920B, Count: 1, Total 88 bytes
    Call Stack (TID 27584):
    ucrtbased.dll!malloc()
    CPMemLeakTest.exe!0x00007FF697B052A3()
    CPMemLeakTest.exe!0x00007FF6970DB1D9()
    CPMemLeakTest.exe!0x00007FF6970DD61E()
    CPMemLeakTest.exe!0x00007FF696F601A0()
    CPMemLeakTest.exe!0x00007FF69701F082()
    CPMemLeakTest.exe!0x00007FF69701F5C4()
    CPMemLeakTest.exe!0x00007FF696F17A78()
    CPMemLeakTest.exe!0x00007FF696F1A2B4()
    CPMemLeakTest.exe!0x00007FF696EF4252()
    CPMemLeakTest.exe!0x00007FF696EF6484()
    CPMemLeakTest.exe!0x00007FF696DD0EDC()
    CPMemLeakTest.exe!0x00007FF696DADAED()
    CPMemLeakTest.exe!0x00007FF697B051E9()
    CPMemLeakTest.exe!0x00007FF697B0508E()
    CPMemLeakTest.exe!0x00007FF697B04F4E()
    CPMemLeakTest.exe!0x00007FF697B05279()
    KERNEL32.DLL!BaseThreadInitThunk() + 0x14 bytes
    ntdll.dll!RtlUserThreadStart() + 0x21 bytes
    Data:
    30 FC DD 97 F6 7F 00 00 FE FF FF FF FF FF FF FF 0....... ........
    C0 6B 00 00 CD CD CD CD 10 58 89 DE F8 01 00 00 .k...... .X......
    D0 0A 91 DE F8 01 00 00 00 00 00 00 00 00 00 00 ........ ........
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ........ ........
    CD CD CD CD CD CD CD CD B8 00 00 00 00 00 00 00 ........ ........
    00 00 00 00 00 00 00 00 ........ ........


    ---------- Block 1 at 0x00000000DE885AF0: 8 bytes ----------
    Leak Hash: 0x96AD00FB, Count: 1, Total 8 bytes
    Call Stack (TID 27584):
    ucrtbased.dll!malloc()
    CPMemLeakTest.exe!0x00007FF697B052A3()
    CPMemLeakTest.exe!0x00007FF6970E1DE3()
    CPMemLeakTest.exe!0x00007FF696F69AEB()
    CPMemLeakTest.exe!0x00007FF696D8D279()
    ucrtbased.dll!initterm() + 0x59 bytes
    CPMemLeakTest.exe!0x00007FF697B04FE9()
    CPMemLeakTest.exe!0x00007FF697B04F4E()
    CPMemLeakTest.exe!0x00007FF697B05279()
    KERNEL32.DLL!BaseThreadInitThunk() + 0x14 bytes
    ntdll.dll!RtlUserThreadStart() + 0x21 bytes
    Data:
    B0 00 00 00 00 00 00 00 ........ ........


    ---------- Block 2 at 0x00000000DE885BE0: 16 bytes ----------
    Leak Hash: 0x567284F1, Count: 1, Total 16 bytes
    Call Stack (TID 27584):
    ucrtbased.dll!malloc()
    CPMemLeakTest.exe!0x00007FF697B052A3()
    CPMemLeakTest.exe!0x00007FF696D8D5D3()
    ucrtbased.dll!initterm() + 0x59 bytes
    CPMemLeakTest.exe!0x00007FF697B04FE9()
    CPMemLeakTest.exe!0x00007FF697B04F4E()
    CPMemLeakTest.exe!0x00007FF697B05279()
    KERNEL32.DLL!BaseThreadInitThunk() + 0x14 bytes
    ntdll.dll!RtlUserThreadStart() + 0x21 bytes
    Data:
    03 00 00 00 CD CD CD CD 00 00 00 00 00 00 00 00 ........ ........


    ---------- Block 206 at 0x00000000DE895810: 16 bytes ----------
    Leak Hash: 0xC660CD2C, Count: 1, Total 16 bytes
    Call Stack (TID 27584):
    ucrtbased.dll!malloc()
    CPMemLeakTest.exe!0x00007FF697B052A3()
    CPMemLeakTest.exe!0x00007FF696DBB19C()
    CPMemLeakTest.exe!0x00007FF6970DB5AF()
    CPMemLeakTest.exe!0x00007FF6970DA24A()
    CPMemLeakTest.exe!0x00007FF6970DB1EB()
    CPMemLeakTest.exe!0x00007FF6970DD61E()
    CPMemLeakTest.exe!0x00007FF696F601A0()
    CPMemLeakTest.exe!0x00007FF69701F082()
    CPMemLeakTest.exe!0x00007FF69701F5C4()
    CPMemLeakTest.exe!0x00007FF696F17A78()
    CPMemLeakTest.exe!0x00007FF696F1A2B4()
    CPMemLeakTest.exe!0x00007FF696EF4252()
    CPMemLeakTest.exe!0x00007FF696EF6484()
    CPMemLeakTest.exe!0x00007FF696DD0EDC()
    CPMemLeakTest.exe!0x00007FF696DADAED()
    CPMemLeakTest.exe!0x00007FF697B051E9()
    CPMemLeakTest.exe!0x00007FF697B0508E()
    CPMemLeakTest.exe!0x00007FF697B04F4E()
    CPMemLeakTest.exe!0x00007FF697B05279()
    KERNEL32.DLL!BaseThreadInitThunk() + 0x14 bytes
    ntdll.dll!RtlUserThreadStart() + 0x21 bytes
    Data:
    D8 65 87 DE F8 01 00 00 00 00 00 00 00 00 00 00 .e...... ........


    ---------- Block 205 at 0x00000000DE910AD0: 40 bytes ----------
    Leak Hash: 0xBECCD097, Count: 1, Total 40 bytes
    Call Stack (TID 27584):
    ucrtbased.dll!malloc()
    CPMemLeakTest.exe!0x00007FF697B052A3()
    CPMemLeakTest.exe!0x00007FF696DBB19C()
    CPMemLeakTest.exe!0x00007FF6970DB45D()
    CPMemLeakTest.exe!0x00007FF6970DB59B()
    CPMemLeakTest.exe!0x00007FF6970DA24A()
    CPMemLeakTest.exe!0x00007FF6970DB1EB()
    CPMemLeakTest.exe!0x00007FF6970DD61E()
    CPMemLeakTest.exe!0x00007FF696F601A0()
    CPMemLeakTest.exe!0x00007FF69701F082()
    CPMemLeakTest.exe!0x00007FF69701F5C4()
    CPMemLeakTest.exe!0x00007FF696F17A78()
    CPMemLeakTest.exe!0x00007FF696F1A2B4()
    CPMemLeakTest.exe!0x00007FF696EF4252()
    CPMemLeakTest.exe!0x00007FF696EF6484()
    CPMemLeakTest.exe!0x00007FF696DD0EDC()
    CPMemLeakTest.exe!0x00007FF696DADAED()
    CPMemLeakTest.exe!0x00007FF697B051E9()
    CPMemLeakTest.exe!0x00007FF697B0508E()
    CPMemLeakTest.exe!0x00007FF697B04F4E()
    CPMemLeakTest.exe!0x00007FF697B05279()
    KERNEL32.DLL!BaseThreadInitThunk() + 0x14 bytes
    ntdll.dll!RtlUserThreadStart() + 0x21 bytes
    Data:
    D0 0A 91 DE F8 01 00 00 D0 0A 91 DE F8 01 00 00 ........ ........
    D0 0A 91 DE F8 01 00 00 01 01 CD CD CD CD CD CD ........ ........
    CD CD CD CD CD CD CD CD ........ ........


    Visual Leak Detector detected 5 memory leaks (428 bytes).
    Largest number used: 871788 bytes.
    Total allocations: 893304 bytes.
    Visual Leak Detector is now exiting.
    The program '[1740] CPMemLeakTest.exe' has exited with code 0 (0x0).

    And, in the project I am working with, VLD detects 2 memory leaks. I put the CP model in a function like this:
    void solve()
    {
      IloEnv env;
      try
      {
        IloModel model(env);
        IloIntervalVarArray... // Adding variables
        model.add(...) // Adding constraints and objective
        ...
        IloCP cp(model);
        if (!cp.solve())
        {
          if (cp.refineConflict())
          {
            cp.writeConflict(cp.out());
          }
          cerr << "CP optimisation failed! Infeasible.\n";
          throw IloAlgorithm::Exception("Failed to optimize");
        }
        else
        {
          cp.out() << "Solution value = " << setprecision(5) << cp.getObjValue() << endl;
        }
        model.end();
        cp.end();
      }
      catch (IloException & ex)
      {
        cout << ex << endl;
        cerr << "Concert exception caught" << endl;
        throw;
      }
      catch (...)
      {
        cerr << "Unknown exception caught" << endl; throw;
      }
      env.end();
    }​
    And I get the following memory leak info, even though I only declare but never call this function in my program.

    WARNING: Visual Leak Detector detected memory leaks!
    ---------- Block 1 at 0x00000000D60E1490: 8 bytes ----------
    Leak Hash: 0xAFC7AE0A, Count: 1, Total 8 bytes
    Call Stack (TID 26604):
    ucrtbased.dll!malloc()
    d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (35): AGSSP.exe!operator new() + 0xA bytes
    AGSSP.exe!IlcParallel::Lock() + 0x23 bytes
    AGSSP.exe!IlcCPONewClassId() + 0xB bytes
    AGSSP.exe!thunk@7ff753108610() + 0xFFB9 bytes
    ucrtbased.dll!initterm() + 0x59 bytes
    d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl (258): AGSSP.exe!__scrt_common_main_seh()
    d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl (331): AGSSP.exe!__scrt_common_main()
    d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp (17): AGSSP.exe!mainCRTStartup()
    KERNEL32.DLL!BaseThreadInitThunk() + 0x14 bytes
    ntdll.dll!RtlUserThreadStart() + 0x21 bytes
    Data:
    A0 01 00 00 00 00 00 00 ........ ........


    ---------- Block 2 at 0x00000000D60E1580: 16 bytes ----------
    Leak Hash: 0x79D2E04F, Count: 1, Total 16 bytes
    Call Stack (TID 26604):
    ucrtbased.dll!malloc()
    d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\heap\new_scalar.cpp (35): AGSSP.exe!operator new() + 0xA bytes
    AGSSP.exe!thunk@7ff753108610() + 0x10313 bytes
    ucrtbased.dll!initterm() + 0x59 bytes
    d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl (258): AGSSP.exe!__scrt_common_main_seh()
    d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl (331): AGSSP.exe!__scrt_common_main()
    d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp (17): AGSSP.exe!mainCRTStartup()
    KERNEL32.DLL!BaseThreadInitThunk() + 0x14 bytes
    ntdll.dll!RtlUserThreadStart() + 0x21 bytes
    Data:
    0D 00 00 00 CD CD CD CD 00 00 00 00 00 00 00 00 ........ ........


    Visual Leak Detector detected 2 memory leaks (128 bytes).
    Largest number used: 92971 bytes.
    Total allocations: 79038167 bytes.
    Visual Leak Detector is now exiting.

    ------------------------------
    Sulivan Cheung
    ------------------------------



  • 4.  RE: Memory leakage using IloCP

    Posted Fri September 18, 2020 10:59 AM
      |   view attached
    In addition, I also tested the embedded example of CP optimizer. For instance, the job-shop scheduling example, I follow the steps below:
    1. Open "installPath/cpoptimizer/examples/src/cpp/sched_jobshop.cpp";
    2. Build a project in Visual Studio (I am using VS Enterprise 2019 version 16.4.2), copy the codes into the project;
    3. Copy the data file "installPath/cpoptimizer/examples/data/jobshop_default.data" to a proper place in the directory of the project such that it can be read;
    4. Modify the properties of the project such that "ilcp/cp.h" can be properly included and "cp.lib" can be linked;
    5. Add "include <vld.h>" to the beginning of the codes;
    6. Compile and run.
    See the attachment for what I get in the console. There are 41 memory leakages detected.

    ------------------------------
    Sulivan Cheung
    ------------------------------

    Attachment(s)

    log
    Memory leak.log   35 KB 1 version


  • 5.  RE: Memory leakage using IloCP

    Posted Mon September 21, 2020 05:24 AM
    Dear Sulivan,

    The first leak is harmless and correspond to global mutex object needed by CPO. It rightly should stay there until the process exits and you can ignore it.
    The second leak is due to the MSVC runtime lib (likely for the same kind of  reasons as the CPO leak).
    I hope this helps,

    Cheers,

    ------------------------------
    Renaud Dumeur
    ------------------------------



  • 6.  RE: Memory leakage using IloCP

    Posted Fri September 25, 2020 01:33 AM
    I am not sure whether there is or there is no memory leak, CPLEX developers are the best personnel to answer that.

    Allow me to point out that Visual Leak Detector (VLD) is quite possibly out of development and may not fit in with Visual Studio 2019 or so. 

    https://marketplace.visualstudio.com/items?itemName=ArkadyShapkin.VisualLeakDetectorforVisualC seems to indicate that development on that was stopped in 2017.

    I have had situations where VLD reports an error in Windows VS 2019 IDE, and yet the very same code is valgrind memory leak free on ubuntu. If possible, try to install Intel VTune and other packages from Intel Compiler Toolkit. That is a huge download however, but it fits in and integrates within Visual Studio IDE on Windows.

    All the best!

    ------------------------------
    CPLEX User
    ------------------------------