Originally posted by: tnad

Our company just purchased cplex and the latest version crashes every time in matlab 2016a and 2016b with an access violation error. It's supposed to work with versions 2015a+ according to the IBM's website. This is really frustrating.

I spent the whole weekend googling many many people having similar issues with no solution. Can anyone help me with the latest combination of matlab and cplex versions that they know works very well? The analysis I run doesn't crash matlab 2011b/cplex 12.4 which are too old. I need to use newer matlab versions.

CPLEX 12.7.1 is indeed supported using MATLAB versions from 2015a to 2017a.  As a commercial customer, you are entitled to the regular IBM Support. I therefore encourage you to open a support request at https://www.ibm.com/support/servicerequest/ (that's the link 'Open a ticket'-'Hardware and software' in the middle of the Support Portal page https://www.ibm.com/support/home/).

In order to investigate, the Support team will need to be able to reproduce the issue. Or at the very least more details, such as a call-stack.

Originally posted by: tnad

I just wanted to update that 12.7 has the same problem, but 12.63 worked perfectly without crashes on matlab 2016a.

It will probably take a considerable effort to troubleshoot 12.7 / 12.71 with IBM support since I'm using several 3rd party matlab toolboxes and I'll have to contact them to figure out how I can supply a code and data that reproduces the issue.

Originally posted by: Laci Ladanyi

Hello,

Unfortunately we've been getting a few reports on the forum here about this issue, but we just can't reproduce it :-(. If you can create a stack trace for us, that would be great. However, when matlab crashes, it just creates a crash log only for the matlab part, so that is not sufficient. The way to get a full stack trace is to start matlab itself under a debugger. 

If you are running things on linux, follow the instructions at https://www.mathworks.com/help/matlab/matlab_external/debugging-on-linux-platforms.html. Of course, you'd skip compiling the mex file, you'd just start matlab under gdb, start it with the flags given on the link, and eventually you'll crash. At that point you should be dropped into a gdb prompt, and you can access the backtrace with the bt command.

Under windows it's very similar (https://www.mathworks.com/help/matlab/matlab_external/debugging-on-microsoft-windows-platforms.html) : start matlab, simultaneously start VS, then attach to the matlab process. Then just continue, and when you have a matlab crash, in VS you should be able to get the trace.

Note that the way matlab operates is that threads can crash under the jvm, the jvm cleans it up, and you can just continue! This sometimes makes it rather hard to know whether you should continue until the next crash, or you have the one you are interested in. On linux the -nojvm flag takes care of this, so it's easier there. On windows... so far I did not manage to disable the jvm.

In any case, let me know if you manage to create a stack trace, and we'll take it from there.

Start matlab itself under a debugger

Originally posted by: tnad

Thanks. I actually have no experience in debugging, so I figured out that I needed to download Visual studio and install the extra debugging addons, then attach the process. After that, it kept throwing exceptions while running the code at locations it never crashed before, so I kept hitting continue and it never crashed. I'm not sure how to do this without exact instructions.

On a side note, when I add a breakpoint at the line where matlab crashes and then I continue immediately the whole analysis runs successfully without crashing. Without the breakpoint, it always crashes when that line runs. I don't know if that helps at all.

Originally posted by: Laci Ladanyi

You've done the right thing. The exceptions thrown are the crashes in the jvm that I have mentioned, so you have correctly continued. It's odd that it never crashes that way :-(. It's also odd that that with just the breakpoint and immediately continuing everything works. Could you copy here the line where it crashes if there is no breakpoint on it? 

BTW, do you use the toolbox or the classes? If you use the toolbox, could you try to use the classes instead? That may narrow things down (and may provide a workaround if it helps). Or, if you use classes now, could you try the toolbox instead (though that is a harder change...)?

Finally, could you enable the datacheck parameter? Just to make sure that all data coming into cplex is "clean".

Thanks!

Originally posted by: AsToN

Hi,

here you have a copy of a detailed crash report taken from Visual Studio 2017. See also my last post in https://www.ibm.com/developerworks/community/forums/html/topic?id=79772c01-f545-44ef-9543-257197f1d9e7&ps=25#repliesPg=1

The resulting crash report is attached. The first lines are immediately thrown when VS attaches to the MATLAB process. Beginning with line 430 the report is about the crash which occurs when I start my MATLAB code that uses a CPLEX.

I hope this brings a little more light into this matter.

Best AsToN

Originally posted by: Laci Ladanyi

We have a few guesses what could happen, but it's all too vague. Is there any chance that you could share the matlab code with us? If so, just email it to me directly to ladanyi at us.ibm.com. If not, there is one more thing you could do to help us out: Do everything as before, but also within matlab execute the statements one-by-one (well, at least when you are in the cplex matlab api :-) and let us know the line where the crash occurs. But if we can get the source, that would be the best, of course. Naturally, we would not use it for anything else than tracking this issue down.

--Laci

Originally posted by: AsToN

Ok, I have prepared a little example using parts of my code and based on a test dataset which I also provide. The m-file 'crashtestCplex' runs the code and the relevant function called is 'crashtestCplexSolve'. The rest just builds/finishes the data. Note that it is not the full code but it includes the relevant parts. Plus it behaves exactly like the full version, I have checked that. So it does show the same weird behaviour throwing fatal erros in an irregular manner. But remember, I also provided a little example once, based on one of CPLEX' very own examples (blend.m) and this also crashed on my system but apparantly not on yours. However, this is the best I can do, and if you don't get this to crash then I guess it is specific to certain system setups.

A few notes on what I have observed so far:

First, the error is not dependent on running the code in parallel (parfor loop for example) or if you run it in a standard sequential loop. Probably you don't need to loop at all, I just do it because it maximizes the chances of a crash (since the error is irregular).

Second, the error seems to involve the building of the object class 'cplex'. It usually does not immediately crash when invoking model = Cplex(); but rather a few moments later. From what I observe when I debug the code step by step the cplexlink-mex file doesn't properly build the object structure. That means it does build the upper levels of the structure but it fails to create the structure fields on the lowest of levels, specifically of the 'Params'-substructure (e.g. the field 'Cur' where you alter the parameter values). Those fields are either not created at all or they are created but not filled with values (i.e. they are completely empty).

Now this failure of creating the object's structue of the 'cplex' class seems to cause the crash. Now usually Matlab would simply throw an ordinary error when trying to access fields which are not existing. However, even though that happens sometimes (namely whenever the procedure survives long enough to allow Matlab to throw this error), the usual behaviour is that Matlab simply terminates with a fatal crash. So summing up, Matlab orders to builds the cplex object, the mex funtion fails to so properly (it builds an incomplete structure), Matlab (or if no further parameters are changed, Cplex itself when trying to start its solving procedure) tries to access fields which don't exist. But Matlab usually doesn't even get to throwing an eception since the cplex-mex function terminates with an error causing Matlab to crash.

Note also that it doesn't help to pause a while after invoking the cplex-object in order to allow the mex-function some more time. I've tried and it does not seem to change anything.

Third, whenever the structure is build properly the whole procedure is fine. It never crashed during an active Cplex solving procedure.

Forth, the error is happening on different operating systems. I personally tried RedHat Linux and Windows 7 both with Matlab R2017a and CPLEX 12.7.1 (both are academic versions).

I hope this helps!

Best, AsToN

Originally posted by: AsToN

Two more things:

The crashes usually appear very early in the loop. When I run the provided testfile it always crashed on the first iteration. Once you restart Matlab it worked fine. However, I can not guarantee that this is always valid. I think I can remember times when it crashed somewhere in the middle of performing iterations or even crashed every time I tried.

And by the way I now that I forgot to add an instruction in my loop that destroys the cplex-object after each completed optimization. You can add it. But to my experience it doesn't change anything.

We found a problem in our code that could very well have triggered these crashes. The problem is fixed in version 12.8.

Originally posted by: AsToN

Yes, I can confirm that it fixed my problems. Well done!