Originally posted by: FidanBoylu

Hi all,
I am using cplex excel connector and My model solves for small size problems, however for larger sizes I keep on getting Error 6: CPLEX engine error. Can anybody let me know what would be the causes of that error? Any debugging procedures you can recommend to understand this?
thank you!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I have the same problem with Cplex-Excel. Bye.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Attach my file.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

is there any documentation regarding these error codes somewhere? they don't really match with Cplex ones with the 4 digit codes. I've been searching and searching,I am seriously in the dark here. it is really annoying since the problem solves for smaller cases so can't say it is something with the model. without knowing what an engine error is, it is really difficult to make a guess..
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I try to migrate a model from What's Best to Excel-Cplex (12.2), but Cplex gives many errors.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

There are lots of potential reasons for "CPLEX engine error". Basically this means that one of the low level CPLEX functions failed unexpectedly.
Since it seems only to happen with large models this may be an out-of-memory situation. Do you happen to observe high memory load right before you get this error?
Can you try to export the model as a SAV file (enter "model.sav" for the "Model to file" option in the GUI or use the ExportModel and ModelFile arguments to the CPXsetSpecial() macro) and solve it with the interactive optimizer? Does it solve with the interactive optimizer or does it show any errors there?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

Daniel,
I have been trying to export to a file for a while, the problem remains, I get the same engine error for some offending cell. So it is not even able to export to file before running the optimizer. I have checked the memory usage and it just goes up from %30 to %35 when I hit the run button. I am attaching my file here. you just need to run module 1. Please help:)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

I have 2240 binary variables and 13,740 constraints for that model I attached (was hoping to go up). would that be the problem? how large can you go? I am running on an 8GB machine.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

Sorry, 6740 binary variables and 13,740 constraints.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

well, I feel that these are very miniscule number of variables from what I read in the forums. I just fail to see what the problem is..
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

With What'sBest I run model with 550,000 variables an 150.000 constraints.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

My model has 250.000 continuous variables and 500 constraints, and the memory usage is %34. I am running on an 8GB machine. Bye.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Thanks for the example model. I reproduced the error here and it is indeed an out of memory error. There is no hard limit to the number of constraints, just the memory limit.
I don't know yet whether the out of memory situation we observe here is really out of memory or a bug. I will keep digging ...
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

By the way: which version of Excel are you using? What is the version number? Is it 32bit or 64bit. Do you have a 32bit or 64bit machine?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

Daniel, I am using Excel 2010 version 14.0.6112.5000 32-bit and I am on a 64-bit machine.
I have cplex_studio122.acad.win-x86-64 installed..
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I finally got to the bottom of that issue. It is not a bug. It is rather a sort of internal limitation. The way in which the SUM statement is handled results in large memory consumption which eventually results in the out-of-memory problem.
Unfortunately there is no other workaround than avoiding SUM or SUMPRODUCT with variable cells -- which is most probably not feasible.

In general, for very large models it might be better/faster to use CPLEX directly from C# or VB and drive Excel from there.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

Daniel,
I was able to remove all SUM and SUMPROD functions from the model except the objective function from which it was not possible to remove. I didn't really need to use the those functions in the constraints at all, was just using to shorten things and more importantly because of the fact that from my readings of the cplex excel manual, , I felt that it was encouraged to use those functions (there was a paragraph about how cplex recognizes these functions).

Now, although I haven't tried the full dataset yet, I am now able to solve larger models that I wasn't able to solve earlier when we started the discussion. THANK YOU for looking into this!

However, I suggest the removal of these functions from the manual and instead a warning about these functions should be added.

Just my two cents to improve the software. Overall, I think the excel connector is a very user friendly product but some improvements on these limitations are necessary such as revision of how sum or sumprod is handled in a more memory friendly way.

thanks again.
Prof. Boylu
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

> I was able to remove all SUM and SUMPROD functions from the model except the objective function from which it was not possible to remove. I didn't really need to use the those functions in the constraints at all, was just using to shorten things and more importantly because of the fact that from my readings of the cplex excel manual, , I felt that it was encouraged to use those functions (there was a paragraph about how cplex recognizes these functions).
>
Ugh, I did not even think that this was possible.

> Now, although I haven't tried the full dataset yet, I am now able to solve larger models that I wasn't able to solve earlier when we started the discussion. THANK YOU for looking into this!
>
> However, I suggest the removal of these functions from the manual and instead a warning about these functions should be added.
>
Even better: We should change the code so that it no longer has problems with these functions :-)

> Just my two cents to improve the software. Overall, I think the excel connector is a very user friendly product but some improvements on these limitations are necessary such as revision of how sum or sumprod is handled in a more memory friendly way.
>
Thank you for your feedback.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Fidan, but don't use SUM or SUMPROD in big problems is very costly. I think that CPLEX team must to improve this. Which was the run time for your problem? found the optimal solution?

Rodrigo
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

Rodrigo,
I was able to avoid sum and sumprod for my model but it may not be feasible for all models, I hope they fix this in the near future. One of the models that is about 1/4th of the problem itself, solved less than 20 mins. I used half the real size,it ran for an hour, came back with "failed to optimize".

Daniel,
I am trying to install the academic initiative latest versions 12.4 (also tried 12.3) to some machines I have here without any luck. Version 12.4 gets stuck in the third step while installing Uninstall files.. 12.3 installs very quick only with empty folders except example files. is there a problem with the links maybe?

thanks!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Fidan, Thank you for your answer, maybe you must programming your problem in C++, is faster and work with larger problems.

Rodrigo
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

Hi Daniel, can you comment on the issues with the academic version installations? I have tried installing on different computers here, doesn't work on any of them.
Can you please check if the download and install files are ok for version 12.4 or 12.3 ...
Thanks!!!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Could you please create a new thread for this installation issue?
Please provide as many information as possible, maybe including screen shots.
We don't see problems with the installers here, so they should be fine. I take it you verified the checksums of the downloaded files?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: FidanBoylu

OK, I started a new thread and was able to solve the issue on my own.

Do I need to start a new thread for the "failed to optimize" error that I get when solving large models. what are the possible causes? would changing any parameters help? which ones?

thanks
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Yes, please start a new thread for new questions (in particular for those that do not match the subject of the current thread).
The reasons for "failed to solve" can be many and it might be hard to figure out details in the Excel connector. One first useful step would be to export the model and solve it using the interactive optimizer. The interactive optimizer is much more verbose than the Excel connector and may provide better insight into why CPLEX fails to solve the model.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I am using Excel 2007 version 12.0.6112.5000 SP3 MSO (12.0.6607.1000) 32-bit and I am on a 64-bit machine, Windows 7 Professional, 8GB RAM. I have cplex_studio124.acad.win-x86-32 installed. Bye.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Beni1989

Hi everybody

I'm an economic student from Switzerland (with no programming background) using CPLEX for excel for the first time to solve a quite small modell.

But by running the optimization, "Failed to setup optimization problem. Error 6: CPLEX engine error" pop up every time.

I also tried to avoid any SUM or SUMPRODUCT functions but it does not work neither.

Does anyone have some other ideas to solve this problem?

thanks!

 

#CPLEXOptimizers
#DecisionOptimization

Are you able to export the model? Or does the error occur before CPLEX has a chance to export the model?

In order to export the model you have to specify an output file name on the third tab in the CPLEX dialog.

#CPLEXOptimizers
#DecisionOptimization