Originally posted by: joaoolavo

Dears,

I just started on cplex. I've a optimization problem to solve on my master thesis and I'd like to use cplex as one of the algorithms.

The issue is that the objective function involves a proprietary program, which is treated as a black-box.

On the other algorithms, I set a Bash script (or a Fortran function that then calls the Bash script, depending on the algorithm) to be the objective function, and there I call Python scripts and the proprietary software.

Is it possible to specify a system program, a script, a C++ function or something like this as the objective function? Is there any workaround so I can achieve this?

I'm using Linux and cplex 12.2.

I appreciate any advice!

Cheers.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

What exactly do you mean by "the objective function"? In CPLEX, you need to specify the objective function (either a linear or a convex quadratic function) in advance, such that CPLEX can then evaluate the objective function for any solution vector.

My guess is that in your case the objective function f(x) is implemented by calling an oracle (for example a simulator of some real-world process), which then gives you the value of f(x). This is something that you cannot do very well with CPLEX. The only possible way of doing this is to apply a branch-and-cut procedure, but this is only possible (without additional hassles) for convex objective functions. I can give you some more hints on how to do this, but first I like to know whether this is what you need.
Tobias
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: joaoolavo

Tobias, your guess is correct. f(x) is returned by a call to a oil reservoir simulator. There's no way to specify the objective function.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Okay. Now the big questions are:
(a) Does the simulator provide gradient information for the objective?
(b) Is the objective function convex?

If both answers are "yes", then you can do something with CPLEX, namely a branch-and-cut approach. You would introduce an artificial variable z that denotes the objective function and minimize (or maximize) z. I guess you have a trivial lower bound (in the minization case) for z, for example 0, so you can initially solve the problem with z >= 0. You need to disable dual reductions for this branch-and-cut approach, since you will separate lazy constraints.
Of course, the optimal solution to this LP relaxation will be z = 0, because z is not contained in any of the constraints. But now you will call your oracle to provide a value f(x*) and a gradient f'(x*) for your current LP solution x*. Then you add the cutting plane

z >= f(x*) + f'(x*)(x-x*)

Note that f'(x*) and (x-x*) are of course vectors, so the latter term is a scalar product between vectors.
Using this additional inequality, CPLEX will now continue the LP solve to get you another point x*. Again you would call the simulator and repeat.

But unfortunately, my guess is that the answer to question (a) is already "no". Then you are in the are of so-called "derivative free optimization", which is a completely different story. You cannot use CPLEX for this in a direct fashion. But there are some approaches for derivative free optimization out there, but I am certainly not an expert on this, and this would be really out of topic for this forum.
Tobias
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Following up on Tobias's answer, you might look at Response Surface Methodology. One possibility is to run the simulation (multiple times?) for each combination of parameter settings in some experimental design, fit a convex quadratic surface to the responses, optimize the convex function using CPLEX, determine new design points in the vicinity of the solution CPLEX gave, and iterate ad nauseum.

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization