Originally posted by: seg.fault

Hello,

 

I setup a profit-maximization problem in C++ where the objective function is defined as the difference between revenue and cost.
From the obtained solution, I need to to retrieve both the objective value and the values of the revenue and cost from which the objective value has been computed.

I'm facing some issues with CPLEX v12.8.0 probably because I didn't understand well how to use variables/expressions and the IloCplex::getValue method.
I report my problem with an example where the objective value, the revenue and the cost should be 1.038, 1.46 and 0.422, respectively..

- Tentative#1: in my original formulation, I defined 2 expressions, one for the revenue and one for the cost, by means of which I defined the objective function as the difference of the two.

The result is: objective value is OK, value of revenue expression is OK, value of cost expression is NOT OK. Specifically:

  * Code fragment:

        IloObjective obj;
        IloNumExpr revenue_expr(env);
        IloNumExpr cost_expr(env);
        //...
        obj = IloMaximize(env, revenue_expr-cost_expr);
        mode.add(obj);
        //...
        cplex.solve();
        //...
        std::cout << "Obj value: " << cplex.getObjValue() << " (revenue-expr: " << cplex.getValue(revenue_expr) << ", cost-expr: " << cplex.getValue(cost_expr) << ")" << std::endl;

  *  Output:

        Obj value: 1.038 (revenue-expr: 1.46, cost-expr: 1.022)

 

- Tentative#2: to solve the above issue, I added two variables (one for the revenue and the other one for the cost) and constrained them to be equal to the revenue and cost expressions defined above, respectively, and then I used these variables to define the objective function in a similar way I have done in Tentative#1.

The result is: objective value is NOT OK, value of revenue expression is OK, value of cost expression is NOT OK. Specifically:
  *  Code fragment:

        IloObjective obj;
        IloNumVar revenue_var(env, "revenue");
        model.add(revenue_var);
        IloNumVar cost_var(env, "cost");
        model.add(cost_var);
        IloNumExpr revenue_expr(env);
        //...
        model.add(IloConstraint(env, revenue_var == revenue_expr));
        IloNumExpr cost_expr(env);
        //...
        model.add(IloConstraint(env, cost_var == cost_expr));
        obj = IloMaximize(env, revenue_var-cost_var);
        mode.add(obj);
        //...
        cplex.solve();
        //...
        std::cout << "Obj value: " << cplex.getObjValue() << " (revenue-var: " << cplex.getValue(revenue_var) << ", revenue-expr: " << cplex.getValue(revenue_expr) << ", cost-var: " << cplex.getValue(cost_var) << ", cost-expr: " << cplex.getValue(cost_expr) << ")" << std::endl;

  *  Output:

        Obj value: 1.46 (revenue-var: 1.46, revenue-expr: 1.46, cost-var: 0, cost-expr: 0.422)

 

- Tentative#3: starting from Tentative#2, I was finally able to get a working solution by simply defining the objective function by means of the expressions instead of the variables.

The result is: objective value, value of revenue expression, and value of cost expression are all OK. Specifically:
  *  Code fragment:

        IloObjective obj;
        IloNumVar revenue_var(env, "revenue");
        model.add(revenue_var);
        IloNumExpr revenue_expr(env);
        //...
        model.add(IloConstraint(env, revenue_var == revenue_expr));
        IloNumVar cost_var(env, "cost");
        model.add(cost_var);
        IloNumExpr cost_expr(env);
        //...
        model.add(IloConstraint(env, cost_var == cost_expr));
        obj = IloMaximize(env, revenue_expr-cost_expr);
        mode.add(obj);
        //...
        cplex.solve();
        //...
        std::cout << "Obj value: " << cplex.getObjValue() << " (revenue-var: " << cplex.getValue(revenue_var) << ", revenue-expr: " << cplex.getValue(revenue_expr) << ", cost-var: " << cplex.getValue(cost_var) << ", cost-expr: " << cplex.getValue(cost_expr) << ")" << std::endl;

  *  Output:

        Obj value: 1.038 (revenue-var: 1.46, revenue-expr: 1.46, cost-var: 0, cost-expr: 0.422)

 

Can anyone be so kind to tell what's happening here?

Thank you very much for your consideration and time.

Best,

Sergio.

#CPLEXOptimizers
#DecisionOptimization

This indeed sounds all pretty weird. I don't see an obvious error in your codes. So my best guess is that there is some problem in the code you did not show. Can you show more or the full code?

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: seg.fault

Thanks Daniel.

Unfortunately, Tentative#3 has similar issues too. I've tried another instance of the problem where the right values for the objective value, the revenue, and the cost are 0.0515, 0.07 and 0.0185, respectively. What I get is a wrong value of the cost expression (while, unlike before, the value of the cost variable is correct), as follows:

    Obj value: 0.0515 (revenue-var: 0.07, revenue-expr: 0.07, cost-var: 0.0185, cost-expr: 0.0385)

Regarding the full code, I report here below the code for Tentative#3 (the code for Tentative#1 or Tentative#2 can be obtained from this code in a straightforward way). Despite I tried to simplify it, it is still very long. I suggest to start reading from lines going from 189 to 320, where I define the revenue and the cost variables/expressions, and the objective function.

            IloEnv env;

            IloModel model(env);

            model.setName("Max-Profit Optimization");

            // Decision Variables

            IloArray<IloBoolVarArray> x_var(env, nt);
            for (std::size_t t = 0; t < nt; ++t)
            {
                x_var[t] = IloBoolVarArray(env, nmachs);
                for (std::size_t i = 0; i < nmachs; ++i)
                {
                    std::ostringstream oss;
                    oss << "x[" << t << "][" << i << "]";
                    x_var[t][i] = IloBoolVar(env, oss.str().c_str());
                    model.add(x_var[t][i]);
                }
            }

            IloArray<IloArray<IloArray<IloIntVarArray>>> y_var(env, nt);
            for (std::size_t t = 0; t < nt; ++t)
            {
                y_var[t] = IloArray<IloArray<IloIntVarArray>>(env, nmachs);

                for (std::size_t i = 0; i < nmachs; ++i)
                {
                    y_var[t][i] = IloArray<IloIntVarArray>(env, njobs);

                    for (std::size_t j = 0 ; j < njobs ; ++j)
                    {
                        y_var[t][i][j] = IloIntVarArray(env, nagentcats);

                        for (std::size_t k = 0 ; k < nagentcats ; ++k)
                        {
                            std::ostringstream oss;
                            oss << "y[" << t << "][" << i << "][" << j << "][" << k << "]";
                            y_var[t][i][j][k] = IloIntVar(env, oss.str().c_str());
                            model.add(y_var[t][i][j][k]);
                        }
                    }
                }
            }

            IloArray<IloBoolVarArray> s_var(env, nt);
            for (std::size_t t = 0; t < nt; ++t)
            {
                s_var[t] = IloBoolVarArray(env, njobs);
                for (std::size_t j = 0; j < njobs; ++j)
                {
                    std::ostringstream oss;
                    oss << "s[" << t << "][" << j << "]";
                    s_var[t][j] = IloBoolVar(env, oss.str().c_str());
                    model.add(s_var[t][j]);
                }
            }

            IloArray<IloNumVarArray> u_var(env, nt);
            for (std::size_t t = 0; t < nt; ++t)
            {
                u_var[t] = IloNumVarArray(env, nmachs);
                for (std::size_t i = 0; i < nmachs; ++i)
                {
                    std::ostringstream oss;
                    oss << "u[" << t << "][" << i << "]";
                    u_var[t][i] = IloNumVar(env, oss.str().c_str());
                    model.add(u_var[t][i]);
                }
            }

            // Constraints

            std::size_t cc = 0; // Constraint counter

            ++cc;
            for (std::size_t t = 0; t < nt; ++t)
            {
                for (std::size_t j = 0; j < njobs; ++j)
                {
                    IloIntExpr rhs_expr(env);
                    for (std::size_t i = 0; i < nmachs; ++i)
                    {
                        rhs_expr += IloSum(y_var[t][i][j]);
                    }

                    std::ostringstream oss;
                    oss << "C" << cc << "_{" << t << "," << j << "}";

                    IloConstraint cons(s_var[t][j] == (rhs_expr > 0));
                    cons.setName(oss.str().c_str());
                    model.add(cons);
                }
            }

            for (std::size_t t = 0; t < nt; ++t)
            {
                for (std::size_t i = 0; i < nmachs; ++i)
                {
                    auto const mach_cat = mach_categories[i];

                    IloNumExpr rhs_expr(env);
                    for (std::size_t j = 0; j < njobs; ++j)
                    {
                        for (std::size_t k = 0; k < nagentcats; ++k)
                        {
                            rhs_expr += y_var[t][i][j][k]*agent_req[k][mach_cat];
                        }
                    }

                    std::ostringstream oss;
                    oss << "C" << cc << "_{" << t << "," << i << "}";

                    IloConstraint cons(u_var[t][i] == rhs_expr);
                    cons.setName(oss.str().c_str());
                    model.add(cons);
                }
            }

            ++cc;
            for (std::size_t t = 0; t < nt; ++t)
            {
                for (std::size_t j = 0; j < njobs; ++j)
                {
                    IloIntExpr lhs_expr(env);
                    for (std::size_t i = 0; i < nmachs; ++i)
                    {
                        lhs_expr += IloSum(y_var[t][i][j]);
                    }

                    IloIntExprArray max_op_expr(env, nagentcats);
                    for (std::size_t k = 0; k < nagentcats; ++k)
                    {
                        max_op_expr[k] = IloIntExpr(env);
                        for (std::size_t i = 0; i < nmachs; ++i)
                        {
                            max_op_expr[k] += y_var[t][i][j][k];
                        }
                    }

                    std::ostringstream oss;
                    oss << "C" << cc << "_{" << t << "," << j << "}";

                    IloConstraint cons(lhs_expr == IloMax(max_op_expr));
                    cons.setName(oss.str().c_str());
                    model.add(cons);
                }
            }

            ++cc;
            for (std::size_t t = 0; t < nt; ++t)
            {
                for (std::size_t i = 0; i < nmachs; ++i)
                {
                    std::ostringstream oss;
                    oss << "C" << cc << "_{" << t << "," << i << "}";

                    IloConstraint cons(u_var[t][i] <= x_var[t][i]);
                    cons.setName(oss.str().c_str());
                    model.add(cons);
                }
            }

            ++cc;
            for (std::size_t t = 0; t < nt; ++t)
            {
                for (std::size_t j = 0; j < njobs; ++j)
                {
                    auto const job_cat = job_categories[j];

                    for (std::size_t k = 0; k < nagentcats; ++k)
                    {
                        IloIntExpr lhs_expr(env);
                        for (std::size_t i = 0; i < nmachs; ++i)
                        {
                            lhs_expr += y_var[t][i][j][k];
                        }

                        std::ostringstream oss;
                        oss << "C" << cc << "_{" << t << "," << j << "," << k << "}";

                        IloConstraint cons(lhs_expr <= IloInt(min_num_agents[t][job_cat][k]));
                        cons.setName(oss.str().c_str());
                        model.add(cons);
                    }
                }
            }

            // Set objective
            IloObjective obj;

            // Revenues
            IloNumVar revenue_var(env, "revenue");
            model.add(revenue_var);
            IloNumExpr revenue_expr(env);
            {
                for (std::size_t t = 0; t < nt; ++t)
                {
                    for (std::size_t i = 0; i < nmachs; ++i)
                    {
                        for (std::size_t j = 0; j < njobs; ++j)
                        {
                            auto const job_cat = job_categories[j];

                            for (std::size_t k = 0; k < nagentcats; ++k)
                            {
                                revenue_expr += job_revenues[job_cat]*y_var[t][i][j][k];
                            }
                        }
                    }
                }

                ++cc;

                IloConstraint cons(revenue_var == revenue_expr);

                std::ostringstream oss;
                oss << "C" << cc;

                cons.setName(oss.str().c_str());
                model.add(cons);
            }

            // Costs
            IloNumVar cost_var(env, "cost");
            model.add(cost_var);
            IloNumExpr cost_expr(env);
            {
                // - Add elecricity costs
                for (std::size_t t = 0; t < nt; ++t)
                {
                    for (std::size_t i = 0; i < nmachs; ++i)
                    {
                        auto const mach_cat = mach_categories[i];
                        auto const mach_power_state = mach_power_states[i];

                        cost_expr += (x_var[t][i]*mach_static_power[mach_cat]+mach_dyn_power[mach_cat]*u_var[t][i])*electricity_price;

                        if (t > 0)
                        {
                            cost_expr += x_var[t][i]*(1-x_var[t-1][i])*mach_poweron_costs[mach_cat]
                                      +  (1-x_var[t][i])*x_var[t-1][i]*mach_poweroff_costs[mach_cat];
                        }
                        else
                        {
                            cost_expr += x_var[0][i]*IloInt(1-mach_power_state)*mach_poweron_costs[mach_cat]
                                      +  (1-x_var[0][i])*IloInt(mach_power_state)*mach_poweroff_costs[mach_cat];
                        }
                    }
                    for (std::size_t i = 0; i < nmachs; ++i)
                    {
                        for (std::size_t j = 0; j < njobs; ++j)
                        {
                            for (std::size_t k = 0; k < nagentcats; ++k)
                            {
                                if (t == 0)
                                {
                                    if (mach_agent_allocations[i].count(j) > 0 && mach_agent_allocations[i].at(j).first == k)
                                    {
                                        cost_expr += IloMax(0, y_var[0][i][j][k] - mach_agent_allocations[i].at(j).second)*agent_alloc_costs[k];
                                    }
                                    else
                                    {
                                        cost_expr += y_var[0][i][j][k]*agent_alloc_costs[k];
                                    }
                                }
                                else
                                {
                                    cost_expr += IloMax(0, y_var[t][i][j][k] - y_var[t-1][i][j][k])*agent_alloc_costs[k];
                                }
                            }
                        }
                    }
                    for (std::size_t j = 0; j < njobs; ++j)
                    {
                        auto const job_cat = job_categories[j];

                        IloIntExpr left_expr(env);

                        for (std::size_t k = 0; k < nagentcats; ++k)
                        {
                            IloIntExpr ysum_expr(env);

                            for (std::size_t i = 0; i < nmachs; ++i)
                            {
                                ysum_expr += y_var[t][i][j][k];
                            }

                            left_expr += ((ysum_expr == 0) + (ysum_expr == min_num_agents[t][job_cat][k])) == 0;
                        }


                        cost_expr += ((s_var[t][j] == 0) + left_expr)*job_penalties[job_cat];
                    }
                }

                ++cc;

                IloConstraint cons(cost_var == cost_expr);

                std::ostringstream oss;
                oss << "C" << cc;

                cons.setName(oss.str().c_str());
                model.add(cons);
            }

            obj = IloMaximize(env, revenue_expr-cost_expr);
            model.add(obj);

            IloCplex cplex(model);

            bool solved = cplex.solve();

            if (!solved || cplex.getStatus() != IloAlgorithm::Optimal)
            {
                throw std::runtime_error("Unable to find an optimal solution");
            }

            std::cout << "Obj value: " << cplex.getObjValue() << " (revenue-var: " << cplex.getValue(revenue_var) << ", revenue-expr: " << cplex.getValue(revenue_expr) << ", cost-var: " << cplex.getValue(cost_var) << ", cost-expr: " << cplex.getValue(cost_expr) << ")" << std::endl;

Thank you so much for your help.

Best,

Sergio

#CPLEXOptimizers
#DecisionOptimization

I looked hard at your code and could still not spot a mistake.

Can you provide a full working example? It you don't want to post it here you can post to daniel(dot)junglas(at)de(dot)ibm(dot)com.

The only thing that caught my eye was that you are using IloMax in cost_expr. That is not a linear thing but it should still work the way you expect it to.

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: seg.fault

Hello Daniel,

Sorry for the delay.

I've isolated the CPLEX code from the rest of my software project and I've created some test cases. I've also added some auxiliary functions for doing consistency checks.

 

In the attached file you find:

• test_solve.cpp: contains the definition of the "main" function as well as of other functions used for testing one of the above Tentative#X solutions. You can control which Tentative#X to run by means of the SOLVE_FUNC macro (see at the beginning of test_solve.cpp)
• solve.{hpp,cpp}: contain the definition os the above Tentative#X solutions as well as of other auxiliary functions (e.g., functions used for consistency checks)
• Makefile: the GNU makefile; please set the variable cplex_home to a value that is suitable for your environment

Compile it with: make clean all

Run it as: ./test_solve >test.log

In the printed output, the label "[MY:" is used to distinguish my message from those printed by CPLEX.

These message can help you understanding what's going on. For instance,

[MY:WARNING] Anomaly in y[5][2][0][0] -- value: 3 != cast-value: 2

denotes a problem found while checking if the value of an integer or a boolean variable as returned by IloCplex::getValue (i.e., as an IloNum value) and the one obtained by casting it to a suitable type (e.g., IloBool or IloInt, respectively) are consistent. In the above case, they are not.

My runtime system is: Fedora Linux 27 x64_86 + GCC 7.3.1 + CPLEX 12.8.0

If you need any additional information, please don't hesitate to contact me.

I really thank you for your help and time.

Best,

Sergio

#CPLEXOptimizers
#DecisionOptimization

Thanks a lot for your code. I can reproduce the problems here.

The "cast" anomalies are a bug in your code. You are doing something like this

if ( cplex.getValue(x) != static_cast<IloBool>(cplex.getValues()) { /* report anomaly */ }

This assumes that values returned for binary or integer variables are perfectly integer. This is however not true. CPLEX could return something like 1e-10 instead of an exact 0. That is what you get from finite precision arithmetic. Moreover, just casting a value like 0.999999999 to int or bool will result in truncation! The result will be 0, not 1! In order to properly convert values you should use functions like lrint() or round(). And to check whether a value is integer you should check whether it is integer within some tolerance. You may want to take a look at CPX_PARAM_EPINT.

I am checking the problems with the objective values now ...

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: seg.fault

Hi Daniel,

Thank you for your reply.

By following your guidelines. I've changed the way the values returned by IloCplex::getValue are converted:

• IloNum -> T conversion:
   (e.g., T is IloInt, IloBool, ...):static_cast<T>(std::round(cplex.getValue(var))
  

• Check for integrality:
   float_eq(cplex.getValue(int_var), v, cplex.getParam(IIloCplex::Param::MIP::Tolerances::Integrality)
  where float_eq is a function to perform floating-point comparison
  

Now things have improved a lot. The integrality check still generates some few false positive probably due to the method used for floating-point comparison (in may case I use a method similar to GNU gsl_fcmp).

For instance, with a new test case I get:

[MY:WARNING] Anomaly in x[2][3] -- value: 1e-10 != rounded-value: 0 (tol: 1e-05)
[MY:WARNING] Anomaly in x[2][7] -- value: 2.33333e-10 != rounded-value: 0 (tol: 1e-05)
[MY:WARNING] Anomaly in y[1][1][1][0] -- value: -1e-09 != rounded-value: 0 (tol: 1e-05)
[MY:WARNING] Anomaly in y[1][3][1][0] -- value: 1e-09 != rounded-value: 0 (tol: 1e-05)

 

Regarding the objective value, I still get unexpected values for its composing expressions (note, objective value == revenue - cost), for instance:

CPLEX -- Obj value: 1.138 (revenue-expr: 1.46, cost-expr: 0.922)

Here above, the cost expression has a wrong value; the right one should be 0.322.

I can't figure out where I am wrong. Any idea?

 

Thank you very much.

 

Sergio

 

 

#CPLEXOptimizers
#DecisionOptimization

Sorry for taking so long to reply. I spent a lot of time looking into this and I am still not exactly sure what the problem is. But I have a strong hunch: In your cost_expr you have terms (and even combinations of such terms) like

left_expr += ((ysum_expr == 0) + (ysum_expr == min_num_agents[t][job_cat][][k])) == 0;

and

cost_expr += ((s_var[t][j] == 0) + left_expr) * job_penalties[job_cat];

I suspect the use of '==' is an issue here. Since the left side of the operator is subject to numerical round-off error (even if the type of the variables involved is binary or integer!) this sort of comparison is shaky at best. A tiny difference will decide whether the term is 0 and 1, thus significantly changing its contribution to the objective value. Since s_var[t][j] is a binary variable, you can replace 's_var[t][j]==0' by '1-s_var[t][j]'. That should be a lot more robust. For ysum_expr things are not that simple but you can still exploit the fact that all y are integer. With that and with the help of IloIfThen you should be able to state the same constraint without using operator == directly in the constraint (it should be ok to use that operator in the constraint to build an IloIfThen).

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: seg.fault

Hi Daniel,

Sorry for this very late reply.

Thank you for your suggestions. While I didn't managed to successfully use 'IloIfThen', by replacing '==' expression with the binary variable trick '(1-s_var[t][i])' or with something equivalent (by means of the problem knowledge) the above issues look disappeared.

Specifically, I changed the solve_tentative1 function (see solve.cpp) as follows:

1. Expression at line 323:

left_expr += ((ysum_expr == 0) + (ysum_expr == min_num_agents[t][job_cat][k])) == 0;

has been replaced with:

left_expr += (ysum_expr > 0) * (ysum_expr < min_num_agents[t][job_cat][k]);

2. Expression at line 326:

cost_expr += ((s_var[t][j] == 0) + left_expr)*job_penalties[job_cat];

has been replaced with:

cost_expr += ((1-s_var[t][j]) + left_expr)*job_penalties[job_cat];

 

Do you think the above replacing expressions are better (for CPLEX) than the original ones?

 

Also, should I change constraints that use '==' as well?

 

For instance, at line 125 there is:

IloConstraint cons(s_var[t][j] == (rhs_expr > 0));

and at line 149 there is:

IloConstraint cons(u_var[t][i] == rhs_expr);

and at line 179 there is:

IloConstraint cons(lhs_expr == IloMax(max_op_expr));

Unfortunately, I can't figure out how to change them.

 

Thank you so much for your time and help.

 

Sergio

#CPLEXOptimizers
#DecisionOptimization