Hi experts,

We have a relatively large LP problem, and when we use barrier to solve it, sometimes the solver returns the NumBest status with or without a solution. 

My questions are:

1. When there is a solution returned, what is the difference between this solution vs. the optimal status solution? 
2. Is the NumBest status solution feasible? Can we use it as a practical solution (maybe not optimal)? I know we can further do crossover, but it usally takes very long time, so would like to understand the tradeoff without doing crossover.
3. When we do crossover, we want just a feasible solution and not need an optimal solution. We tried to set Simplex Limits upperObj to a very small number (-1.7976931348623157e+308), and lowerObj to a very big numer (1.7976931348623157e+308). Although it seems to work, the solving status is always "Error", and the cplex status is "AbortObjLim", even if there is a solution.  It looks strange. So my questions are, is our setting wrong? Is there better way to achieve this (stop crossover when there is a feasible solution)? 
   
   

Thanks a lot.

1. The solution status NumBest means that there have been some numerical difficulties and the optimality cannot be proven. The best solution found so far is available. 
https://www.ibm.com/docs/en/cofz/12.10.0?topic=api-cpx-stat-num-best

2. There is a usable solution but it is not proven optimal. It is not a basic solution and you can and up with many variables having fractional values instead of being at their bounds. 

3. Unfortunately, there is no way to stop crossover when there is a solution only. 

About numerical dificulties that can happen in Barrier optimization, a few hints are given here:
https://www.ibm.com/docs/en/icos/22.1.1?topic=difficulties-during-optimization

There is also a more detailed approach for solving a model with Barrier that can be helpful here:
https://www.gams.com/blog/2022/04/how-to-tune-cplex-options-for-times-models/

Regards

Philippe

Hi experts,

We have a relatively large LP problem, and when we use barrier to solve it, sometimes the solver returns the NumBest status with or without a solution. 

My questions are:

1. When there is a solution returned, what is the difference between this solution vs. the optimal status solution? 
2. Is the NumBest status solution feasible? Can we use it as a practical solution (maybe not optimal)? I know we can further do crossover, but it usally takes very long time, so would like to understand the tradeoff without doing crossover.
3. When we do crossover, we want just a feasible solution and not need an optimal solution. We tried to set Simplex Limits upperObj to a very small number (-1.7976931348623157e+308), and lowerObj to a very big numer (1.7976931348623157e+308). Although it seems to work, the solving status is always "Error", and the cplex status is "AbortObjLim", even if there is a solution.  It looks strange. So my questions are, is our setting wrong? Is there better way to achieve this (stop crossover when there is a feasible solution)? 
   
   

Thanks a lot.

Hi Philippe, 

Thanks a lot for your advice. May I further clarify my understanding with you? Really thanks.

1. As I know, the barrier algo stops when it finds feasible primal and dual solutions that are complementary, and at this time, the solution is also optimal. When there are numerical difficulties, the algo cannot find an optimal solution, meaning reaching the feasible primal and dual solution, then does that mean there will not be any feasible solution?  But why the solver can return a solution sometimes? Can we consider this solution as a strict feasible solution(just not optimal)?
2. At NumBest status, the solver may return the best solution found so far. May I know how the "best" is measured?
3. Another question, related to the barrier algo, is when a barrier solve is aborted with time limit, there is no solution yet, and then when we try to use cplex.GetValue(objectiveExpression) to get the objective expression's value, seems it returns the last objective value before abort. But at the last log row, we can see the Prim Inf is still very high(4.96e+7). So can we consider the objective value (-6.9e+07) as a possible objective value of a feasible solution? Or this value cannot be used for any purpose actually? Thanks a lot.

------------------------------
Sophia Niu

Original Message:
Sent: Mon October 14, 2024 11:34 AM
From: PhR
Subject: Barrier's solution with NumBest status

1. The solution status NumBest means that there have been some numerical difficulties and the optimality cannot be proven. The best solution found so far is available. 
https://www.ibm.com/docs/en/cofz/12.10.0?topic=api-cpx-stat-num-best

2. There is a usable solution but it is not proven optimal. It is not a basic solution and you can and up with many variables having fractional values instead of being at their bounds. 

3. Unfortunately, there is no way to stop crossover when there is a solution only. 

About numerical dificulties that can happen in Barrier optimization, a few hints are given here:
https://www.ibm.com/docs/en/icos/22.1.1?topic=difficulties-during-optimization

There is also a more detailed approach for solving a model with Barrier that can be helpful here:
https://www.gams.com/blog/2022/04/how-to-tune-cplex-options-for-times-models/

Regards

Philippe

Original Message:
Sent: Mon October 07, 2024 05:03 PM
From: Sophia Niu
Subject: Barrier's solution with NumBest status

Hi experts,

We have a relatively large LP problem, and when we use barrier to solve it, sometimes the solver returns the NumBest status with or without a solution. 

My questions are:

1. When there is a solution returned, what is the difference between this solution vs. the optimal status solution? 
2. Is the NumBest status solution feasible? Can we use it as a practical solution (maybe not optimal)? I know we can further do crossover, but it usally takes very long time, so would like to understand the tradeoff without doing crossover.
3. When we do crossover, we want just a feasible solution and not need an optimal solution. We tried to set Simplex Limits upperObj to a very small number (-1.7976931348623157e+308), and lowerObj to a very big numer (1.7976931348623157e+308). Although it seems to work, the solving status is always "Error", and the cplex status is "AbortObjLim", even if there is a solution.  It looks strange. So my questions are, is our setting wrong? Is there better way to achieve this (stop crossover when there is a feasible solution)? 
   
   

Thanks a lot.

------------------------------
Sophia Niu
------------------------------

Hi Philippe, 

Thanks a lot for your advice. May I further clarify my understanding with you? Really thanks.

1. As I know, the barrier algo stops when it finds feasible primal and dual solutions that are complementary, and at this time, the solution is also optimal. When there are numerical difficulties, the algo cannot find an optimal solution, meaning reaching the feasible primal and dual solution, then does that mean there will not be any feasible solution?  But why the solver can return a solution sometimes? Can we consider this solution as a strict feasible solution(just not optimal)?
2. At NumBest status, the solver may return the best solution found so far. May I know how the "best" is measured?
3. Another question, related to the barrier algo, is when a barrier solve is aborted with time limit, there is no solution yet, and then when we try to use cplex.GetValue(objectiveExpression) to get the objective expression's value, seems it returns the last objective value before abort. But at the last log row, we can see the Prim Inf is still very high(4.96e+7). So can we consider the objective value (-6.9e+07) as a possible objective value of a feasible solution? Or this value cannot be used for any purpose actually? Thanks a lot.

------------------------------
Sophia Niu

Original Message:
Sent: Mon October 14, 2024 11:34 AM
From: PhR
Subject: Barrier's solution with NumBest status

1. The solution status NumBest means that there have been some numerical difficulties and the optimality cannot be proven. The best solution found so far is available. 
https://www.ibm.com/docs/en/cofz/12.10.0?topic=api-cpx-stat-num-best

2. There is a usable solution but it is not proven optimal. It is not a basic solution and you can and up with many variables having fractional values instead of being at their bounds. 

3. Unfortunately, there is no way to stop crossover when there is a solution only. 

About numerical dificulties that can happen in Barrier optimization, a few hints are given here:
https://www.ibm.com/docs/en/icos/22.1.1?topic=difficulties-during-optimization

There is also a more detailed approach for solving a model with Barrier that can be helpful here:
https://www.gams.com/blog/2022/04/how-to-tune-cplex-options-for-times-models/

Regards

Philippe

Original Message:
Sent: Mon October 07, 2024 05:03 PM
From: Sophia Niu
Subject: Barrier's solution with NumBest status

Hi experts,

We have a relatively large LP problem, and when we use barrier to solve it, sometimes the solver returns the NumBest status with or without a solution. 

My questions are:

1. When there is a solution returned, what is the difference between this solution vs. the optimal status solution? 
2. Is the NumBest status solution feasible? Can we use it as a practical solution (maybe not optimal)? I know we can further do crossover, but it usally takes very long time, so would like to understand the tradeoff without doing crossover.
3. When we do crossover, we want just a feasible solution and not need an optimal solution. We tried to set Simplex Limits upperObj to a very small number (-1.7976931348623157e+308), and lowerObj to a very big numer (1.7976931348623157e+308). Although it seems to work, the solving status is always "Error", and the cplex status is "AbortObjLim", even if there is a solution.  It looks strange. So my questions are, is our setting wrong? Is there better way to achieve this (stop crossover when there is a feasible solution)? 
   
   

Thanks a lot.

------------------------------
Sophia Niu
------------------------------

Hi Sophia,

The Barrier optimizer, as well as primal and dual simplex, gives access to some solution information even when the optimization is stopped prematurely, because of a limit or numerical difficulties or when an infeasibility is detected.

As explained partly here https://www.ibm.com/docs/en/icos/22.1.1?topic=infeasibility-interpreting-solution-quality
the values you can have access to are related to the optimization stage the engine stopped in. 

A way to check that you have a feasible solution available in the case of premature ending is to check that a call to the method isPrimalFeasible() returns True. In this case, the values you can get correspond to a solution to the LP. 

In the case of the "NumBest" status returned, there are primal infeasibilities close to zero and a solution is available but without any optimality proof made. In the course of optimization, the Barrier method can find several feasible solutions (with primal infeasibilities low enough) that are not optimal. The solution given is the best among those. 

Original Message:
Sent: Wed October 16, 2024 02:00 AM
From: Sophia Niu
Subject: Barrier's solution with NumBest status

Hi Philippe, 

Thanks a lot for your advice. May I further clarify my understanding with you? Really thanks.

1. As I know, the barrier algo stops when it finds feasible primal and dual solutions that are complementary, and at this time, the solution is also optimal. When there are numerical difficulties, the algo cannot find an optimal solution, meaning reaching the feasible primal and dual solution, then does that mean there will not be any feasible solution?  But why the solver can return a solution sometimes? Can we consider this solution as a strict feasible solution(just not optimal)?
2. At NumBest status, the solver may return the best solution found so far. May I know how the "best" is measured?
3. Another question, related to the barrier algo, is when a barrier solve is aborted with time limit, there is no solution yet, and then when we try to use cplex.GetValue(objectiveExpression) to get the objective expression's value, seems it returns the last objective value before abort. But at the last log row, we can see the Prim Inf is still very high(4.96e+7). So can we consider the objective value (-6.9e+07) as a possible objective value of a feasible solution? Or this value cannot be used for any purpose actually? Thanks a lot.

------------------------------
Sophia Niu

Original Message:
Sent: Mon October 14, 2024 11:34 AM
From: PhR
Subject: Barrier's solution with NumBest status

1. The solution status NumBest means that there have been some numerical difficulties and the optimality cannot be proven. The best solution found so far is available. 
https://www.ibm.com/docs/en/cofz/12.10.0?topic=api-cpx-stat-num-best

2. There is a usable solution but it is not proven optimal. It is not a basic solution and you can and up with many variables having fractional values instead of being at their bounds. 

3. Unfortunately, there is no way to stop crossover when there is a solution only. 

About numerical dificulties that can happen in Barrier optimization, a few hints are given here:
https://www.ibm.com/docs/en/icos/22.1.1?topic=difficulties-during-optimization

There is also a more detailed approach for solving a model with Barrier that can be helpful here:
https://www.gams.com/blog/2022/04/how-to-tune-cplex-options-for-times-models/

Regards

Philippe

Original Message:
Sent: Mon October 07, 2024 05:03 PM
From: Sophia Niu
Subject: Barrier's solution with NumBest status

Hi experts,

We have a relatively large LP problem, and when we use barrier to solve it, sometimes the solver returns the NumBest status with or without a solution. 

My questions are:

1. When there is a solution returned, what is the difference between this solution vs. the optimal status solution? 
2. Is the NumBest status solution feasible? Can we use it as a practical solution (maybe not optimal)? I know we can further do crossover, but it usally takes very long time, so would like to understand the tradeoff without doing crossover.
3. When we do crossover, we want just a feasible solution and not need an optimal solution. We tried to set Simplex Limits upperObj to a very small number (-1.7976931348623157e+308), and lowerObj to a very big numer (1.7976931348623157e+308). Although it seems to work, the solving status is always "Error", and the cplex status is "AbortObjLim", even if there is a solution.  It looks strange. So my questions are, is our setting wrong? Is there better way to achieve this (stop crossover when there is a feasible solution)? 
   
   

Thanks a lot.

------------------------------
Sophia Niu
------------------------------

Hi Sophia,

The Barrier optimizer, as well as primal and dual simplex, gives access to some solution information even when the optimization is stopped prematurely, because of a limit or numerical difficulties or when an infeasibility is detected.

As explained partly here https://www.ibm.com/docs/en/icos/22.1.1?topic=infeasibility-interpreting-solution-quality
the values you can have access to are related to the optimization stage the engine stopped in. 

A way to check that you have a feasible solution available in the case of premature ending is to check that a call to the method isPrimalFeasible() returns True. In this case, the values you can get correspond to a solution to the LP. 

In the case of the "NumBest" status returned, there are primal infeasibilities close to zero and a solution is available but without any optimality proof made. In the course of optimization, the Barrier method can find several feasible solutions (with primal infeasibilities low enough) that are not optimal. The solution given is the best among those. 

Original Message:
Sent: Wed October 16, 2024 02:00 AM
From: Sophia Niu
Subject: Barrier's solution with NumBest status

Hi Philippe, 

Thanks a lot for your advice. May I further clarify my understanding with you? Really thanks.

1. As I know, the barrier algo stops when it finds feasible primal and dual solutions that are complementary, and at this time, the solution is also optimal. When there are numerical difficulties, the algo cannot find an optimal solution, meaning reaching the feasible primal and dual solution, then does that mean there will not be any feasible solution?  But why the solver can return a solution sometimes? Can we consider this solution as a strict feasible solution(just not optimal)?
2. At NumBest status, the solver may return the best solution found so far. May I know how the "best" is measured?
3. Another question, related to the barrier algo, is when a barrier solve is aborted with time limit, there is no solution yet, and then when we try to use cplex.GetValue(objectiveExpression) to get the objective expression's value, seems it returns the last objective value before abort. But at the last log row, we can see the Prim Inf is still very high(4.96e+7). So can we consider the objective value (-6.9e+07) as a possible objective value of a feasible solution? Or this value cannot be used for any purpose actually? Thanks a lot.

------------------------------
Sophia Niu

Original Message:
Sent: Mon October 14, 2024 11:34 AM
From: PhR
Subject: Barrier's solution with NumBest status

1. The solution status NumBest means that there have been some numerical difficulties and the optimality cannot be proven. The best solution found so far is available. 
https://www.ibm.com/docs/en/cofz/12.10.0?topic=api-cpx-stat-num-best

2. There is a usable solution but it is not proven optimal. It is not a basic solution and you can and up with many variables having fractional values instead of being at their bounds. 

3. Unfortunately, there is no way to stop crossover when there is a solution only. 

About numerical dificulties that can happen in Barrier optimization, a few hints are given here:
https://www.ibm.com/docs/en/icos/22.1.1?topic=difficulties-during-optimization

There is also a more detailed approach for solving a model with Barrier that can be helpful here:
https://www.gams.com/blog/2022/04/how-to-tune-cplex-options-for-times-models/

Regards

Philippe

Original Message:
Sent: Mon October 07, 2024 05:03 PM
From: Sophia Niu
Subject: Barrier's solution with NumBest status

Hi experts,

We have a relatively large LP problem, and when we use barrier to solve it, sometimes the solver returns the NumBest status with or without a solution. 

My questions are:

1. When there is a solution returned, what is the difference between this solution vs. the optimal status solution? 
2. Is the NumBest status solution feasible? Can we use it as a practical solution (maybe not optimal)? I know we can further do crossover, but it usally takes very long time, so would like to understand the tradeoff without doing crossover.
3. When we do crossover, we want just a feasible solution and not need an optimal solution. We tried to set Simplex Limits upperObj to a very small number (-1.7976931348623157e+308), and lowerObj to a very big numer (1.7976931348623157e+308). Although it seems to work, the solving status is always "Error", and the cplex status is "AbortObjLim", even if there is a solution.  It looks strange. So my questions are, is our setting wrong? Is there better way to achieve this (stop crossover when there is a feasible solution)? 
   
   

Thanks a lot.

------------------------------
Sophia Niu
------------------------------

For Barrier feasibility tolerance you can use the relative complementarity tolerance:
https://www.ibm.com/docs/en/icos/22.1.1?topic=parameters-convergence-tolerance-lp-qp-problems 
That is briefly explained here:
https://www.ibm.com/docs/en/icos/22.1.1?topic=optimizer-introducing-barrier

Unfortunately, there is no equivalent of isPrimalFeasible in interactive. Interactive only covers the essential interface of CPLEX.

There is no feature for multiple solutions in LP. To gather feasible solutions reached during the course of optimization, you can use a continuous callback (available in some langages), however this an advanced usage and can have some impact on performance. For C++ interface you can have a look here: https://www.ibm.com/docs/en/icos/22.1.0?topic=classes-ilocplexcontinuouscallbacki 

Regards

Hi Sophia,

The Barrier optimizer, as well as primal and dual simplex, gives access to some solution information even when the optimization is stopped prematurely, because of a limit or numerical difficulties or when an infeasibility is detected.

As explained partly here https://www.ibm.com/docs/en/icos/22.1.1?topic=infeasibility-interpreting-solution-quality
the values you can have access to are related to the optimization stage the engine stopped in. 

A way to check that you have a feasible solution available in the case of premature ending is to check that a call to the method isPrimalFeasible() returns True. In this case, the values you can get correspond to a solution to the LP. 

In the case of the "NumBest" status returned, there are primal infeasibilities close to zero and a solution is available but without any optimality proof made. In the course of optimization, the Barrier method can find several feasible solutions (with primal infeasibilities low enough) that are not optimal. The solution given is the best among those. 

Original Message:
Sent: Wed October 16, 2024 02:00 AM
From: Sophia Niu
Subject: Barrier's solution with NumBest status

Hi Philippe, 

Thanks a lot for your advice. May I further clarify my understanding with you? Really thanks.

1. As I know, the barrier algo stops when it finds feasible primal and dual solutions that are complementary, and at this time, the solution is also optimal. When there are numerical difficulties, the algo cannot find an optimal solution, meaning reaching the feasible primal and dual solution, then does that mean there will not be any feasible solution?  But why the solver can return a solution sometimes? Can we consider this solution as a strict feasible solution(just not optimal)?
2. At NumBest status, the solver may return the best solution found so far. May I know how the "best" is measured?
3. Another question, related to the barrier algo, is when a barrier solve is aborted with time limit, there is no solution yet, and then when we try to use cplex.GetValue(objectiveExpression) to get the objective expression's value, seems it returns the last objective value before abort. But at the last log row, we can see the Prim Inf is still very high(4.96e+7). So can we consider the objective value (-6.9e+07) as a possible objective value of a feasible solution? Or this value cannot be used for any purpose actually? Thanks a lot.

------------------------------
Sophia Niu

Original Message:
Sent: Mon October 14, 2024 11:34 AM
From: PhR
Subject: Barrier's solution with NumBest status

1. The solution status NumBest means that there have been some numerical difficulties and the optimality cannot be proven. The best solution found so far is available. 
https://www.ibm.com/docs/en/cofz/12.10.0?topic=api-cpx-stat-num-best

2. There is a usable solution but it is not proven optimal. It is not a basic solution and you can and up with many variables having fractional values instead of being at their bounds. 

3. Unfortunately, there is no way to stop crossover when there is a solution only. 

About numerical dificulties that can happen in Barrier optimization, a few hints are given here:
https://www.ibm.com/docs/en/icos/22.1.1?topic=difficulties-during-optimization

There is also a more detailed approach for solving a model with Barrier that can be helpful here:
https://www.gams.com/blog/2022/04/how-to-tune-cplex-options-for-times-models/

Regards

Philippe

Original Message:
Sent: Mon October 07, 2024 05:03 PM
From: Sophia Niu
Subject: Barrier's solution with NumBest status

Hi experts,

We have a relatively large LP problem, and when we use barrier to solve it, sometimes the solver returns the NumBest status with or without a solution. 

My questions are:

1. When there is a solution returned, what is the difference between this solution vs. the optimal status solution? 
2. Is the NumBest status solution feasible? Can we use it as a practical solution (maybe not optimal)? I know we can further do crossover, but it usally takes very long time, so would like to understand the tradeoff without doing crossover.
3. When we do crossover, we want just a feasible solution and not need an optimal solution. We tried to set Simplex Limits upperObj to a very small number (-1.7976931348623157e+308), and lowerObj to a very big numer (1.7976931348623157e+308). Although it seems to work, the solving status is always "Error", and the cplex status is "AbortObjLim", even if there is a solution.  It looks strange. So my questions are, is our setting wrong? Is there better way to achieve this (stop crossover when there is a feasible solution)? 
   
   

Thanks a lot.

------------------------------
Sophia Niu
------------------------------