Decision Optimization

Decision Optimization

Delivers prescriptive analytics capabilities and decision intelligence to improve decision-making.

 View Only
Expand all | Collapse all

constraint programming feature selection neural network

  • 1.  constraint programming feature selection neural network

    Posted Tue February 16, 2016 03:03 PM

    Originally posted by: FiFila


    Hi all;

     

    This is a great kind of you if anybody gives some guidance about the best implementation environment for my problem or any useful guidance to solve my problem ?

    What I want to do is improvement of a solution for a problem by constraint programming which was solved by NN before . My database is very large. There is no obligation for me to choose any environment for implementation.

    My problem is to choose from a list of parts that each part has a number of characteristics and there are some constraints for them. For example a shortage and limitation of some of these characteristics.
     (For example:
    If X and Y are two parts having constraints listed in the following:
     if X1..Xn is a list of characteristics of part A, I have a constraint 4X1+5X4<10, X3<5, X7<23,...
     if W1..Wm is a list of characteristics of part B, I have a constraint 3W1+...6W3=12, W5<42,....
     I want to know the possibility of different outputs (combination of these parts (A,B,...) with consideration of their constraints or I want to know which outputs can be make by consideration of these constraints). 

    Would you please give me some guidance to know how to combine its NN solution with constraint programming?

     

    Thanks

    Fila


    #ConstraintProgramming-General
    #DecisionOptimization


  • 2.  Re: constraint programming feature selection neural network

    Posted Tue February 23, 2016 01:04 PM

    Originally posted by: GGR


    Hi Fifila

     

    I am not sure understanding your issue. Anyway I will try to rephrase. If I am wrong please correct me

     

    Your NN simulates the value affectation to variable from which you want X

    I think your constraints correspond to transformation of your input with some step functions on the linear combinations which is a feature that is declarable in a layer (although I am not a specialist of NN).

     

    If you cannot model with your NN language such a feature because it is an NP hard problem, you can use combinatorial technique to declare a new component  ( Although such a layer in the NN will certainly be quite time expensive). In your case in all constraint are linear as well as some objective, I suggest rather use Linear Programming than Constraint Programming.

     

    Hope that helps

     

     

     


    #ConstraintProgramming-General
    #DecisionOptimization


  • 3.  Re: constraint programming feature selection neural network

    Posted Wed February 24, 2016 01:06 AM

    Originally posted by: FiFila


    Hi GGR;

    Thanks alot for your help.

    My problem is a type of assembly problem and there are some products which are containing some features. The database is very huge. Each of these features can be available or not or some of them are available for any product. Each of these features has a number of selections. I want to know if there is some restrictiosn in some products, it is not listed in outputs list to select for buying. Also gives some suggestions for other products that are so similar to available repository.  This is modeled with GA+NN, I want to improve it by NN+CP  I want to know is it possible to implement in CP optimizer because I saw a similar solution for a problem in or-tools. It has one hidden layer, one input and one output layer. Because if I want to implement the nonlinear functions such as Tanh, I do not know how to implement this. 

     

    I hope this is clear about my problem. It is kind of you if you give me some guidance.

     

    Best 

     


    #ConstraintProgramming-General
    #DecisionOptimization


  • 4.  Re: constraint programming feature selection neural network

    Posted Wed February 24, 2016 10:31 AM

    Originally posted by: GGR


    Hi

    It seems you want to model a product configuration problem.

    That is indeed a classical example of use of CP Optimizer. You will find attached a example: the configuration of an aquarium with numerous conditions of compatibilities, aesthetic, capacities, ....

     

     

     


    #ConstraintProgramming-General
    #DecisionOptimization


  • 5.  Re: constraint programming feature selection neural network

    Posted Thu March 17, 2016 01:10 AM

    Originally posted by: FiFila


    Hi Rogerie;

    Thanks for your reply. 

    My real problem in nonlinear problem. This is the reason I should work with constraint programming. 

    As I am a beginner in constraint programming, I do not know how to implement iloconstraint , wrapping

    For instance:

    A very simple implementation is that I have a neural network (2 layers) with 10 inputs and only one output. I want to minimization the output. 

    For 1 layers:  range[1..10]; x[1..10]:input,  y: output.  y=select i in range which y=prod(x[i]*w[i]+b[i]) is minimized. (however I should implement in some more layers)

    by consideration of these constraints: 

    x[1]+2*x[2]*x[2]!=20;    x[3]+0.2x[4]!=2.1; and similar constraints .....

     

     

    I think for the constraint I should use model.add() for instance:  model.add(x[1]+2*x[2]*x[2]<>=20);  model.add(x[3]+0.2x[4]<>2.1)

    But I do not know how to combination them with NN (for instance the above small example). For example for very simple example that I have written above.

    I also do not know how,when, and why I should use wrapper??

    Would you please give me some guidance to how to implement this simple example? and also when and how I should use wrapper?

     

    Best regards

    Fila


    #ConstraintProgramming-General
    #DecisionOptimization


  • 6.  Re: constraint programming feature selection neural network

    Posted Thu March 17, 2016 05:38 AM

    Originally posted by: rdumeur


    Dear Fila,

     

    First I'd like to know exactly what you want to do with CPO. I understand that you want to censor the output of your GA+NN system using constraint programming. That is,if the GA+NN provides some variable assignments, you want to filter out assignments that do not honor a set of constraints.

    Is it correct?

    Cheers,


    #ConstraintProgramming-General
    #DecisionOptimization


  • 7.  Re: constraint programming feature selection neural network

    Posted Fri March 18, 2016 03:09 AM

    Originally posted by: FiFila


    Dear Renaud;

    Thanks for your reply. 

    No, the inputs are not integer at all of the layers because of existence of tansig (a NN transfer function). (I start with 2 layers NN at first step)  

    Yes, I want to filter out the unwanted output. I want to implement some constraint on existence NN. 

    Cheers

    Fila


    #ConstraintProgramming-General
    #DecisionOptimization


  • 8.  Re: constraint programming feature selection neural network

    Posted Thu March 17, 2016 05:50 AM

    Originally posted by: rdumeur


    Dear Fila,

     

    Another question: is your problem expressed with integer variables only? 

    Cheers,


    #ConstraintProgramming-General
    #DecisionOptimization


  • 9.  Re: constraint programming feature selection neural network

    Posted Thu March 17, 2016 10:37 AM

    Originally posted by: FiFila


    Hi Renaud;

    Thanks for your reply. 

    No, the inputs are not integer at all of the layers because of existence of tansig (a NN transfer function). (I start with 2 layers NN at first step)  

    Yes, I want to filter out the unwanted output. I want to implement some constraint on existence NN. 

    Cheers

    Fila


    #ConstraintProgramming-General
    #DecisionOptimization


  • 10.  Re: constraint programming feature selection neural network

    Posted Fri March 18, 2016 03:44 AM

    Originally posted by: rdumeur


    Dear FiFila,

    Since your variables are floating points and CPO works on integer variable, you will need to scale up their domains by some power of 10. The define float expressions (x/<floating scaling constant>) in your model. Using these expressions, you will be able to express the constraints on your variables.

    Here is a simple skeleton code that invokes the CPO solver to check if some floating point values are consistent with a given model:

     

    #include <ilcp/cpext.h>
    
    ILOSTLBEGIN
    
    const IloNum    VarScale = 100000.0;
    
    ILCGOAL6(CheckValuesGoal, IloInt, nx,
             IloInt, nw,
             IloNum*, xvalues, 
             IloNum*, wvalues,
             IlcIntVarArray, xvars,
             IlcIntVarArray, wvars)
    { 
      for(IlcInt i = 0; i < nx; ++i)
        xvars[i].setValue(IloRound(xvalues[i]*VarScale));
      for(IlcInt i = 0; i < nw; ++i)
        wvars[i].setValue(IloRound(wvalues[i]*VarScale));
      return 0;
    }
    
    ILOCPGOALWRAPPER6(CheckValuesGoalWrapper,cp,
                      IloInt, nx,
                      IloInt, nw,
                      IloNum*, xvalues, 
                      IloNum*, wvalues,
                      IloIntVarArray, xvars,
                      IloIntVarArray, wvars) {
      return CheckValuesGoal(cp, nx, nw, xvalues, wvalues, cp.getIntVarArray(xvars), cp.getIntVarArray(wvars));
    }
    
    int main() {
      IloEnv env;
      IloModel model(env);
      // some var range 
      IloInt        xmin = IloCeil(2.1*VarScale);
      IloInt        xmax = IloFloor(10.3*VarScale);
      IloInt        nx = 10;
      IloInt        wmin = IloCeil(0.7*VarScale);
      IloInt        wmax = IloFloor(17.8*VarScale);
      IloInt        nw = 20;
    
      IloIntVarArray        xvars(env, nx, xmin, xmax);
      IloIntVarArray        wvars(env, nw, wmin, wmax);
      IloNumExprArray        x(env, nx);
      IloNumExprArray        w(env, nw);
      for(IloInt i = 0; i < nx; ++i)
        x[i] = xvars[i] / VarScale;
      for(IloInt i = 0; i < nw; ++i)
        w[i] = wvars[i] / VarScale;
      
      // your constraints.
      model.add(4*x[0] <= 2*w[2]);
      model.add(3*x[2] >= 12*w[4]);
    
      IloCP  cp(model);
    
      IloNum*  xvalues = 0; // replace with your value arrays
      IloNum*  wvalues = 0; //      ... idem ..
    
      if(cp.solve(CheckValuesGoalWrapper(env, nx, nw, xvalues, wvalues, xvars, wvars))) 
        cout << "Values are correct" << endl;
    
      cp.end();
      env.end();
    
      return 0;
    }
    

     

    The ILOCPGOALWRAPPER macro creates a "concert" goal wrap[per which, when processed by the solver, will produce an "ILC" goal 'CheckValueGoal'.

    This goal will simply assign values (xvalues,wvalues) to xvars and wvars variables (scaling them by some factor).

    If the values are not consistent with the constraint expressed in the model, a failure with occur and 'cp.solve' will return false.

     

    I hope this helps.

     

    Cheers,

     


    #ConstraintProgramming-General
    #DecisionOptimization


  • 11.  Re: constraint programming feature selection neural network

    Posted Fri March 18, 2016 04:12 AM

    Originally posted by: FiFila


    Dear Renauld;

    Thanks for your email but would you please let me know how I can add the some other constraints (For instance I want to add NN  : 

    A very simple implementation is that I have a neural network (2 layers) with 10 inputs and only one output. I want to minimization the output. 

    For 1 layers:  range[1..10]; x[1..10]:input,  y: output.  y=select i in range which y=(x[i]*w[i]+b[i]) is minimized (This is for one layer). (however I should implement in some more layers) ) 

    I do not know how I can do the above code. I know I should do something like" iloconstraint:NN" ... and in this function I should implement the following  { For 1 layers:  range[1..10]; x[1..10]:input,  y: output.  y=select i in range which y=(x[i]*w[i]+b[i]) is minimized } . 

    Would you please help me to complete your presented code to implement this simple model? 

     

    Best regards

    Fila


    #ConstraintProgramming-General
    #DecisionOptimization


  • 12.  Re: constraint programming feature selection neural network

    Posted Fri March 18, 2016 05:56 AM

    Originally posted by: rdumeur


    Dear FiFila,

     

    To select the "i" var value that minimize the cost function you should use the element constraint. 

    http://www.ibm.com/support/knowledgecenter/SSSA5P_12.6.3/ilog.odms.cpo.help/refcppcpoptimizer/html/functions/IloElement.html?lang=en

    For instance:

     IloIntVar index(env,0,n-1);
      model.add(IloMinimize(env, x[index]*w[index]));
    

    Cheers,

     


    #ConstraintProgramming-General
    #DecisionOptimization


  • 13.  Re: constraint programming feature selection neural network

    Posted Fri March 18, 2016 01:44 PM

    Originally posted by: FiFila


    Dear Renauld;

     

    Thanks for your reply. As my model is not one layer therefore I should call several times model.add(IloMinimize(env,x[index]*w[index])); and I think this is impossible because this constraint should be achieved in parallel to other constraints.

     

    Let me explain more my problem. As it has been shown in the attachment you can see my small example (x[1..5]) are the inputs of layer1. Each layer is replaced with Tansig function (tansig(xij)->xij) and then  sum(x1i*w1ji+b1j)=x2j (i,j=1..5 and x2j are inputs of layer 2) &&  sum(x2i*w2i+b2i)=y (y is output), I want to minimize the "y" considering the constraints for instance: x[1]+2*x[2]*x[2]!=20;    x[3]+0.2x[4]!=2.1; and similar constraints .....  (These constraint can be written by model.add() but what is my problem is how to implement the NN section and how to achieve the minimization of neural network) {because I have at least 3 layers }.

    The outputs of this program are the amount of x1j (inputs of first layer)

    when I want to model.add(other constraints). I think that this is not possible to call a constraint several times in combination of other constraint (to be able to call this function several times base on needing). This is the reason I think I should call a function (IloConstraint:NN). Although I do not know how to write a function for iloconstraint.

    Best regards

    Fila

     

    Best

    Fila


    #ConstraintProgramming-General
    #DecisionOptimization