### Solving Integer Programs using the "Branch and Bound" heuristic

• Basis for the Branch and Bound method

• The basis for the branch and bound method are these facts:

 max(IP)   ≤   max(LP) (LP = the linear relaxation of the IP) If the LP is infeasible (i.e., has no solution), then the IP is also infeasible If the LP problem has an integer-valued optimal solution, then the solution is equal to the optimal solution for the IP problem.

• What is the "branch-and-bound" heuristic

• Branch and bound uses 2 heuristics

• The branch heurstic

 The branch heuristic is used to force the Simplex Method away from a non-integer valued solution (I.e., the branch heuristic will make the Simplex Method find integer valued solutions) The branch heuristic will generate many different LP problems in the process !

• The bound heuristic

 The bound heuristic is used to limit the number of LP problems generated by the branch heuristic

• How to solve the Integer Program:

• We will start with the linear relaxation of the Integer Program

Example:

• Then apply the branch and bound heurstic on the linear relaxation until we find the optimal integer solution

• The Branch heuristic will produce many Linear Programs LPk that must be solved by using the Simplex Method

 Each Linear Program LPk that has non-integer valued solution may be split (= branch) Each Linear Program LPk that has an integer valued solution be prune (= bound) the unsolved Linear Programs

• One of these Linear Programs generated by the branch procedure will have the optimal integer solution

• The "Branch" heuristic

• Branching:

• Branching = to split a LP problem (A) into 2 sub-problems (A1 and A2) where:

 Optimal solution(A)   =   max( Optimal solution( A1) , Optimal solution( A2) )

• The subproblems will help us obtain an integer valued solution

 The branch procedure prohibits the Simplex Method from using some non-integer solution and forces the Simplex Method for find integer-valued solutions.

• The branch procedure (general description)

• Suppose a LP problem (A) has a non-integer valued variable xk as optimal solution:

Idea:

 We force the Simplex Algorithm to not use the fraction optimal value

• The branch procedure will create the 2 LP problems by adding one of the 2 constraints:

 ``` Linear Program 1: add the constraint xk ≤ "rounded-down-value" Linear Program 2: add constraint xk ≥ "rounded-up-value" ```

Example:

Note that:

 In both new LP problems (A1 and A1), the solution where xk = 3.75 are now infeasible !!!

• A concrete example of the branch heuristic:

• Consider the following Integer Program:

• Corresponding linear relaxation:

Optimal solution: (solved using the lp_solve tool)

 ``` Value of objective function: 41.25000000 Actual values of the variables: x1 2.25 x2 3.75 ```

Graphically:

• We will now force the Simplex Method to avoid using x2 = 3.75

The optimal integer solution have be one of the following:

 ``` (1) x2 ≤ 3 (2) x2 ≥ 4 ```

Therefore, we will replace the original LP problem with 2 derived LP's, each with one additional constraint:

One of these 2 LP's must contains the optimum solution (and hopefully, the solution is integer valued)

• How does the bound heuristic "help" the Simplex Method find integer valued solutions

• Before we discuss the bound heuristic, let's solve the Linear Programs generated by the previous branch example:

• Branching on the non-integer value x2 = 3.75:

We replace:

with these 2 derived LP problems:

• We can use lp_solve to find the solution of the 2 new LP problems

Here are the inputs for lp_solve:

Solutions:

The following figure shows these 2 solution in more detail:

We can see that the branch heuristic has successfully forced the Simplex Method to find an integer solution by:

 "Cutting" the feasible solution space into 2 halves along an integer-valued line and creating integer valued corner points for the Simplex Method !!!

• Summary:

 The Branch heuristic "helps" the Simpelx Method find integer valued solutions by creating integer valued corner points in the feasibility region (Because the Simplex Method will only use corner points for optimal solutions !!!)

• The incumbent solution

• Definition: incumbent solution

 incombent solution (or incumbent for short) = the best solution value found so far that satistfies the integer-valued constraints.

Example:

• When we solve the first LP problem:

we do not have any incumbent solution yet

(Because the solution does not satisfy the integer value constraints)

• After we branch on x2 = 3.75 and solved these 2 derived LP's:

we have an incumbent solution:

 ``` incumbent Z* = 39 (the best solution value so far) ```

• Note:

 The incumbent solution is traditionally denoted by Z*

• The bound heuristic(s)

• Recall:

• The branch heuristic will generate many derived Linear Program problems

• Some of these Linear Program may produce the optimal solution for the Integer Program

 To find a solution for the Integer Program, the branch heuristic will split a Linear Program into 2 linear program using some variable that has a non-integer valued solution

• Purpose of The Bound heuristic(s):

 A bound heuristic is used to remove some Linear Programs that are generated by the branch heuristic (Because those Linear Programs will certainly not produce a better solution)

• There are 3 bounding technqiues to remove a Linear Program LP:

1. If the LP is infeasible, we can delete LP, because:

 If the LP is infeasible, then the corresponding Integer program (IP) is also infeasible

2. If:   solution( LP ) Z* (incumbent solution),   we can delete LP, because:

• From the property:

 ``` OptimalSOlution( IP ) ≤ OptimalSolution( LP ) ≤ Z* ```

we know that we will not find a better solution than Z* using this LP !!!

3. Finally, we can also delete the LP when:

 The solution of LP is integer valued !!! (Because we have solve the Integer Program !!!)

Note:

 We must update the incumbent solution in this case !!!!

• The branch and bound algorithm

• Branch and Bound Algorithm:

 ``` Input: Integer Program IP; SetOfLP = Linear Relaxation of IP; Z* = NULL; while ( SetOfLP ≠ empty ) { LP1 = pick an arbitrary LP from SetOfLP; Solve LP1 with Simplex Method: Max1 = optimum value of LP1 Sol1 = solution values of the variables if ( LP1 is infeasible ) { delete LP1 from SetOfLP; // Bound heuristic 1 } else if ( Z* ≠ NULL && Max1 ≤ Z* ) { delete LP1 from SetOfLP; // Bound heuristic 2 } else if ( Sol1 has only integer values ) { if ( Max1 > Z* ) Z* = Max1; // Better incumbent solution found ! delete LP1 from SetOfLP; // Bound heuristic 3 } else { /* =============================== Apply "branch" heuristic =============================== */ Select a variable xk that has a non-integer solution value; Replace LP1 in SetOfLP with 2 derived LP programs: LP1-A: add a constraint "xk ≤ roundDownValue" LP1-A: add a constraint "xk ≥ roundUpValue" } } ```

• Worked out example

• Integer Program:

• Branch and Bound Solution:

• Initialization:

• Iteration 1:

• Current state:

• Pick an LP from SetOfLP and solve it:

• According to the Branch and Bound Algorithm, we perform a branch operation

(we can pick a "branch" (split) variable arbitrarily)

• Iteration 2:

• Current state:

• Pick an LP from SetOfLP and solve it:

• According to the Branch and Bound Algorithm, we perform a bound operation

But: we must update Z*

• Iteration 3:

• Current state:

• Pick an LP from SetOfLP and solve it:

• According to the Branch and Bound Algorithm, we perform a branch operation

(We have to branch on x1 because it's the only non-integer valued "branch" (split) variable)

• Iteration 4:

• Current state:

• Pick an LP from SetOfLP and solve it:

• According to the Branch and Bound Algorithm, we perform a branch operation

(We have to branch on x1 because it's the only non-integer valued "branch" (split) variable)

• Iteration 5:

• Current state:

• Pick an LP from SetOfLP and solve it:

• According to the Branch and Bound Algorithm, we perform a bound operation

• Iteration 6:

• Current state:

• Pick an LP from SetOfLP and solve it:

• According to the Branch and Bound Algorithm, we perform a bound operation

Note:

 Because the new solution (37) is ≤ incumbent solution (39), the incumbent solution is unchanged !

• Iteration 7:

• Current state:

• Pick an LP from SetOfLP and solve it:

• According to the Branch and Bound Algorithm, we perform a bound operation

Note:

 Because the new solution (40) is > incumbent solution (39), we update the incumbent solution !

• Iteration 8:

• Current state:

• DONE !!!

 ``` Optimal Solution: Object function value = 40 x1 = 0 x2 = 5 ```