### Handling "≥"-constraints: the two-phase Simplex method

• An example of Linear Program with "≥"-constraints

• Consider the following Linear Program (constrained optimilization problem with linear objective and constraints):

 ``` max: z = x1 + x2 s.t.: x1 + 2x2 ≤ 8 3x1 + 2x2 ≤ 12 x1 + 3x2 ≥ 3 <----- **** New "feature **** ```

• The feasibility region:

• Converting   "≥"-constraints into  "="-constraints

• Fact:

 A   ""≥"-constraint" is converted into a   ""="-constraint" using surplus variables with a negative sign !!!

• Example:

 ``` Constraint: x1 + x2 ≥ 5 (x1 ≥ 0, x2 ≥ 0 are implied) Equivalent constraint: x1 + x2 - s1 = 5 (x1 ≥ 0, x2 ≥ 0 and s1 ≥ 0 are implied) Notice: negative sign (-s1) ```

• The effect of   "≥"-constraints   on the initial Simplex Tableau

• Linear Program:

 ``` max: z = x1 + x2 s.t.: x1 + 2x2 ≤ 8 3x1 + 2x2 ≤ 12 x1 + 3x2 ≥ 3 ```

Re-write the Linear Programming problem as a system of linear equations (=):

 ``` z = x1 + x2 ⇒ z - x1 - x2 = 0 x1 + 2x2 ≤ 8 ⇒ x1 + 2x2 + s1 = 8 3x1 + 2x2 ≤ 12 ⇒ 3x1 + 2x2 + s2 = 12 x1 + 3x2 ≥ 3 ⇒ x1 + 3x2 - s3 = 3 ```

• Put the linear system into a matrix:

 ``` Linear system: z - x1 - x2 = 0 x1 + 2x2 + s1 = 8 3x1 + 2x2 + s2 = 12 x1 + 3x2 - s3 = 3 Lined up: -x1 - x2 + z = 0 x1 + 2x2 + s1 = 8 3x1 + 2x2 + s2 = 12 x1 + 3x2 - s3 = 3 Matrix form: x1 x2 s1 s2 s3 z | RHS ===============================+==== -1 -1 0 0 0 1 | 0 1 2 1 0 0 0 | 8 3 2 0 1 0 0 | 12 1 3 0 0 -1 0 | 3 ```

Note:

• We may detect the basis easier if you write the above matrix as follows:

 ``` x1 x2 s1 s2 s3 z | RHS ===============================+==== -1 -1 0 0 0 1 | 0 1 2 1 0 0 0 | 8 3 2 0 1 0 0 | 12 1 3 0 0 1 0 | -3 ```

• The basis variables are:

 s1           s2 s3 z

• The corresponding "basic" solution is:   (I put "basic" in quotes because it is not a basic solution !!!)

 x1   =   0       (non-basic variable = 0)) x2   =   0       (non-basic variable = 0)) s1   =   8       (basic variable = RHS)) s2   =   12     (basic variable = RHS)) s3   =   −3     (basic variable = RHS))     ******** Trouble !!! ******         z   =   0        (basic variable = RHS))

Big trouble:

• s3   =   −3     violates the (implied) constraint:

 s3   ≥   0

This solution is not feasible !!!

• This "basic" solution is not basic at all --- because a basic solution must be feasible

• Techniques to find an initial (feasible) basic solution

• There are two (2) techniques that you can use to find a (feasible) basic solution when the Linear Program contains "≥"-constraint:

 The Big-M method The two-phase Simplex method

I like the two-phase Simplex method because it's more intuitive (so I'll teach you that :-))

• Artificial variables: a trick used in Simplex Method to obtain a feasible basic solution

• Simplex Tableau:

 ``` x1 x2 s1 s2 s3 z | RHS ===============================+==== -1 -1 0 0 0 1 | 0 1 2 1 0 0 0 | 8 3 2 0 1 0 0 | 12 1 3 0 0 -1 0 | 3 ```

• Trick (technique) to obtain an initial basic solution:

• For every "≥"-constraint, add an artificial variable with a canonical unit basic vector

Example:

 ``` x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== -1 -1 0 0 0 0 1 | 0 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ^ ^ | | | artificial variable | (with canonical unit vector) | The ≥-constraint ```

• Purpose of artificial variables:

• The artificial variables (such as a1 above) are not part of the optimization problem

In other words:

 We cannot apply the Simplex Method as long as the Simplex Tableau contains artificial variables (On the other hand, surplus variables (introduced by "≤"-constraints are OK))

• The artificial variables (like a1) are used temporarilly to help us find (compute) a feasible basic solution

• Problem:

 We must remove all artificial variables from the Simplex tableau first !!!

• Removing artificial variables

• Our Simplex tableau so far is as follows:

 ``` x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== -1 -1 0 0 0 0 1 | 0 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ```

The corresponding solution is:

 ``` z = 0 s1 = 8 s2 = 12 a1 = 3 <--- "Remove" this variable ```

• How to "remove" a variables from an equation:

 A variable will have no effect on (= contribute nothing to) an equation if its value is equal to 0 !!

• Therefore:

• We must find a solution (i.e., give values to all variables in the Simplex tableau) that has the property that:

 all artificial variables are equal to 0

• Finding a solution with all artificial variable = 0

• Our Simplex tableau so far is as follows:

 ``` x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== -1 -1 0 0 0 0 1 | 0 <-- Objective function 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ```

Fact:

• Only the constraints determine the feasibility of a solution

• In other words:

 The objective function does not have any effect on the feasibility of a solution (See: the feasibility region in this webpage: click here )

• Trick to obtain a solution where all artificial variables are equal to 0:

• Construct a LP using the same set of constraints but with a different objective function where:

• Optimal solution of the LP is found using:

 All artificial variables = 0 !!!

(Because All artificial variables, the artificial variables now has no effect on the equations and we can "remove" them !!!)

• An objective function where the optimal solution is found using:

 All artificial variables = 0

is:

 ``` max: -a1 - a2 - ... - am (Remember that: a1 ≥ 0 a2 ≥ 0 ... am ≥ 0 ) ```

Because:

• -a1 - a2 - ... - am ≤ 0 (is always negative) since: a1 ≥ 0 a2 ≥ 0 ... am ≥ 0 )

• The maximum value is therefore: 0

• And the only way to obtain this maximum is:

 ``` a1 = 0 a2 = 0 ... am = 0 ```

Now:

• We solve the Linear Program with:

• Same set of constraints

• But using the objective function:

 ``` max: -a1 - a2 - ... - am ```

• When the Simplex Method terminates, we will have:

 a solution (= values of variables) where all artificial variables = 0 And we can remove the artificial variables from the Linear Program

More importantly:

• Because the constraints used are the same as those in our original Linear Program (and a feasible solution is only determined by the constraints used):

• The solution that we obtained by solving a different objective function:

 ``` max: -a1 - a2 - ... - am ```

is also a feasible solution of the original Linear Program !!!

(Because they use the same set of constraints !!!)

• Terminology:

• Phase 1 of the Simplex method:

solving (= finding value for variables) an Linear Program with the same constraints, but using the objective function:

 ``` max: -a1 - a2 - ... - am ```

• Overview of the two-phased Simplex Method

• The 2-phased Simplex Method:

 ``` 1. Set up an initial Simplex Tableau: 1a. Add a surplus variable si to each: "≤"-constraint and "≥"-constraint Note: surplus variables of "≥"-constraint has negative sign ! 2. If there is one or more "≥"-constraints then: 2a: Add an artificial variable ai to each ≥ constraint 2b. Use the Simplex method to solve: max: -a1 - a2 - ... - an Using same set of constraints Note: you need to fix the Simplex Tableau first (see example) 2c. When Simplex method terminates, replace the objective row of the Final Simplex Tableau by the original objective function 3. Use the Simplex method to solve the LP Note: you need to fix the Simplex Tableau first (see example) ```

• Note:

 Phase 1 of the 2-phased Simplex Method is only necessary when there are one or more "≥"-constraints in the LP

• Example: 2-phased Simplex Method

• Consider the LP with an   "≥"-constraint:

 ``` max: z = x1 + x2 s.t.: x1 + 2x2 ≤ 8 3x1 + 2x2 ≤ 12 x1 + 3x2 ≥ 3 ```

• Step 1: Add a surplus variable to each "≤"- and "≥"-constraint:

 ``` max: z - x1 - x2 = 0 s.t.: x1 + 2x2 + s1 = 8 3x1 + 2x2 + s2 = 12 x1 + 3x2 + s3 = 3 ```

Re-written in table form:

 ``` x1 x2 s1 s2 s3 z | RHS ===============================+==== -1 -1 0 0 0 1 | 0 1 2 1 0 0 0 | 8 3 2 0 1 0 0 | 12 1 3 0 0 -1 0 | 3 ```

(which did not have a feasible initial basic solution)

• Step 2: (this step is necessary because there is one "≥"-constraint)

• Add an artificial variable to each "≥"-constraint:

 ``` x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== -1 -1 0 0 0 0 1 | 0 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ^ ^ | | ```

• Remove the artificial variables by solving:

 ``` max: z = -a1 Or: max: z + a1 ```

using the same set of constraints.

The Linear Program that we must solve is:

 ``` x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== 0 0 0 0 0 1 1 | 0 (Objective function = z + a1) 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ```

• Before we can solve this Linear Program:

 ``` x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== 0 0 0 0 0 1 1 | 0 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ^ ^ ^ | | | We onloy have 3 canonical unit vectors !!! ```

We must "fix" it (because there are 4 rows and only 3 canonical unit vectors) by:

 Making all artificial variables into basic variables

Before fixing:

 ``` x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== 0 0 0 0 0 1 1 | 0 (row 1) 1 2 1 0 0 0 0 | 8 (row 2) 3 2 0 1 0 0 0 | 12 (row 3) 1 3 0 0 -1 1 0 | 3 (row 4) ```

Make a1's column into a unit vector by subtracting   row 4 from row 1:

 ``` x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== -1 -3 0 0 1 0 1 | -3 (row 1) 1 2 1 0 0 0 0 | 8 (row 2) 3 2 0 1 0 0 0 | 12 (row 3) 1 3 0 0 -1 1 0 | 3 (row 4) ```

Now we have 4 canonical unit vector and we can apply the Simplex Method !

• Run the Simplex Method:

• Iteration 1:

 ``` largest neg coef | V x1 x2 s1 s2 s3 a1 z | RHS ====================================+==== -1 -3 0 0 1 0 1 | -3 (row 1) 1 2 1 0 0 0 0 | 8 (row 2) (test ratio = 4) 3 2 0 1 0 0 0 | 12 (row 3) (test ratio = 6) 1 3 0 0 -1 1 0 | 3 (row 4) (test ratio = 1) Normalize pivot row: x1 x2 s1 s2 s3 a1 z | RHS =====================================+==== -1 -3 0 0 1 0 1 | -3 (row 1) + 3 x row 4 1 2 1 0 0 0 0 | 8 (row 2) - 2 x row 4 3 2 0 1 0 0 0 | 12 (row 3) - 2 x row 4 0.3 1 0 0 -0.3 0.3 0 | 1 (row 4) Sweep the pivot column: x1 x2 s1 s2 s3 a1 z | RHS ==========================================+==== 0.0 0.0 0.0 0.0 0.0 1.0 1 | 0.0 0.3 0.0 1.0 0.0 0.7 -0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 -0.7 0 | 10.0 0.3 1.0 0.0 0.0 -0.3 0.3 0 | 1.0 ```

• Iteration 2:

 No negative coefficient left

Done.

• Basic solution:

 ``` x1 = 0 x2 = 1 s1 = 6 s2 = 10 s3 = 0 a1 = 0 ```

Because a1 = 0, we can now remove this variable without affecting the values of the other variables:

 ``` x1 x2 s1 s2 s3 z | RHS =====================================+========= 0.0 0.0 0.0 0.0 0.0 1 | 0.0 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ```

Notice that the baisc solution:

 ``` x1 = 0 x2 = 1 s1 = 6 s2 = 10 s3 = 0 a1 = 0 ```

satisfy all original constraint (so it is a feasible solution for the original problem)

• Step 3: Use the Simplex method (and the found feasible solution) to solve the original LP

• The Simplex Tableau:

 ``` x1 x2 s1 s2 s3 z | RHS =====================================+========= 0.0 0.0 0.0 0.0 0.0 1 | 0.0 (Objective: max -a1) 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 <--- constraints 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ```

represents the following feasible solution:

 ``` x1 = 0 x2 = 1 s1 = 6 s2 = 10 s3 = 0 ```

(These values satisfies all constraints !!!)

• We now use this feasible solution to solve the original objective function

This is done by:

• Keeping all the constraint rows

• Replace the objective function by the original objective function:

 ``` max: z = x1 + x2 or: max: z - x1 - x2 = 0 ```

In other words:

 ``` Our Simplex Tableau: x1 x2 s1 s2 s3 z | RHS =====================================+========= 0.0 0.0 0.0 0.0 0.0 1 | 0.0 (Objective: max z+a1) 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 <--- constraints 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ^ | This was a unit vector New Simplex Tableau: x1 x2 s1 s2 s3 z | RHS =======================================+==== -1.0 -1.0 0.0 0.0 0.0 1 | 0.0 (Objective: max z-s1-s2) 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 <--- Keep constraints 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ^ | It is no longer a unit vector: fix it first ! ```

• Preparation for Simplex: fix all corrupted unit vectors

 ``` x1 x2 s1 s2 s3 z | RHS =======================================+==== -1.0 -1.0 0.0 0.0 0.0 1 | 0.0 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ^ | Corrupted unit vector ```

Fix it: (make the column into a unit vector again) by adding   row 4   to   row 1:

 ``` x1 x2 s1 s2 s3 z | RHS =======================================+==== -0.7 0.0 0.0 0.0 -0.3 1 | 1.0 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ```

• Note:

• Corresponding basic solution:

 x1 = 0     (non-basic) x2 = 1     (basic, equal to RHS) (z = 1)     (basic, equal to RHS)

which is a corner point in the feasible region:

• Now we can run the Simplex Method to solve the original problem:

• Starting Simplex Tableau:

 ``` Largest neg coef. | v x1 x2 s1 s2 s3 z | RHS =======================================+==== -0.7 0.0 0.0 0.0 -0.3 1 | 1.0 0.3 0.0 1.0 0.0 0.7 0 | 6.0 (Test ratio = 18) 2.3 0.0 0.0 1.0 0.7 0 | 10.0 (Test ratio = 4.3) 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 (Test ratio = 3) Next pivot = 0.3 ```

Current basic solution:   x1 = 0,   x2 = 1,   (z = 1)

• After pivot operation:

 ``` Most neg. coef. | v x1 x2 s1 s2 s3 z | RHS =======================================+==== 0.0 2.0 0.0 0.0 -1.0 1 | 3.0 0.0 -1.0 1.0 0.0 1.0 0 | 5.0 (Test ratio = 5) 0.0 -7.0 0.0 1.0 3.0 0 | 3.0 (Test ratio = 1) 1.0 3.0 0.0 0.0 -1.0 0 | 3.0 (Test ratio = illegal) Next pivot = 3.0 ```

Current basic solution:   x1 = 3,   x2 = 0,   (z = 3)

• After pivot operation:

 ``` Most neg. coef. | v x1 x2 s1 s2 s3 z | RHS =======================================+==== 0.0 -0.3 0.0 0.3 0.0 1 | 4.0 0.0 1.3 1.0 -0.3 0.0 0 | 4.0 (Test ratio = 3) 0.0 -2.3 0.0 0.3 1.0 0 | 1.0 (Test ratio = illegal) 1.0 0.7 0.0 0.3 0.0 0 | 4.0 (Test ratio = 6) Next pivot = 1.3 ```

Current basic solution:   x1 = 4,   x2 = 0,   (z = 4)

• After pivot step:

 ``` x1 x2 s1 s2 s3 z | RHS =======================================+==== 0.0 0.0 0.3 0.2 0.0 1 | 5.0 0.0 1.0 0.7 -0.2 0.0 0 | 3.0 0.0 0.0 1.7 -0.2 1.0 0 | 8.0 1.0 0.0 -0.5 0.5 0.0 0 | 2.0 No improving pivots left ===> optimal ! ```

Corresponding solution:   (x1 = 2,   x2 = 3,   z = 5)