### Knapsack problems and constrained optimization

• Knapsack problems

• Knapsack problem(s):

• You are given a knapsack which has a certain capacity (volume and/or weight):

• You have a number of items:

 And so on....

Each item has:

 A value (that gives a measure of benefit if you pack one of these items into the knapsack) A weight (or volume) (that gives a measure of size reduction to the capacity of the knapsack if you pack one of these items into the knapsack)

• The problem:

• Pack as many items as they will fit in the knapsack such that:

 The total value packed in the knapsack is maximized

• Applications:

 The knapsack problem is one of the most studied problems in combinatorial optimization, with many real-life applications. Example: pack food in a knapsack for maximum nutritient value.

• Variations of knapsack problems:

• Many variations of knapsack problems exist:

 How many items (of each kind) do you have How many knapsacks do you have More types of constraints And so on...

• Note:

• This course will not study all kinds of variations.

• We will study the following 2 variants:

 The 0,1 knapsack problem The unbounded knapsack problem (as a way to introduce the next topic)

• A more general group of problems: Constrained optimization

• Fact:

• Knapsack problems is a subset of a larger class of problems called:

 constrainted optimization

• Optimization problems:

• Optimization problem = determine the optimum value for a certain system

Example:

 ``` max f( x1, x2, ..., xn ); ```

(It the system can decribed as by a function f(x1, x2, ..., xn), the optimization problem is solved by finding the zero's of the derivitive function:

 ``` f'(x1, x2, ..., xn) = 0 ```

• Types of optimization problems

1. Unconstrainted optimization problems

 There are no constrainted on the range of values that x1, x2, ..., xn can take on

Example:

 ``` max ( -x12 + 3x1 - x24 + 5x2 + 4 ) ```

2. Constrainted optimization problems

 The range of values that x1, x2, ..., xn can take on is limited

Example:

 ``` max ( -x12 + 3x1 - x24 + 5x2 + 4 ) subject to: x1 ≤ 9 (we only have 5 units of resource 1) x1 + x2 ≤ 5 (Resources 1 and 2 are related) ```

• The general form of a constrained opimization problem:

 ``` max f( x1, x2, ..., xn ); subject to: g1( x1, x2, ..., xn ) = c1 // Equality constraints g2( x1, x2, ..., xn ) = c2 ... gu( x1, x2, ..., xn ) = cu h1( x1, x2, ..., xn ) ≤ d1 // Inequality constraints h2( x1, x2, ..., xn ) ≤ d2 ... hv( x1, x2, ..., xn ) ≤ dv ```

• Application areas of constrained optimization

• Resource allocation problems:

 You have a limited amount of resources You have to maximize the profit using the limited (bounded) resources

• Decision problems

 You have a number of task to do in a limited amount of time Select the set of task that will maximize some profit function

• Remark:

 I will formulate the knapsack problems as constrained optimization problems There are many important problems in real life that are constrained optimization problems

• Problem description: the 0,1 knapsack problem

• Problem description:

• Given N types of items:

 ...

• The item type i has a weight wi

• The item type i has a value vi

0,1 refers to:

 There is exactly one item of each type of item available (I.e.: you can pack an item (you will then one of that item) or you can do not pack that item (you will then zero of that item)

• Given a knapsack with capacity W:

• Problem:

• Pack as many items into the knapsack such that the total value of the items packed is maximized

Note: you cannot exceed the capacity of the knapsack !

• Formulating the 0,1-knapsack problem as a constained optimization problem:

• Define:

 x1 = # items of type 1 packed into the knapsack x2 = # items of type 2 packed into the knapsack ... xN = # items of type N packed into the knapsack

• Then:

• Total weight of item packed = w1 × x1   +   w2 × x2   +   ...   +   wN × xN

 wi = weight of an item i

• Total value of item packed = v1 × x1   +   v2 × x2   +   ...   +   vN × xN

 vi = value of an item i

• The constrainted optimilization formulation of the 0,1 knapsack problem:

 ``` max: v1×x1 + v2×x2 + ... + vN×xN s.t.: w1×x1 + w2×x2 + ... + wN×xN ≤ W x1 = 0, 1, 2, ..., c1 x2 = 0, 1, 2, ..., c2 ... xN = 0, 1, 2, ..., cN ```

• Problem description: unbounded knapsack

• Problem description:

• Given N types of items:

 ...

• The item type i has a weight wi

• The item type i has a value vi

Unbounded referes to:

 There are an unbounded (= infinite) number of each type of item available (I.e.: unbounded supply)

• Given a knapsack with capacity W:

• Problem:

• Pack as many items into the knapsack such that the total value of the items packed is maximized

Note: you cannot exceed the capacity of the knapsack !

• The unbounded knapsack problem as a constained optimization problem:

• Define:

 x1 = # items of type 1 packed into the knapsack x2 = # items of type 2 packed into the knapsack ... xN = # items of type N packed into the knapsack

• Then:

 Total weight of item packed = w1 × x1   +   w2 × x2   +   ...   +   wN × xN          Total value of item packed = v1 × x1   +   v2 × x2   +   ...   +   vN × xN

• The unconstrained knapsack problem:

 ``` max: v1×x1 + v2×x2 + ... + vN×xN s.t.: w1×x1 + w2×x2 + ... + wN×xN ≤ W ```

• The bounded knapsack problem

• The bounded knapsack problem

• The bounded knapsack problem:

• The bounded knapsack problem (UKP) places an general upper bound on the number of copies of each kind of item:

 ``` x1 ≤ C1 x2 ≤ C2 ... xN ≤ CN ```

• The Mathematical formulation of the bounded knapsack problem :

 ``` max: v1×x1 + v2×x2 + ... + vN×xN s.t.: w1×x1 + w2×x2 + ... + wN×xN ≤ W x1 = 0, 1, 2, ..., c1 x2 = 0, 1, 2, ..., c2 ... xN = 0, 1, 2, ..., cN ```

• The unbounded knapsack problem is an example of:

 integer programming problem

that we will learn to solve later (when we talk about the branch and bound technique).