### Solving recurrence relations

• Solving recurrence relations with Series expansion

• What to do:

 Rewrite Cn-1, Cn-2, Cn-3, etc with the recurrence formula Gather the sum in such a form that you can discover a pattern Rewrite the recurrence relation until you reach the initial condition Find an expression for the serie sum

• Checking the solution

• It is important that you check the correctness of the solution whenever possible

• What to do to check the correctness::

 Substitute solution in all initial conditions and verify correctness Substitute solution in the recurrence relation and verify correctness

This cannot always be done...

It can be done if you have obtained the exact solution

If you have used approximation of simplification to obtain the solution, the answer cannot be checked... only "estimated" (the highest order terms will match, the lower order terms will not match)

• Example 1: Series expansion

• Solve:

 Cn = Cn-1 + 1     C1 = 1

• Algorithm that give rise to such a recurrence relation:

 ``` for ( i = 0; i < N; i++) do_marker(); i = 0 1 2 3 ... n-1 # do_marker(): 1 1 1 1 ... 1 | | +-----------------------+ amount of work done = Cn-1 i = 0 1 2 3 ... n-1 n # do_marker(): 1 1 1 1 ... 1 1 | | +----------------------------+ amount of work done = Cn Cn = Cn-1 + 1 ```

• Solution:

 ``` Cn = Cn-1 + 1 Rewrite using: n => n-1 Cn = Cn-1 + 1 Cn-1 = C(n-1)-1 + 1 Cn-1 = Cn-2 + 1 Cn = (Cn-1 + 1) + 1 = Cn-2 + 1 + 1 Rewrite using: Cn-2 = Cn-3 + 1 Cn = (Cn-3 + 1) + 1 + 1 = Cn-3 + 1 + 1 + 1 Rewrite using: Cn-3 = Cn-4 + 1 Cn = (Cn-4 + 1) + 1 + 1 + 1 = Cn-4 + 1 + 1 + 1 + 1 and so on... Discover pattern: Cn = Cn-4 + 1 + 1 + 1 + 1 ^^^^^^^^^^^^^ 4 1's Conclusion: Cn = Cn-k + 1 + 1 + ... + 1 ^^^^^^^^^^^^^^^ k 1's Go to an initial condition: C1 Cn = Cn-(n-1) + 1 + 1 + ... + 1 ^^^^^^^^^^^^^^^^ (n-1) 1's Cn = C1 + (n-1) = 1 + (n-1) = n Solution: Cn = n ```

• Check:

• Cn-1 = Cn + 1
• C1 = 1

Solution: Cn = n

 ``` C1 = 1 1 = 1 Check Cn = Cn-1 + 1 n = n - 1 + 1 n = n Check ```

• Example 2: Series expansion (2.1 in book)

• Solve:

 Cn = Cn-1 + n     C1 = 1

• Algorithm that give rise to such a recurrence relation:

 ``` Cn = Cn-1 + n ^^^^^ ^^^^^^ ^^^^^ Amount of work Amount of work Amount of work needed to solve needed to solve DONE to reduce a problem of a problem of problem size from size(n) size(n-1) n to n-1 ```

In other words:

 The algorithm examines all n (remaining) items (takes n steps) After one loop, the algorithm will remove one item from the input (resulting in n-1 remaining items) The algorithm will need to process the remaining n-1 items - incurring Cn-1 executing cost

Example of such type of algorithm: inserting sort (discussed later)

• Solution:

 ``` Cn = Cn-1 + n Rewrite using: n => n-1 Cn = Cn-1 + n Cn-1 = C(n-1)-1 + (n-1) Cn-1 = Cn-2 + (n-1) Cn = (Cn-2 + (n-1)) + n = Cn-2 + (n-1) + n Rewrite using: Cn-2 = Cn-3 + (n-2) Cn = (Cn-3 + (n-2)) + (n-1) + n = Cn-3 + (n-2) + (n-1) + n Rewrite using: Cn-3 = Cn-4 + (n-3) Cn = (Cn-4 + (n-3)) + (n-2) + (n-1) + n = Cn-4 + (n-3) + (n-2) + (n-1) + n and so on... Discover pattern: Cn = Cn-4 + (n-3) + (n-2) + (n-1) + n Cn = Cn-5 + (n-4) + (n-3) + (n-2) + (n-1) + n Cn = Cn-6 + (n-5) + (n-4) + (n-3) + (n-2) + (n-1) + n etc. Go to an initial condition: C1 Cn = C1 + 2 + 3 + ... + (n-3) + (n-2) + (n-1) + n Cn = C1 + 2 + ... + (n-1) + n = 1 + 2 + ... + (n-1) + n 1 + 2 + ... + n = n(n+1)/2 (See: click here) = n(n+1)/2 Solution: Cn = n(n+1)/2 ```

• Check:

• Cn = Cn-1 + n
• C1 = 1

Solution: Cn = n(n+1)/2

 ``` C1 = 1 1(1+1)/2 = 1 2/2 = 1 1 = 1 Check Cn = Cn-1 + n ? n(n+1)/2 = (n-1)(n-1+1)/2 + n ? n(n+1)/2 = (n-1)n/2 + n ? (n*n + n)/2 = (n*n - n)/2 + n ? n*n/2 + n/2 = n*n/2 - n/2 + n ? n*n/2 + n/2 = n*n/2 + n/2 Check ```

• Example 3: Series expansion

• Solve:

 Cn = Cn-1 + 2n - 1     C1 = 1

• Algorithm that give rise to such a recurrence relation:

 ``` for ( i = 0; i < N; i++ ) for ( j = 0; j < N; j++ ) do_marker(); // Marker operation i = 0 1 2 3 ... n-1 n j = 0 1 1 1 1 ... 1 1 j = 1 1 1 1 1 ... 1 1 j = 2 1 1 1 1 ... 1 1 .... .... j = n-1 1 1 1 1 ... 1 1 j = n 1 1 1 1 ... 1 1 ```

• Solution:

 ``` Cn = Cn-1 + 2n - 1 Rewrite using: n => n-1 Cn = Cn-1 + 2n - 1 Cn-1 = C(n-1)-1 + 2(n-1) - 1 Cn-1 = Cn-2 + 2(n-1) - 1 Cn = (Cn-2 + 2(n-1) - 1) + 2n - 1 = Cn-2 + (2(n-1)-1) + (2n-1) Rewrite using: Cn-2 = Cn-3 + 2(n-2) - 1 Cn = (Cn-3 + 2(n-2) - 1) + (2(n-1)-1) + (2n-1) = Cn-3 + (2(n-2)-1) + (2(n-1)-1) + (2n-1) Rewrite using: Cn-3 = Cn-4 + 2(n-3) - 1 Cn = (Cn-4 + 2(n-3) - 1) + (2(n-2)-1) + (2(n-1)-1) + (2n-1) = Cn-4 + (2(n-3)-1) + (2(n-2)-1) + (2(n-1)-1) + (2n-1) and so on... Discover the pattern: Cn = Cn-4 + (2(n-3)-1) + (2(n-2)-1) + (2(n-1)-1) + (2n-1) Cn = Cn-5 + (2(n-4)-1) + (2(n-3)-1) + (2(n-2)-1) + (2(n-1)-1) + (2n-1) and so on... Work to an initial condition: C1 Cn = C1 + (2(2)-1) + (2(3)-1) + ... + (2(n-2)-1) + (2(n-1)-1) + (2n-1) = C1 + [2(2) + 2(3) + ... + 2(n-1) + 2n] - [1 + 1 + ... + 1] ^^^^^^^^^^^^^^^^^ n-1 1's = C1 + 2 [2 + 3 + ... + 2(n-1) + 2n] - (n-1) = 1 + 2 [2 + 3 + ... + (n-1) + n] - (n-1) = 2 + 2 [2 + 3 + ... + (n-1) + n] - (n-1) - 1 = 2 [1 + 2 + 3 + ... + (n-1) + n] - n + 1 - 1 = 2 [1 + 2 + 3 + ... + (n-1) + n] - n 1 + 2 + ... + (n-1) + n = n(n+1)/2 = 2 [ n(n+1)/2 ] - n = n(n+1) - n = n2 + n - n = n2 Solution: Cn = n2 ```

• Example 4: Series expansion (2.2 in book)

• Solve:

 Cn = Cn/2 + 1     C1 = 1

• Algorithm that give rise to such a recurrence relation: binary search

 ``` Cn = Cn/2 + 1 ^^^^^ ^^^^^^ ^^^^^ Amount of work Amount of work Amount of work needed to solve needed to solve DONE to reduce a problem of a problem of problem size from size(n) size(n/2) n to n/2 ```

In other words:

 The algorithm performs one operation (takes 1 statement) After one operation, the algorithm will remove half of the items from the input (resulting in n/2 remaining items) The algorithm will need to process the remaining n/2 items - incurring Cn/2 executing cost

• Before we can apply the expansion technique, we need to rewrite the recurrence relation into the familiar form

This can be achieved using the following substitution:

 n = 2k

We have:

 ``` Cn = Cn/2 + 1 n = 2k ==> C2k = C2k/2 + 1 <==> C2k = C2k-1 + 1 C1 = 1 <==> C20 = 1 ```

• Solution:

 ``` C2k = C2k-1 + 1 Rewrite using: k => k-1 C2k = C2k-1 + 1 C2(k-1) = C2(k-1)-1 + 1 C2k-1 = C2k-2 + 1 C2k = (C2k-2 + 1) + 1 = C2k-2 + 1 + 1 Rewrite using: C2k-2 = C2k-3 + 1 C2k = (C2k-3 + 1) + 1 + 1 = C2k-3 + 1 + 1 + 1 Rewrite using: C2k-3 = C2k-4 + 1 C2k = (C2k-4 + 1) + 1 + 1 + 1 = C2k-4 + 1 + 1 + 1 + 1 and so on... Discover pattern: C2k = C2k-4 + 1 + 1 + 1 + 1 ^^^^^^^^^^^^^^^ 4 1's C2k = C2k-5 + 1 + 1 + 1 + 1 + 1 and so on... Work to the initial condition: C1 = C20 C2k = C2k-k + 1 + 1 + ... + 1 ^^^^^^^^^^^^^^^^ k 1's C2k = C1 + k = 1 + k = k + 1 Rewrite it back into Cn: n = 2k <=> k = lg(n) C2k = k - 1 ==> Cn = lg(n) + 1 Solution: Cn = lg(n) + 1 ```

• Check:

• Cn = Cn/2 + 1
• C1 = 1

Solution: Cn = lg(n) + 1

 ``` C1 = 1 ? lg(1) + 1 = 1 ? 0 + 1 = 1 Check Cn = Cn/2 + 1 lg(n) + 1 = lg(n/2) + 1 + 1 ? lg(n) + 1 = lg(n) - lg(2) + 1 + 1 ? lg(n) + 1 = lg(n) - 1 + 1 + 1 ? lg(n) + 1 = lg(n) + 1 Check ```

• Example 5: Series expansion (2.3 in book)

• Solve:

 Cn = Cn/2 + n     C1 = 0

• Action done by algorithm that give rise to such a recurrence relation:

 ``` Cn = Cn/2 + n ^^^^^ ^^^^^^ ^^^^^ Amount of work Amount of work Amount of work needed to solve needed to solve DONE to reduce a problem of a problem of problem size from size(n) size(n/2) n to n/2 ```

In other words:

 The algorithm performs n operation (takes n statement, most likely examining all n input items) After n operations, the algorithm will remove half of the items from the input (resulting in n/2 remaining items) The algorithm will need to process the remaining n/2 items - incurring Cn/2 executing cost

• Again, before we can apply the expansion technique, we need to rewrite the recurrence relation into the familiar form

This can be achieved using the following substitution:

 n = 2k

We have:

 ``` Cn = Cn/2 + n n = 2k ==> C2k = C2k/2 + 2k <==> C2k = C2k-1 + 2k C1 = 0 <==> C20 = 0 ```

• Solution:

 ``` C2k = C2k-1 + 2k Rewrite using: k => k-1 C2k = C2k-1 + 2k C2(k-1) = C2(k-1)-1 + 2k-1 C2k-1 = C2k-2 + 2k-1 C2k = (C2k-2 + 2k-1) + 2k = C2k-2 + 2k-1 + 2k Rewrite using: C2k-2 = C2k-3 + 2k-2 C2k = (C2k-3 + 2k-2) + 2k-1 + 2k = C2k-3 + 2k-2 + 2k-1 + 2k Rewrite using: C2k-3 = C2k-4 + 2k-3 C2k = (C2k-4 + 2k-3) + 2k-2 + 2k-1 + 2k = C2k-4 + 2k-3 + 2k-2 + 2k-1 + 2k and so on... Discover pattern: C2k = C2k-4 + 2k-3 + 2k-2 + 2k-1 + 2k ^^^^^^^^^^^^^^^^^^^^ C2k = C2k-5 + 2k-4 + 2k-3 + 2k-2 + 2k-1 + 2k and so on... Work to the initial condition: C1 = C20 C2k = C20 + 21 + 22 + ... + 2k-1 + 2k C2k = C1 + 21 + 22 + ... + 2k-1 + 2k = 0 + 21 + 22 + ... + 2k-1 + 2k = 1 + 21 + 22 + ... + 2k-1 + 2k - 1 1 - xk+1 1 + x1 + x2 + ... + xk-1 + xk = ---------- 1 - x 1 - 2k+1 = --------- - 1 1 - 2 2k+1 - 1 = --------- - 1 2 - 1 = 2k+1 - 1 - 1 C2k = 2k+1 - 2 Rewrite it back into Cn: n = 2k <=> 2n = 2k+1 C2k = 2k+1 - 2 ==> Cn = 2n - 2 Solution: Cn = 2n - 2 ```

• Check:

• Cn = Cn/2 + n
• C1 = 1

Solution: Cn = 2n - 2

 ``` C1 = 0 ? 2×1 - 2 = 0 ? 0 = 0 Check Cn = Cn/2 + n 2n - 2 = (2(n/2) - 2) + n ? 2n - 2 = n - 2 + n ? 2n - 2 = 2n - 2 Check ```

• Example 6: Series expansion (2.4 in book)

• Solve:

 Cn = 2 Cn/2 + n     C1 = 0

• Algorithm that give rise to such a recurrence relation: binary search

 ``` Cn = 2 Cn/2 + n ^^^^^ ^^^^^^^^ ^^^^^ Amount of work Amount of work Amount of work needed to solve needed to solve DONE to reduce a problem of two problems of problem size from size(n) size(n/2) n into 2 groups of n/2 items ```

In other words:

 The algorithm performs n operation (takes n statement, most likely examining all n input items) After n operations, the algorithm will split the input into 2 groups of n/2 items The algorithm will need to process the 2 groups of n/2 items - incurring 2×Cn/2 executing cost

Example algorithm: Quick Sort (discussed later)

• Again, before we can apply the expansion technique, we need to rewrite the recurrence relation into the familiar form

This can be achieved using the following substitution:

 n = 2k

We have:

 ``` Cn = 2Cn/2 + n n = 2k ==> C2k = 2C2k/2 + 2k <==> C2k = 2C2k-1 + 2k C1 = 0 <==> C20 = 0 ```

• Solution:

 ``` C2k = 2C2k-1 + 2k Rewrite using: k => k-1 C2k = 2C2k-1 + 2k C2(k-1) = 2C2(k-1)-1 + 2k-1 C2k-1 = 2C2k-2 + 2k-1 C2k = 2×(2C2k-2 + 2k-1) + 2k = 22×C2k-2 + 2×2k-1 + 2k = 22×C2k-2 + 2k + 2k Rewrite using: C2k-2 = 2C2k-3 + 2k-2 C2k = 22×(2C2k-3 + 2k-2) + 2k + 2k = 23×C2k-3 + 2k + 2k + 2k Rewrite using: C2k-3 = 2C2k-4 + 2k-3 C2k = 23×(C2k-4 + 2k-3) + 2k + 2k + 2k = 24×C2k-4 + 2k + 2k + 2k + 2k and so on... Discover pattern: C2k = 24×C2k-4 + 2k + 2k + 2k + 2k ^^^^^^^^^^^^^^^^ 4 terms C2k = 25×C2k-5 + 2k + 2k + 2k + 2k + 2k and so on... Work to the initial condition: C1 = C20 C2k = 2k×C2k-k + 2k + 2k + ... + 2k + 2k ^^^^^^^^^^^^^^^^^^^^^^ k terms C2k = 2k×C1 + k×2k = 0 + k×2k C2k = k×2k <-------- Rewrite it back into Cn: n = 2k <=> k = lg(n) C2k = k×2k ==> Cn = lg(n)×n Solution: Cn = n lg(n) ```

• Check:

• Cn = 2Cn/2 + n
• C1 = 0

Solution: Cn = n lg(n)

 ``` C1 = 0 ? 1 lg(1) = 0 ? 0 = 0 Check Cn = 2 Cn/2 + n n lg(n) = 2 ((n/2)×lg(n/2)) + n ? n lg(n) = 2×(n/2)×lg(n/2) + n ? n lg(n) = n×lg(n/2) + n ? n lg(n) = n×(lg(n) - 1) + n ? n lg(n) = n×lg(n) - n) + n ? n lg(n) = n lg(n) Check ```

• Example 7: Series expansion (2.5 in book)

• Solve:

 Cn = 2 Cn/2 + 1     C1 = 1

• Algorithm that give rise to such a recurrence relation: binary search

 ``` Cn = 2 Cn/2 + 1 ^^^^^ ^^^^^^^^ ^^^^^ Amount of work Amount of work Amount of work needed to solve needed to solve DONE to reduce a problem of two problems of problem size from size(n) size(n/2) n into 2 groups of n/2 items ```

In other words:

 The algorithm performs 1 (constant number) operation After this constant number of operations, the algorithm will split the input into 2 groups of n/2 items The algorithm will need to process the 2 groups of n/2 items - incurring 2×Cn/2 executing cost

• Again, before we can apply the expansion technique, we need to rewrite the recurrence relation into the familiar form

This can be achieved using the following substitution:

 n = 2k

We have:

 ``` Cn = 2Cn/2 + 1 n = 2k ==> C2k = 2C2k/2 + 1 <==> C2k = 2C2k-1 + 1 C1 = 1 <==> C20 = 1 ```

• Solution:

 ``` C2k = 2C2k-1 + 1 Rewrite using: k => k-1 C2k = 2C2k-1 + 1 C2(k-1) = 2C2(k-1)-1 + 1 C2k-1 = 2C2k-2 + 1 C2k = 2×(2C2k-2 + 1) + 1 = 22×C2k-2 + 2 + 1 Rewrite using: C2k-2 = 2C2k-3 + 1 C2k = 22×(2C2k-3 + 1) + 2 + 1 = 23×C2k-3 + 22 + 2 + 1 Rewrite using: C2k-3 = 2C2k-4 + 1 C2k = 23×(C2k-4 + 1) + 22 + 2 + 1 = 24×C2k-4 + 23 + 22 + 2 + 1 and so on... Discover pattern: C2k = 24×C2k-4 + 23 + 22 + 2 + 1 C2k = 25×C2k-5 + 24 + 23 + 22 + 2 + 1 and so on... Work to the initial condition: C1 = C20 C2k = 2k×C2k-k + 2k-1 + 2k-2 + ... + 22 + 2 + 1 C2k = 2k×C1 + 2k-1 + 2k-2 + ... + 22 + 2 + 1 = 2k + 2k-1 + 2k-2 + ... + 22 + 2 + 1 2k+1 - 1 C2k = ---------- (See: click here) 2 - 1 = 2k+1 - 1 Rewrite it back into Cn: n = 2k <=> 2k+1 = 2n C2k = 2k+1 - 1 ==> Cn = 2n - 1 Solution: Cn = 2n - 1 ```

• Check:

• Cn = 2Cn/2 + 1
• C1 = 0

Solution: Cn = 2n - 1

 ``` C1 = 1 ? 2×1 - 1 = 1 ? 1 = 1 Check Cn = 2 Cn/2 + 1 2n - 1 = 2 (2(n/2) - 1) + 1 ? 2n - 1 = 2 ( n - 1) + 1 ? 2n - 1 = 2n - 2 + 1 ? 2n - 1 = 2n - 1 Check ```