### Appying what we have learned to perform analysis on program loops

• Example 1 loop analysis

• Consider the following nested loop:

 ``` for ( i = 1; i <= N; i++ ) for ( j = i; j > 0; j-- ) do_marker(); ```

Find the running time Cn of the above loop when N = n.

(The running time is the number of times that the operation do_marker() is executed).

• Solution 1: direct approach

Count the number of iterations when N=n

 ``` for ( i = 1; i <= N; i++ ) for ( j = i; j > 0; j-- ) .... i = 1 2 3 4 ..... n ---------------------------------------------------------------- j = | 1 2 3 4 ..... n | 1 2 3 ..... n-1 | 1 2 ..... n-2 V 1 ... ... 2 1 ----------------------------------------------------------------- #iters = 1 2 3 4 .... n ```

Running time:

 ``` Cn = 1 + 2 + 3 + .... + (n-1) + n = n(n+1)/2 (See: click here) ```

• Solution 2: Set up the recurrence relation...

Because the outer loop variable changes as follows:

 ``` for ( i = 1; i <= N; i++ ) ... i is increased by 1 ```

we establish a relationship between Cn and Cn+1

Define:

 Cn = # times do_marker() is executed when N=n      Cn+1 = # times do_marker() is executed when N=n+1

Find relationship between Cn and Cn+1 by counting the number of executions (use "loop visualization"):

 ``` for ( i = 1; i <= N; i++ ) for ( j = i; j > 0; j-- ) .... i = 1 2 3 4 ..... n n+1 ---------------------------------------------------------------- j = | 1 2 3 4 n n+1 | 1 2 3 n-1 n | 1 2 n-2 n-1 V 1 ... ... ... ... 2 3 1 2 1 ```

The red indices are the values that i and j take on when N = n

The darkmagenta and the red indices are the values that i and j take on when N = n+1

• From the diagram above, we see that:

 Cn+1 = Cn + (n + 1)

(There are n+1 additional darkmagenta indices)

• Initial condition

 ``` Let N = 1 for ( i = 1; i <= N; i++ ) for ( j = i; j > 0; j-- ) .... i runs from 1 to 1 j runs from 1 downto 1 ```

Therefore:

 C1 = 1

• The complete recurrence relation:

 Cn+1 = Cn + (n + 1) C1 = 1

Or: (substitute n ==> n-1)

 Cn = Cn-1 + n C1 = 1

• Solution:

 ``` Cn = Cn-1 + n Subst: n ==> n-1 ==> Cn-1 = Cn-2 + (n-1) ==> Cn = Cn-2 + (n-1) + n ==> Cn = Cn-3 + (n-2) + (n-1) + n .... ==> Cn = C1 + 2 + 3 + ... + (n-2) + (n-1) + n C1 = 1 ==> Cn = 1 + 2 + 3 + ... + (n-2) + (n-1) + n ==> Cn = n(n+1)/2 (See: click here) ```

• Example 2 loop analysis

• Consider the following loop:

 ``` for ( i = 1; i < N; i = 2*i ) do_marker(); ```

Find the running time Cn of the above loop when N = n.

(The running time is the number of times that the operation do_marker() is executed).

• Solution 1: direct approach

Count the number of iterations when N=n

 ``` for ( i = 1; i < N; i = 2*i ) do_marker(); i = 1 2 4 ..... 2k (n) 2k+1 ---------------------------------------------------------------- #iters = k ```

(k is the integer such that: 2k < n <= 2k+1 )

Running time:

 ``` Cn = k n ~= 2k <=> k ~= lg(n) = lg(n) Solution: Cn = lg(n) ```

• Solution 2: Set up the recurrence relation...

Because the outer loop variable changes as follows:

 ``` for ( i = 1; i < N; i = 2*i ) ... i is doubled after one iteration ```

we establish a relationship between Cn and C2n

Define:

 Cn = # times do_marker() is executed when N = n C2n = # times do_marker() is executed when N = 2n

Find relationship between Cn and C2n by counting the number of executions (use "loop visualization"):

 ``` for ( i = 1; i < N; i = 2*i ) do_marker(); i = 1 2 4 ..... n 2n ```

The red indices are the values that i takes on when N = n

The darkmagenta and the red indices are the values that i takes on when N = 2n

• From the diagram above, we see that:

 C2n = Cn + 1

(There is 1 additional darkmagenta index)

• Initial condition

 ``` Let N = 1 for ( i = 1; i < N; i = 2*i ) ... 1 < 1 will fail immediately ```

Therefore:

 C1 = 0

• The complete recurrence relation:

 C2n = Cn + 1 C1 = 0

Or: (substitute n ==> n/2)

 Cn = Cn/2 + 1 C1 = 0

• Solution:

 ``` Cn = Cn/2 + 1 Substitute: n = 2k ==> C2k = C2k-1 + 1 Subst: k ==> k-1 ==> C2k-1 = C2k-2 + 1 ==> C2k = C2k-2 + 1 + 1 ==> C2k = C2k-3 + 1 + 1 + 1 ==> C2k = C2k-4 + 1 + 1 + 1 + 1 .... ==> C2k = C2k-k + k(1) <==> C2k = C1 + k(1) C1 = 0 ==> C2k = k Substitute: n = 2k <==> k = lg(n) ==> Cn = lg(n) ```

• Example 3 loop analysis

• Consider the following nested loop:

 ``` for ( i = 1; i <= N; i++ ) for ( j = i; j > 0; j = j/2 ) do_marker(); ```

Find the running time Cn of the above loop when N = n.

(The running time is the number of times that the operation do_marker() is executed).

• Solution 1: direct approach

Count the number of iterations when N=n

 ``` for ( i = 1; i <= N; i++ ) for ( j = i; j > 0; j = j/2 ) .... i = 1 2 3 4 5 6 7 8 9 10 ... ---------------------------------------------------------------- j = | 1 2 3 4 5 6 7 8 9 10 | 1 1 2 2 2 2 4 4 4 | 1 1 1 1 2 2 2 V 1 1 1 ----------------------------------------------------------------- #iters = 1 2 2 3 3 3 3 4 4 4 ... ```

Running time:

 There is definitely a pattern.... However, the pattern is not easily recognizable This is a case where we would only use the recurrence relation technique to find the solution !

• Solution 2: Set up the recurrence relation...

Because the outer loop variable changes as follows:

 ``` for ( i = 1; i <= N; i++ ) ... ```

we establish a relationship between Cn and Cn+1

Define:

 Cn = # times do_marker() is executed when N=n Cn+1 = # times do_marker() is executed when N=n+1

Find relationship between Cn and Cn+1 by counting the number of executions (use "loop visualization"):

 ``` for ( i = 1; i <= N; i++ ) for ( j = i; j > 0; j = j/2 ) .... i = 1 2 3 4 5 6 7 8 ... n n+1 ---------------------------------------------------------------- j = | 1 2 3 4 5 6 7 8 ... n n+1 | 1 1 2 2 2 2 4 ... n/2 (n+1)/2 | 1 1 1 1 2 ... n/4 (n+1)/4 V 1 ... n/8 (n+1)/8 n/16 (n+1)/16 ... ... 1 1 ```

The red indices are the values that i and j take on when N = n

The darkmagenta and the red indices are the values that i and j take on when N = n+1

• From the diagram above, we see that:

 Cn+1 = Cn + lg(n + 1)

(There are lg(n+1) additional darkmagenta indices)

• Initial condition

 ``` Let N = 1 for ( i = 1; i <= N; i++ ) for ( j = i; j > 0; j = j/2 ) .... i runs from 1 to 1 j runs from 1 downto 1 ```

Therefore:

 C1 = 1

• The complete recurrence relation:

 Cn+1 = Cn + lg(n + 1) C1 = 1

Or: (substitute n ==> n-1)

 Cn = Cn-1 + lg(n) C1 = 1

• Solution:

 ``` Cn = Cn-1 + lg(n) Subst: n ==> n-1 ==> Cn-1 = Cn-2 + lg(n-1) ==> Cn = Cn-2 + lg(n-1) + lg(n) ==> Cn = Cn-3 + lg(n-2) + lg(n-1) + lg(n) .... ==> Cn = C1 + lg(2) + lg(3) + ... + lg(n-2) + lg(n-1) + lg(n) C1 = 1 ==> Cn = 1 + lg(2) + lg(3) + ... + lg(n-2) + lg(n-1) + lg(n) ==> Cn = 1 + lg(2.3.4.(n-2).(n-1).n) ==> Cn = 1 + lg(1.2.3.4.(n-2).(n-1).n) <==> Cn = 1 + lg(n!) Approximate: n! ==> (n/e)n ( click here) ==> Cn = 1 + lg((n/e)n) <==> Cn = 1 + n×lg((n/e)) ==> Cn ~= n×lg((n/e)) ```

• Example 4 loop analysis

• Consider the following nested loop:

 ``` for ( i = 1; i < N; i++ ) for ( j = 0; j < N; j = j + i ) do_marker(); ```

Find the running time Cn of the above loop when N = n.

(The running time is the number of times that the operation do_marker() is executed).

• Solution 1: direct approach

Count the number of iterations when N=n

 ``` for ( i = 1; i < N; i++ ) for ( j = 0; j < N; j = j + i ) do_marker(); i = 1 2 3 4 5 ... n ---------------------------------------------------------------- j = | 0 0 0 0 0 ... 0 | 1 2 3 4 5 ... n | 2 4 6 8 10 ... V 3 6 9 12 15 ... 4 8 12 16 20 ... .. .. .. .. .. .. .. .. .. n .. .. .. .. .. .. .. n .. .. .. .. .. n .. .. .. n .. n ----------------------------------------------------------------- #iters = n n/2 n/3 n/4 n/5 ... n/n + + + + + ... + 1 1 1 1 1 ... 1 ```

Running time:

 ``` Cn = (n + n/2 + n/3 + ... + n/n) + n(1) = n (1 + 1/2 + 1/3 + ... + 1/n) + n (1 + 1/2 + ... + 1/n) ~= ln(n) + 0.6 (See: click here) ~= n×( ln(n) + 0.6 ) + n ```

• Solution 2: Set up the recurrence relation...

Because the outer loop variable changes as follows:

 ``` for ( i = 1; i < N; i++ ) ... i is increased by 1 ```

we establish a relationship between Cn and Cn+1

Define:

 Cn = # times do_marker() is executed when N=n      Cn+1 = # times do_marker() is executed when N=n+1

However, when we try to find the relationship between Cn and Cn+1, we get in a bit of trouble:

 ``` for ( i = 1; i < N; i++ ) for ( j = 0; j < N; j = j + i ) do_marker(); i = 1 2 3 4 5 ... n n+1 ---------------------------------------------------------------- j = | 0 0 0 0 0 ... 0 0 | 1 2 3 4 5 ... n n+1 | 2 4 6 8 10 ... ? V 3 6 9 12 15 ... 4 8 12 16 20 ... .. .. .. .. .. .. .. .. .. n .. .. .. .. ? .. .. .. n .. .. .. ? .. .. n .. .. ? .. n .. ? n n+1 ```

The red indices are the values that i and j take on when N = n

The darkmagenta and the red indices are the values that i and j take on when N = n+1

 The ? indicate the fact that we do not know if there is an additional iteration or not

• We cannot use the recursive relation technique to find the running time for this problem....

• Example 5 loop analysis

• Consider the following unusual loop:

 ``` i = 0; sum = 0; while ( sum < N ) { do_marker(); sum = sum + i; i++; } ```

Find the running time Cn of the above loop when N = n.

(The running time is the number of times that the operation do_marker() is executed).

• The cumulative loop index

This is the first example of a cumulative loop index:

 ``` i = 0; // i is the running index sum = 0; // sum is the cumulatibe index while ( sum < N ) { do_marker(); sum = sum + i; // sum tallies a total i++; // i runs: 0, 1, 2, 3, ... } ```

• Solution: use the direct approach

Count the number of iterations when N=n

You need to keep track of the running index as well as the cumulative index.

 ``` i = 0; // i is the running index sum = 0; // sum is the cumulatibe index while ( sum < N ) { do_marker(); sum = sum + i; // sum tallies a total i++; // i runs: 0, 1, 2, 3, ... } i = 0 1 2 3 4 5 6 ... k-1 k sum = 0 0 1 3 6 10 15 ... sk < n <= sk+k+1 ---------------------------------------------------------------- #iters = 1 1 1 1 1 1 1 1 ... 1 | | +-------------------------------------+ k+1 times ```

(sk is the sum 1 + 2 + 3 + ... + k-1)

Running time:

 ``` Cn = k+1 where: sk < n <= sk+1 The algorithm stops when: sk >= n To find k, we must find the smallest k such that: sk >= n <==> 1 + 2 + 3 + ... + (k-1) >= n (See: click here) <==> (k-1)k/2 >= n <==> (k-1)k >= 2n <==> k2 - k >= 2n Approximate: k2 >> k for large k ==> k2 >= 2n ----- ==> k >= \/ 2n Solution: ---- ---- Cn = \/ 2n + 1 ~= \/ 2n ```

• Example 6 loop analysis

• Consider the following nested loop:

 ``` for ( i = 1; i <= N; i=2*i ) for ( j = 0; j < i; j++ ) do_marker(); ```

Find the running time Cn of the above loop when N = n.

(The running time is the number of times that the operation do_marker() is executed).

• Solution 1: direct approach

Count the number of iterations when N=n

 ``` for ( i = 1; i <= N; i=2*i ) for ( j = 0; j < i; j++ ) do_marker(); i = 1 2 4 8 16 ... 2k <= n < 2k+1 ---------------------------------------------------------------- j = | 0 0 0 0 0 ... 0 | 1 1 1 1 ... 1 | 2 2 2 ... 2 V 3 3 3 ... 3 4 4 ... 4 5 5 ... 5 6 6 ... 6 7 7 ... 7 8 ... 8 9 ... 9 .. ... .. 15 ... 15 ... .. 2k-1 ----------------------------------------------------------------- #iters = 1 2 4 8 16 ... 2k ```

Running time:

 ``` Cn = 1 + 2 + 4 + ... + 2k where 2k <= n Cn = 1 + 2 + 22 + ... + 2k = (2k+1 - 1) / (2 - 1) (See: click here) = (2k+1 - 1) / (2 - 1) = (2k+1 - 1) The value k is the smallest integer such that: 2k <= n So we can approximate: 2k ~= n <==> 2k+1 ~= 2n Therefore: Cn = (2k+1 - 1) ~= 2n - 1 ~= 2n ```

• Solution 2: Set up the recurrence relation...

Because the outer loop variable changes as follows:

 ``` for ( i = 1; i <= N; i=2*i ) ... ```

we establish a relationship between Cn and C2n

Define:

 Cn = # times do_marker() is executed when N=n      C2n = # times do_marker() is executed when N=2n

Find relationship between Cn and C2n by counting the number of executions (use "loop visualization"):

 ``` for ( i = 1; i <= N; i=2*i ) for ( j = 0; j < i; j++ ) do_marker(); i = 1 2 4 8 16 ... n 2n ---------------------------------------------------------------- j = | 0 0 0 0 0 ... 0 0 | 1 1 1 1 ... 1 1 | 2 2 2 ... 2 2 V 3 3 3 ... 3 3 4 4 ... 4 4 5 5 ... 5 5 6 6 ... 6 6 7 7 ... 7 7 8 ... 8 8 9 ... 9 9 .. ... .. .. 15 ... 15 15 ... .. .. ~n ~n .. .. .. ~2n ```

The red indices are the values that i and j take on when N = n

The darkmagenta and the red indices are the values that i and j take on when N = 2n

• From the diagram above, we see that:

 C2n = Cn + 2n

(There are 2n additional darkmagenta indices)

• Initial condition

 ``` Let N = 1 for ( i = 1; i <= N; i=2*i ) for ( j = 0; j < i; j++ ) do_marker(); i runs from 1 to 1 j runs from 0 to 0 ```

Therefore:

 C1 = 1

• The complete recurrence relation:

 C2n = Cn + 2n C1 = 1

Or: (substitute n ==> n/2)

 Cn = Cn/2 + n C1 = 1

• Solution:

 ``` Cn = Cn/2 + n Subst: n = 2k n/2 = 2k-1 ==> C2k = C2k-1 + 2k Subst: C2k-1 = C2k-2 + 2k-1 ==> C2k = C2k-2 + 2k-1 + 2k ==> C2k = C2k-3 + 2k-2 + 2k-1 + 2k ==> C2k = C2k-4 + 2k-3 + 2k-2 + 2k-1 + 2k ... ==> C2k = C20 + 21 + 22 + 23 + ... + 2k-2 + 2k-1 + 2k C20 = C1 = 1 ==> C2k = 1 + 21 + 22 + 23 + ... + 2k-2 + 2k-1 + 2k <==> C2k = (2k+1 - 1)/(2 - 1) (See: click here) <==> C2k = 2k+1 - 1 Rewrite the expression back into Cn using: n = 2k C2k = 2k+1 - 1 <==> C2k = 2×2k - 1 <==> Cn = 2n - 1 Therefore: Cn ~= 2n (just like before) ```