### Expressions that are helpful in analyzing running time of algorithms

• Motivation

• When analyzing the running time of computer algorithms, you will encounter some common expressions

• We will first study how to handle such expression here, so we can apply the techniques to the analysis when we encounter them.

• The log(N) function

• The following is an overview of the logarithmic function which you (should) have learned in high school

• ln(x): the Natural log function:

• ln(x) = the natural log function

• ln(x) is in fact equal to the area under the graph of 1/x between 1 and x

Example:

• log(x): the base-10 log function:

 ``` ln(x) log(x) = -------- ln(10) ```

• lg(x): the base-2 log function:

 ``` ln(x) lg(x) = -------- ln(2) ```

The base-2 log function is frequently used in the analysis of computer algorithms

• Properties of logarithmic functions:

 ``` ln(a.b) = ln(a) + ln(b) ln(a/b) = ln(a) - ln(b) ln(1/b) = -ln(b) ln(an) = n ln(a) log(a.b) = log(a) + log(b) log(a/b) = log(a) - log(b) log(1/b) = -log(b) log(an) = n log(a) lg(a.b) = lg(a) + lg(b) lg(a/b) = lg(a) - lg(b) lg(1/b) = -lg(b) lg(an) = n lg(a) ```

• Harmonic numbers

• The Nth Harmonic number HN is defined as:

 ``` 1 1 1 1 HN = 1 + --- + --- + --- + ... + --- 2 3 4 N ```

Examples:

 ``` H1 = 1 1 HN = 1 + --- = 1.5 2 1 1 HN = 1 + --- + --- = 1.8333333... 2 3 ```

• The following figure shows that the value of HN is very close to the value of log(N+1):

The value of HN is equal to the area of the squares from 1 to N

The value of ln(N) is equal to the area under the graph of f(x)=1/x from 1 to N+1

• For large value of N, we can use the following approximation formula for HN

 ``` 1 HN ~= ln(N) + γ + ---- 12N γ ~= 0.57721... (Euler's constant) ```

• Fibonacci numbers

• The Nth Fibonacci number FN (made famous by "the DaVinci Code") is defined as:

 ``` F0 = 1 F1 = 1 FN = FN-1 + FN-2 for N >= 2 ```

Examples:

 ``` F1 = 1 F2 = 1 F3 = F2 + F1 = 1 + 1 = 2 F4 = F3 + F2 = 2 + 1 = 3 F5 = F4 + F3 = 3 + 2 = 5 ```

• We can show that:

 ``` ---- FN+1 1 + \/ 5 ----- --> φ = ------------ FN 2 The proof goes as follows: FN+1 FN + FN-1 ----- = --------- FN FN FN+1 FN-1 <=> ----- = 1 + ------ FN FN Substitute: FN+1 xN = ----- FN 1 => xN = 1 + ------ xN-1 Approximate both xN and xN-1 with x (because they converge to the same value): 1 => x = 1 + ----- x <=> x2 = x + 1 --- 1 + \/ 5 => Solution: x = ----------- 2 ```

• For large value of N, we can use the following approximation formula for FN

 ``` φN FN ~= ------- (rounded to the nearest integer) --- \/ 5 φ ~= 1.61803... (the "Golden ratio" - See: click here) ```

• The Factorial function

• The Nth Factorial number N! is defined as:

 ``` 0! = 1 N! = 1 × 2 × 3 × .... × N for N >= 1 ```

Examples:

 ``` 0! = 1 1! = 1 2! = 1 × 2 = 2 3! = 1 × 2 × 3 = 6 4! = 1 × 2 × 3 × 4 = 24 ```

• For large value of N, we can use the following Sterling's formula to approximate the value of N!:

 ``` N +- -+ | N | /---------- N! ~= | ----- | / 2 π N | e | \/ +- -+ where: π ~= 3.1415926535... e ~= 2.718281828... (Base of the natural log function) ```

• The "order of growth" of various expressions

• The order of growth of HN is O(ln(N))

Proof:

• From the definition of O(f(N)) (see: click here ), we must find constants c0 and N0, such that:

 ``` HN < c0 ln(N) for all N > N0 ```

• For large N, we can approximate HN with:

 ``` 1 HN ~= ln(N) + γ + ---- 12N ```

• This means that we must find constants c0 and N0, such that:

 ``` 1 HN ~= ln(N) + γ + ---- < c0 ln(N) for all N > N0 12N ```

• A good value to try is: c0 = 2 (any value greater than 1 will work):

 ``` 1 ln(N) + γ + ---- < 2 ln(N) for all N > N0 12N 1 <=> γ + ---- < ln(N) 12N 1 <=> 0.6 + ---- < ln(N) 12N ```

• The inequality will be true if N is large enough; say N = 10:

 ``` 1 0.6 + ---- < ln(N) N = 10 12N 1 ==> 0.6 + ---- < ln(10) ln(10) ~= 2.3 120 ==> 0.6 + 0.08333 < 2.3 true ```

• Therefore: for c0 = 2 and N0 = 10, we have that:

 ``` 1 HN ~= ln(N) + γ + ---- < c0 ln(N) for all N > N0 12N In other words: 1 HN ~= ln(N) + γ + ---- < 2 ln(N) for all N > 10 12N ```

• The order of growth of FN is O(φN)

Proof:

• This is obvious from the fact that for large value of N:

 ``` φN FN ~= ------- --- \/ 5 ```

• Use the following value for c0 and N0:

 ``` c0 = 1 N0 = 1 ```

• See:

 ``` φN FN ~= ------- < 1 φN For N > 1 --- \/ 5 ```

• The order of growth of N! is O((N/e)N×sqrt(N))

Proof:

 The proof is similar to the one for FN

• The serie sum: 1 + 2 + 3 + .... + (n-1) + n

• A number of serie sums appear often in the running time analysis of computer algorithms.

One such serie sum is:

 ``` 1 + 2 + 3 + .... + (n-1) + n ```

• This serie sum can be reduced as follows:

 ``` 1 + 2 + 3 + .... + (n-2) + (n-1) + n Regroup the items as follows: (1+n) + (2+(n-1)) + (3+(n-3)) + .... Number of groups that you can form: n/2 Value of each group = (n+1) Total sum = n(n+1)/2 Therefore: 1 + 2 + 3 + .... + (n-2) + (n-1) + n = n(n+1)/2 ```

• The serie sum: 1 + x + x2 + x3 .... + xn

• Another serie sum that appears often in the running time analysis of computer algorithms is:

 ``` 1 + x + x2 + x3 .... + xn ```

• This serie sum can be reduced as follows:

 ``` S = 1 + x + x2 + x3 .... + xn Multiply both side by x: x S = x (1 + x + x2 + .... + xn) = x + x2 + x3 .... + xn+1 Subtract the 2 equations: S = 1 + x + x2 + x3 .... + xn x S = x + x2 + x3 .... + xn + xn+1 - (subtract) ----------------------------------------------------------- (1-x)S = 1 - xn+1 1 - xn+1 => S = -------- 1 - x Therefore: 1 - xn+1 1 + x + x2 + x3 .... + xn = -------- 1 - x ```