### The Birth and Death Markov processes

• Birth and Death processes

• Definition:

• A birth/death Markov process is a Markov process where:

 ``` pi,i+1 = bi (birth) pi,i-1 = di (death) pi,i = 1 - bi - di pi,k = 0 for k ≤ i-2 or k ≥ i+2 ```

• The transition diagram of a birth/death process looks like this: In other words:

• The possible events in a birth/death Markov provess are:

 Exactly one birth Exactly one death

(The event that there is no birth and no death is the case that no event occurs)

• The Poisson Birth/Death process

• Definitions:

 A birth = an arrival (of a client) into the system death = a departure (of a client) from the system

• Poisson birth/death process:

 Ҏ[ an arrival occurs in time interval Δt ] = λ × Δt Ҏ[ a departure occurs in time interval Δt ] = μ × Δt λ = arrival rate (see: click here ) μ = departure rate

• State transition diagram for a Poisson birth/death process: Resulting state equations:

 ``` p0 = p0 (1 - λΔt) + p1 μΔt p1 = p0 λΔt + p1 (1 - λΔt - μΔt) + p2 μΔt p2 = p1 λΔt + p2 (1 - λΔt - μΔt) + p3 μΔt p3 = p2 λΔt + p3 (1 - λΔt - μΔt) + p4 μΔt ... Or: p0 λΔt = p1 μΔt p1(λΔt + μΔt) = p0 λΔt + p2 μΔt p2(λΔt + μΔt) = p1 λΔt + p3 μΔt p3(λΔt + μΔt) = p2 λΔt + p4 μΔt ... Or: p0 λ = p1 μ p1(λ + μ) = p0 λ + p2 μ p2(λ + μ) = p1 λ + p3 μ p3(λ + μ) = p2 λ + p4 μ ... Or: p0 λ = p1 μ p1 λ + p1 μ = p0 λ + p2 μ p2 λ + p2 μ = p1 λ + p3 μ p3 λ + p3 μ = p2 λ + p4 μ ... Or: p0 λ = p1 μ p1 λ = p2 μ p2 λ + p2 μ = p1 λ + p3 μ p3 λ + p3 μ = p2 λ + p4 μ ... Or: p0 λ = p1 μ p1 λ = p2 μ p2 λ = p3 μ p3 λ + p3 μ = p2 λ + p4 μ ... ```

And so on....

Final set of equations:

 ``` p0 λ = p1 μ p1 λ = p2 μ p2 λ = p3 μ p3 λ = p4 μ ... Or: p1 = λ/μ × p0 p2 = λ/μ × p1 p3 = λ/μ × p2 p4 = λ/μ × p3 ... Or: p1 = λ/μ × p0 .... (1) p2 = (λ/μ)2 × p0 .... (2) p3 = (λ/μ)3 × p0 .... (3) p4 = (λ/μ)4 × p0 .... (4) ... ```

We can express every pi, i = 1, 2, 3, ... in terms on p0

Unfortunately, we do not (yet) know the value of p0...

• We need one more equation to solve this system, which is:

 ``` p0 + p1 + p2 + p3 + ... = 1 .... (5) ```

Substituting pi in equation (5) and we get:

 ``` p0 + (λ/μ)1 × p0 + (λ/μ)2 × p0 + (λ/μ)3 × p0 + .... = 1 p0 × ( 1 + (λ/μ)1 + (λ/μ)2 + (λ/μ)3 + .... ) = 1 S = 1 + x1 + x2 + x3 + x4 + ... xS = x1 + x2 + x3 + x4 + ... - ------ ------------------------------------ (1-x)S = 1 Therefore: 1 1 + x1 + x2 + x3 + x4 + ... = ------- 1 - x 1 p0 × ------- = 1 1-(λ/μ) p0 = 1 - (λ/μ) = 1 - ρ ...... (6) ```

Notation:

 ρ = λ/μ

• Steady state probability distribution of a Poisson birth/death process with arrival rate λ and departure rate μ:

 ``` p0 = (λ/μ)0 × (1-(λ/μ)) p1 = (λ/μ)1 × (1-(λ/μ)) p2 = (λ/μ)2 × (1-(λ/μ)) p3 = (λ/μ)3 × (1-(λ/μ)) .... Or: p0 = ρ0 × (1 - ρ) p1 = ρ1 × (1 - ρ) p2 = ρ2 × (1 - ρ) p3 = ρ3 × (1 - ρ) .... Or: Ҏ[ k customers in system ] = pk = ρk × (1 - ρ) ```

• Solving Markov chain using rate transition diagram

• A popular method for finding the equilibrium (steady state) probability distribution of a Markov chain is using rate transition diagrams

• Rate transition diagram:

 A rate transition diagram is obtained by removing the transitions that goes from a state into the same state from a state transition diagram

• Example:

• State transition diagram: • State diagram without loops to itself: After normalization by dividing by Δt: • Note:

 λ = arrival rate of clients μ = departure rate of clients The weights on the arcs in the digram are the rates of arrival and departure Hence the name: rate transition diagram

• Setting up equilibrium equations using a rate transition diagram

• Algorithm:

• Find a cut in the Markov chain that divide the Markov chain into 2 disjoint pieces

• In the equilibrium state, the number of transitions from one side the cut to the other side must be equal to the reverse direction

Example: Equilibrium equation:

 ``` Flow from left to right: p2 × λ Flow from left to right: p3 × μ Equilibrium: p2 × λ = p3 × μ ```

• Complete example: equalibrium equations for the M/M/1 queing system

• Cut 1: Equilibrium equation:

 ``` Flow from left to right: λ × p0 Flow from right to left: μ × p1 Equilibrium: λ × p0 = μ × p1 ```

• Cut 2: Equilibrium equation:

 ``` Flow from left to right: λ × p1 Flow from right to left: μ × p2 Equilibrium: λ × p1 = μ × p2 ```

• And so on....

• Resulting set of equation for the equilibrium state:

 ``` p0 λ = p1 μ p1 λ = p2 μ p2 λ = p3 μ p3 λ = p4 μ ... ```

This is the same set of equation as before... (see: click here)