Postscript version of these notes.

STAT 380 Lecture 18

Summary of Markov Process Results

• Chapman-Kolmogorov equations:

  

• Exponential holding times: starting from state i time, , until process leaves i has exponential distribution, rate denoted .
• Sequence of states visited, is Markov chain - transition matrix has . Y sometimes called skeleton.
• Communicating classes defined for skeleton chain. Usually assume chain has 1 communicating class.
• Periodicity irrelevant because of continuity of exponential distribution.

• Instantaneous transition rates from i to j:

  

• Kolmogorov backward equations:

  

• Kolmogorov forward equations:

  

• For strongly recurrent chains with a single communicating class:

  

• Stationary initial probabilities satisfy:

  

• Transition probabilities given by

  

  where has entries