Continuous Time Markov ChainsΒΆ

Authors: Thomas J. Sargent and John Stachurski

This lecture series provides a short introduction to the fascinating field of continuous time Markov chains. It will, in time, be integrated into our QuantEcon lectures. Focus is shared between theory, applications and computation. Mathematical ideas are combined with computer code to help clarify and build intuition, as well as to bridge the gap between theory and applications. The presentation is relatively rigorous but the aim is towards applications rather than mathematical curiosities (which are plentiful, if one starts to look). Applications are mainly drawn from economics and operations research.

Solved exercises

There are many solved exercises and we recommend readers attempt all of them, or at least review the solutions.

Computer code

The code is written in Python and is accelerated through a combination of NumPy (vectorized code) and just-in-time compilation (via Numba). QuantEcon provides a fast-paced introduction to scientific computing with Python that covers these topics.

Background: Markov chains in discrete time

The lectures are well suited to those who have some knowledge of discrete time Markov chains and wish to learn more about their continuous time cousins. A suitable preliminary discussion of discrete time Markov chains can be found here.

Prerequisites: Probability and Analysis

Readers are assumed to be familiar with probability and a small amount of analysis. Later lectures, which deal with infinite state spaces, assume that require that readers are comfortable with the basics of linear analysis in Banach space.

The lectures are written using Jupyter Book.