Markov and Diffusion Processes

Chapter 5 investigates Markov processes in some depth. The first section studies various aspects of the transition measures p(s,x,t,A) of general stochastic processes, defined μ-a.e. in terms of the conditional probabilities μ[X t −1 (A |σ(X s )]. The goal of this investigation is to establish that there is a version of this transition measure for which p(s,x,t,⋅) is a probability measure on B(ℝ d ) for all choices of parameters, and that this version can be well-defined by μ[X t −1 (A)|X s =x] almost everywhere. The final result is established following a development on conditional distributions. The next section then focuses on the transition measures of Markov processes, developing a number of their special properties, including and following from the Chapman-Kolmogorov equation. Since every Markov process has a transition measure with certain identifiable properties, the next section investigates the reverse implication. That is, starting with a special transition measure, to be called a Markov transition measure, a probability space and Markov process with the given transition measure can be derived with the aid of Kolmogorov's existence theorem. Further, subject to additional requirements on this transition measure, there exists a continuous version of this Markov process as was the case for Brownian motion in Chapter 1 . Following these developments, a more general definition of a Markov process is introduced that replaces the natural sigma algebras σ(X t∈[0,s] ) with a more general collection, called a filtration on ( S,σ(S),μ). One result from this investigation proves that every stochastic process with independent increments is a Markov process, and this provides a simple proof that d-dimensional Brownian motion is Markov. In this general setting, the effect of changing this filtration, or considering versions of the original Markov process, are investigated. The final section then introduces diffusion processes, a special class of Markov processes with smoother transition measures. These processes play an important role in the Book IX study of stochastic differential equations (SDEs).