Chapman–Kolmogorov equation

In mathematics, specifically in the theory of Markovian stochastic processes in probability theory, the Chapman–Kolmogorov equation is an identity relating the joint probability distributions of different sets of coordinates on a stochastic process. The equation was derived independently by both the British mathematician Sydney Chapman and the Russian mathematician Andrey Kolmogorov.

Mathematical description

Suppose that { f_i } is an indexed collection of random variables, that is, a stochastic process. Let

p_{{i_{1},\ldots ,i_{n}}}(f_{1},\ldots ,f_{n})

be the joint probability density function of the values of the random variables f₁ to f_n. Then, the Chapman–Kolmogorov equation is

p_{{i_{1},\ldots ,i_{{n-1}}}}(f_{1},\ldots ,f_{{n-1}})=\int _{{-\infty }}^{{\infty }}p_{{i_{1},\ldots ,i_{n}}}(f_{1},\ldots ,f_{n})\,df_{n}

i.e. a straightforward marginalization over the nuisance variable.

(Note that we have not yet assumed anything about the temporal (or any other) ordering of the random variables — the above equation applies equally to the marginalization of any of them.)

Application to Time Dilated Markov chains

When the stochastic process under consideration is Markovian, the Chapman–Kolmogorov equation is equivalent to an identity on transition densities. In the Markov chain setting, one assumes that i₁ < ... < i_n. Then, because of the Markov property,

p_{{i_{1},\ldots ,i_{n}}}(f_{1},\ldots ,f_{n})=p_{{i_{1}}}(f_{1})p_{{i_{2};i_{1}}}(f_{2}\mid f_{1})\cdots p_{{i_{n};i_{{n-1}}}}(f_{n}\mid f_{{n-1}}),

where the conditional probability $p_{{i;j}}(f_{i}\mid f_{j})$ is the transition probability between the times $i>j$ . So, the Chapman–Kolmogorov equation takes the form

p_{{i_{3};i_{1}}}(f_{3}\mid f_{1})=\int _{{-\infty }}^{\infty }p_{{i_{3};i_{2}}}(f_{3}\mid f_{2})p_{{i_{2};i_{1}}}(f_{2}\mid f_{1})\,df_{2}.

Informally, this says that the probability of going from state 1 to state 3 can be found from the probabilities of going from 1 to an intermediate state 2 and then from 2 to 3, by adding up over all the possible intermediate states 2.

When the probability distribution on the state space of a Markov chain is discrete and the Markov chain is homogeneous, the Chapman–Kolmogorov equations can be expressed in terms of (possibly infinite-dimensional) matrix multiplication, thus:

P(t+s)=P(t)P(s)\,

where P(t) is the transition matrix of jump t, i.e., P(t) is the matrix such that entry (i,j) contains the probability of the chain moving from state i to state j in t steps.

As a corollary, it follows that to calculate the transition matrix of jump t, it is sufficient to raise the transition matrix of jump one to the power of t, that is

P(t)=P^{t}.\,

The differential form of the Chapman–Kolmogorov equation is known as master equation.

References

Weisstein, Eric W. "Chapman–Kolmogorov Equation". MathWorld.

Chapman–Kolmogorov equation

Mathematical description

Application to Time Dilated Markov chains

See also

References

Further reading