Kolmogorov Forward Equation

Example 1

Consider SDE for particle $X(t)\in \mathbb{R}$ with

$$dX=-V^{\prime }(X)dt+\sqrt{2D}dBD>0\text{ constant}$$

$V(X)$ is called a potential. The SDE represents “go in the direction of $-V^{\prime }(X)$ plus some noise”. The drift $-V^{\prime }(X)dt$ moves the particles to the nearest well (negative means it will decrease) and diffusion here means it will get random kicks.

$"\\begin{document}\n\\begin{tikzpicture}\n$

source code

Kolmogorov Forward Equation

Let $p(x,t)$ be the PDF of $X(t)$, so ${\lim }_{|x|\to \infty }p(x,t)=0$. What happens to $p(k,t)$ as $t\to \infty$? Recall Ito’s Lemma: for any function $f$,

$$df=(-V^{\prime }{f}^{\prime }+D{f}^{\prime \prime })dt+f^{\prime }\sqrt{2D}dB$$

Take the expectation,

$$\begin{array}{rl}d(\mathbb{E}\left[f(x)\right])& =\mathbb{E}\left[-V^{\prime }{f}^{\prime }+D{f}^{\prime \prime }\right]dt+\mathbb{E}\left[f^{\prime }\sqrt{2D}dB\right]\\ (\star )\frac{d}{dt}\mathbb{E}\left[f(X(t))\right]& =\mathbb{E}\left[-V^{\prime }{f}^{\prime }+D{f}^{\prime \prime }\right]\end{array}$$

If $f,V^{\prime }{f}^{\prime },D{f}^{\prime \prime }$ all have the same form, then we get an ODE which we can solve. E.g. if they’re all $\sim X^2$ then we would have an ODE for $y(t)=\mathbb{E}\left[X(t{)}^2\right]$.

Instead, we will use $p(x,t)$ to write down both sides of $(\star )$.

$$\begin{array}{rl}\frac{d}{dt}\mathbb{E}\left[f(X)\right]& =\frac{d}{dt}\left[{\int }_{-\infty }^{\infty }dxf(x)p(x,t)\right]\\ & ={\int }_{-\infty }^{\infty }dxf(x)\frac{\partial p}{\partial t}\end{array}$$

and

$$\mathbb{E}\left[-V^{\prime }{f}^{\prime }+D{f}^{\prime \prime }\right]={\int }_{-\infty }^{\infty }dx(-V^{\prime }(x)f^{\prime }(x)+D{f}^{\prime \prime }(x))\cdot p(x,t)$$

The goal is write this as an integral against $f$, $\int \boxed{}fdx$ and equate this with the LHS. We’ll do this by Integration By Parts.

Starting with the expectation on the RHS:

$$\begin{array}{rl}RHS& ={\int }_{-\infty }^{\infty }(-V^{\prime }{f}^{\prime }+D{f}^{\prime \prime })pdx\\ & =-{\int }_{-\infty }^{\infty }{V}^{\prime }p{f}^{\prime }dx+{\int }_{\infty }^{\infty }Dp{f}^{\prime \prime }dx\end{array}$$

Apply integration by parts to the first term:

$$-{\int }_{-\infty }^{\infty }(V^{\prime }p)f^{\prime }dx={\left[-V^{\prime }pf\right]}_{-\infty }^{\infty }+{\int }_{-\infty }^{\infty }(V^{\prime }p{)}^{\prime }fdx$$

Since $p(x,t)\to 0$ as $x\to \pm \infty$, the boundary term vanishes:

$$-{\int }_{-\infty }^{\infty }(V^{\prime }p)f^{\prime }dx={\int }_{-\infty }^{\infty }(V^{\prime }p{)}^{\prime }fdx$$

Apply integration by parts to the second term:

$${\int }_{-\infty }^{\infty }(Dp)f^{\prime \prime }dx={\left[Dp{f}^{\prime }\right]}_{-\infty }^{\infty }-{\int }_{-\infty }^{\infty }D{p}^{\prime }{f}^{\prime }dx$$

The boundary term vanishes again:

$${\int }_{-\infty }^{\infty }(Dp)f^{\prime \prime }dx=-{\int }_{-\infty }^{\infty }D{p}^{\prime }{f}^{\prime }dx$$

Substitute these back into the RHS equation:

$$RHS={\int }_{-\infty }^{\infty }(V^{\prime }p{)}^{\prime }fdx-{\int }_{-\infty }^{\infty }D{p}^{\prime }{f}^{\prime }dx$$

Apply integration by parts a second time to the remaining $f^{\prime }$ term (the second term):

$$-{\int }_{-\infty }^{\infty }(D{p}^{\prime })f^{\prime }dx={\left[-D{p}^{\prime }f\right]}_{-\infty }^{\infty }+{\int }_{-\infty }^{\infty }(D{p}^{\prime }{)}^{\prime }fdx$$

The boundary term vanishes, leaving (assuming constant $D$):

$${\int }_{-\infty }^{\infty }D{p}^{\prime \prime }fdx$$

Combine the terms to express the entire RHS as an integral against $f$:

$$RHS={\int }_{-\infty }^{\infty }\left((V^{\prime }p{)}^{\prime }+D{p}^{\prime \prime }\right)fdx$$

Equate the RHS with the LHS:

$${\int }_{-\infty }^{\infty }\frac{\partial p}{\partial t}fdx={\int }_{-\infty }^{\infty }\left(\frac{\partial }{\partial x}(V^{\prime }p)+D\frac{{\partial }^{2}p}{\partial x^2}\right)fdx$$

Since this holds for any (reasonable) function $f$, the integrands must be equal, yielding the Kolmogorov Forward Equation equation:

$$\frac{\partial p}{\partial t}=\frac{\partial }{\partial x}(V^{\prime }p)+D\frac{{\partial }^{2}p}{\partial x^2}$$

This is also called the Fokker-Planck Equation.

The reason why the boundary terms head to $0$ is because $p(x,t)\to 0$ as $x\to \pm \infty$. The idea is that since $p(x,t)$ is a PDF, ${\int }_{-\infty }^{\infty }p(x,t)dx=1$, and so $p(x,t)$ must taper off (i.e. go to $0$) as we integrate over this infinite range to ensure it is equal to $1$. Likewise, $\partial p/\partial x\to 0$.

For $V(X)=0$ we get

• SDE: $dX=\sqrt{2D}dB$
• FPE: $\frac{\partial p}{\partial t}=D\frac{{\partial }^{2}p}{\partial x^2}$

Stationary Distributions

The FPE works with the same SDE as before.

$$\frac{\partial p}{\partial t}=\frac{\partial }{\partial x}\left(V^{\prime }(x)p+D\frac{\partial p}{\partial x}\right)$$

where $p(x,t)$ is a PDF of $X(t)$. Suppose there is a stationary distribution

$$p(x,t)\to p_0(x)\text{ as }t\to \infty$$

Then $p_0(x)$ must satisfy the FPE. But

$$\frac{\partial p_0}{\partial t}=0(\text{stationary})$$

implying that

$$0=\frac{\partial }{\partial x}\underset{\text{constant}}{\underbrace{\left(V^{\prime }(x)p_0+D\frac{\partial p_0}{\partial x}\right)}}⟹V^{\prime }(x)p_0+D\frac{\partial p_0}{\partial x}=A$$

Typically $p_0,\frac{\partial p_0}{\partial x}\to 0$ as $|x|\to \infty$, so $A=0$. We get

$$\begin{array}{rl}{V}^{\prime }(x)p_0+D\frac{\partial p_0}{\partial x}& =0\\ V^{\prime }(x)+D\frac{1}{p_0}\frac{\partial p_0}{\partial x}& =0\\ \frac{\partial }{\partial x}\left(V(x)+D\log p_0\right)& =0\end{array}$$

The inner term is still constant. So,

$$\begin{array}{rl}V(x)+D\log p_0& =c\\ p_0(x)& =\frac{1}{Z}\exp \left(-\frac{1}{D}V(x)\right)\end{array}$$

where $Z$ is the normalization. (Recall this is intractable.)

Remarks:

• Boltzmann Distribution for energy $V(x)$ and temperature $D$.
• Common in statistical mechanics because it arises from SDE.
• Because $V(x)$ is negative in the exponent, the probability $p_0(x)$ will be highest where $V(x)$ is the lowest. The particle is most likely to be found at the very bottom of potential wells. We can see this in Example 1’s graph.

We want to ask what does the probability cloud look like after the system has been running for an infinitely long time. Eventually, the “spread” of the random noise $D$ will reach a perfect equilibrium. The probability cloud will stop changing shape. Mathematically, that is equivalent to setting the time derivative $\partial p_0/\partial t=0$.

Algorithm using Euler-Maruyama

We can use Euler-Maruyama for the SDE.

• $X_{i+1}=X_i+V^{\prime }(X_i)\delta t+\sqrt{2\delta t}\cdot Z_i$
• $Z_0,Z_1,\cdots ,\sim \mathcal{N}(0,1)$ iid.
• $X_n\sim f$ as $n\to \infty$.

This also works in higher dimensions:

$$d\underline{X}=-\nabla V(\underline{x})dt+\sqrt{2D}d\underline{B}$$

This SDE has a PDF

$$p(x,t)\to \frac{1}{Z}\exp \left(-\frac{1}{D}V(\underline{x})\right)$$

as $t\to \infty$. We can solve PDEs numerically by simulating it is a corresponding SDE. For example, we can use the OU process:

$$dX=-V^{\prime }(X)dt+\sqrt{2D}dBX(t)\in \mathbb{R}$$

has potential $V(X)=\frac{1}{2}\alpha X^2⟹V^{\prime }(X)=\alpha X$. Then, the PDF $p(x,t)$ has

$$\underset{t\to \infty }{\lim }p(x,t)\to p_0(x)=\frac{1}{Z}\exp \left(\frac{-\alpha x^2}{2D}\right)\sim \mathcal{N}\left(0,\frac{D}{\alpha }\right)$$

For general $V$, the local min $x_0$ gives $V^{\prime }(x_0)=0$. This implies that

$$V(x)\approx V(x_0)+\frac{1}{2}{V}^{\prime \prime }(x_0)(x-x_0{)}^2$$

for $x$ close to $x_0$ via Taylor Series. Thus,

$$X(t)\text{ approximately }\sim \mathcal{N}\left(x_0,\frac{D}{V^{\prime \prime }(x_0)}\right)$$

as $t\to \infty$. The hopping ability is

$$\sqrt{\frac{D}{V^{\prime \prime }(x_0)}}\sim |x_1-x_0|$$

where $x_1$ is the bottom of the second well. If $\sigma$ is smaller than the distance between the well, then the particle will be unable to jump to the second well. However, if $D$ is high enough (imagine this is the temperature), then the particle can jump to $x_1$.