Stochastic Differential Equations

Stochastic Processes

Read Stochastic Process.

Random Walk on the Reals

We want to generalize stochastic processes to continuous time and continuous state spaces. Consider the random walk on $\mathbb{R}$.

Let $Z_0,\dots ,Z_n\sim \mathcal{N}(0,1)$ be Normal iid RVs and consider

$$X_{n+1}=X_n+\delta t^{\alpha }\cdot Z_n,n=0,1,2,\dots$$

and $X_0=0$. Recall this form from the continuum limit. Expanding this sequence yields:

• $X_1=\delta t^{\alpha }{Z}_1$
• $X_2=\delta t^{\alpha }(Z_1+Z_2)$
• $X_n={\sum }_{i=0}^{n-1}\delta t^{\alpha }{Z}_i$ Because the sum of normally distributed RVs is normal, $X_n$ is normally distributed. The Moments of this process are
• $\mathbb{E}\left[X_n\right]=0$
• $\text{Var}(X_n)=n\delta t^{2\alpha }$ This establishes the distribution

$$X_n\sim \mathcal{N}\left(0,n\delta t^{2\alpha }\right)$$

Continuum Limit of the Random Walk

To determine if there is a continuous limit $X(t)$ such that $X_n\to X(t)$ as $n\to \infty$, the observation time $t=n\delta t$ must remain fixed. We can analyze the variance under this limit:

$$\text{Var}(X_n)=n\delta t^{2\alpha }=t\cdot \delta t^{2\alpha -1}\to \{\begin{array}{ll}0& \alpha >\frac{1}{2}\\ \infty & \alpha <\frac{1}{2}\\ t& \alpha =\frac{1}{2}\end{array}$$

as $\delta t\to 0$. In fact, the continuum limit only exists for $\alpha =1/2$, i.e. there exists a finite, non-zero $X(t)$ such that $X_n\to X(t)$ as $n\to \infty ,\delta t\to 0$ with $n\cdot \delta t=t$ fixed.

We look at the variance because the mean provides no information. The discussion above is also non-rigorous. The continuum limit $X(t)$ is called Brownian motion/Wiener process.

Brownian Motion and SDE Formulation

• The continuum limit $X(t)$ is called Brownian Motion or Wiener process.
• Discretized form for $X_n\in \mathbb{R}$: $$X_{n+1}=X_n+\sqrt{\delta t}\cdot Z_n$$ for $Z_0,Z_1,\dots ,\sim \mathcal{N}(0,1)$ iid.
• The SDE notation for $X(t)\in \mathbb{R}$ is written as $$\underset{\text{increment of X}}{\underbrace{dx}}=\underset{\text{Brownian increment}}{\underbrace{dB}}$$
• We can think of SDEs in terms of discrete equations with $\delta t\to 0$.

Definition (Brownian Motion, Wiener Process)

Brownian Motion (BM) is a time-homogeneous, continuous-time, continuous state Markov Process. At Stochastic Process $B(t)\in \mathbb{R}$ for $t\ge 0$ is a Brownian Motion if:

1. $B(0)=0$ with probability $1$.
2. $B(t)-B(s)\sim N(0,t-s)$ for all $t\ge s\ge 0$. In particular, $B(t)\sim N(0,t)$.
3. For any $0=t_0\le t_1\le \cdots \le t_n$, the RVs $B(t_1)-B(t_0),\dots ,B(t_n)-B(t_{n-1})$ are independent. These are called independent increments
4. $B(t)$ is a continuous function for $t\ge 0$.

From $(2)$, $B(t+\delta t)-B(t)\sim N(0,\delta t)$ for $\delta t>0$. This implies that

$$B(t+\delta t)-B(t)=\sqrt{\delta t}\cdot Z$$

for $Z\sim \mathcal{N}(0,1)$. See Brownian Motion and SDE Formulation.

BM is continuous but nowhere differentiable. From above, we see that

$$\frac{B(t+\delta t)-B(t)}{\delta t}=\frac{\sqrt{\delta t}\cdot Z}{\delta t}=\frac{1}{\sqrt{\delta t}}\cdot Z$$

for $Z\sim \mathcal{N}(0,1)$.

Properties of Brownian Motion

• Since $B(t)$ for $t\ge 0$ is a Markov process, then for $0\le t_1\le \dots t_n\le t$, we have that $$\mathbb{P}(B(t)\le b\mid B(t_1),\dots ,B(t_n))=\mathbb{P}(B(t)\le b\mid B(t_n))$$
• From $(2)$, we can have $B(t)-B(0)\sim \mathcal{N}(0,t)$. This implies that $\mathbb{E}\left[B(t)\right]=0$ and $\mathbb{E}\left[B(t{)}^2\right]=t$, i.e. the mean and variance. We can write $B(t)=\sqrt{t}Z$ for $Z\sim \mathcal{N}(0,1)$.
• The following processes are BMs. It is sufficient to check $\tilde{B}(t)$ for all four properties above, if $B(t)$ is already a BM.
  • $\tilde{B}(t)=-B(t),t\ge 0$
  • $\tilde{B}(t)=B(t+t_0)-B(t_0)$ for any $t_0>0$
  • $\tilde{B}(t)=B(\lambda t)/\sqrt{\lambda }$ for any $\lambda >0$
• The transition probability density is denoted as $p(y,t\mid x,s)$ for $s<t$, where $$\begin{array}{rl}{\int }_a^{b}p(y,t\mid x,s)dy& =\mathbb{P}(a<B(t)<b\mid B(s)=x)\\ & =\mathbb{P}(a-x<B(t)-B(s)<b-x)\\ & =\mathbb{P}(a<\sqrt{t-s}\cdot Z+x<b)\end{array}$$ for $Z\sim \mathcal{N}(0,1)$. This implies that $p(y,t\mid x,s)$ is a PDF of Normal $\mathcal{N}(x,t-s)$, or $$=\frac{1}{\sqrt{2\pi (t-s)}}\exp \left(-\frac{(y-x{)}^2}{2(t-s)}\right)$$
• $B(t),B(s)$ are not independent. Indeed, their Covariance is as follows: $$\begin{array}{rl}\text{Cov}(B(t),B(s))& =\mathbb{E}\left[B(t)\cdot B(s)\right]-\mathbb{E}\left[B(t)\right]\cdot \mathbb{E}\left[B(s)\right]\\ & =\mathbb{E}\left[B(t)\cdot B(s)\right]-0\\ & =\mathbb{E}\left[[B(t)-B(s)]\cdot [B(s)-B(0)]+B(s{)}^2\right]\\ & =\mathbb{E}\left[B(t)-B(s)\right]\cdot \mathbb{E}\left[B(s)-B(0)\right]+\mathbb{E}\left[B(s{)}^2\right]\\ & =0+\mathbb{E}\left[B(s{)}^2\right]\\ & =\text{Var}(B(s))\\ & =s\end{array}$$ In particular, $\mathbb{E}\left[B(t)\cdot B(s)\right]=\min (s,t)$.

Example 1

We know $\mathbb{E}\left[B(3)\right]=0$ and $\mathbb{E}\left[B(3{)}^2\right]=3$. Then

$$\begin{array}{rl}\mathbb{P}(B(8)\le 4\mid B(2)=1)& =\mathbb{P}(B(8)-B(2)\le 3)\\ & =\mathbb{P}(\sqrt{6}Z\le 3)\\ & =\mathbb{P}\left(Z\le \sqrt{3/2}\right)\\ & =\frac{1}{\sqrt{2\pi }}{\int }_0^{\sqrt{3/2}}\exp \left(-\frac{1}{2}{x}^2\right)dx\\ & =\mathbf{\Phi }\left(\frac{3}{2}\right)\end{array}$$

relating how BM and the normal distribution.

Ornstein-Uhlenbeck Process (OU)

The Ornstein-Uhlenbeck (OU) process is defined by the stochastic differential equation:

$$dX=-\alpha Xdt+\sqrt{2D}dB$$

with initial condition $X(0)=x_0$, and $\alpha ,D,x_0\in \mathbb{R}$. Note that $D>0$.

Euler-Maruyama’s Method for OU Process

We want to simulate

$$dX=-\alpha Xdt+\sqrt{2D}dBX(0)=x_0$$

where $\alpha ,D,x_0\in \mathbb{R}$ and $D>0$ are constants. We can approximate $X(t)$ for $t\in [0,T]$ via

1. Set $T>0,n\in {\mathbb{Z}}_{>0},\delta t=T/n$.
2. Let $t_k=k\delta t$ with $k=0,1,2,\dots ,n$
3. For $i=0$ to $n-1$,
   1. Generate $Z\sim \mathcal{N}(0,1)$
   2. Set $X_{i+1}=X_i-\alpha X_i\delta t+\sqrt{2D\delta t}\cdot Z$
4. Output $X_0,X_1,\dots ,X_n$ We get that $X_i\approx X(t_i)$.

Moments of the OU Process

Evaluate the first moment by applying the expectation operator to the SDE.

$$dX=-\alpha Xdt+\sqrt{2D}dB$$

Taking the Expectation $\mathbb{E}$ yields the following. The second term is zero since the differential operator commutes with the expectation and property 2.

$$d\mathbb{E}[X]=-\alpha \mathbb{E}[X]dt$$

This translates to an ordinary differential equation (ODE) for the expectation:

$$\frac{d}{dt}\mathbb{E}[X]=-\alpha \mathbb{E}[X]$$

To solve this ODE, let $y(t)=\mathbb{E}[X(t)]$. The differential equation becomes:

$$\frac{dy}{dt}=-\alpha y$$

Solving this linear ODE with the initial condition $y(0)=x_0$ yields:

$$y=x_0{e}^{-\alpha t}$$

Substituting the expectation back provides the exact first moment:

$$\mathbb{E}[X]=x_0{e}^{-\alpha t}$$

How can we solve for higher moments, e.g. $d\left(\mathbb{E}\left[X^2\right]\right)=\mathbb{E}\left[d(X^2)\right]$? We use Ito Calculus and Stochastic Calculus.

General Stochastic Differential Equations

In $1D$, SDEs are equations of the form

$$dX=b(X,t)dt+\sigma (X,t)dBX(0)=x_0$$

where $B(t)$ is Brownian Motion, $X(t)\in \mathbb{R}$ and constants $b(X,t),\sigma (X,t)\in \mathbb{R}$. We use the following terminology:

• Drift Term: $b(X,t)dt$
• Diffusion Term: $\sigma (X,t)dt$
• Time Homogeneous: $b,\sigma$ is independent of $t$. Otherwise it is time “inhomogeneous”.
• Additive Noise: $\sigma$ independent of $X$.
• Multiplicative Noise: $\sigma$ depends on $X$.

Example 2

Let particle $X(t)$ be in a force field $F(X)$. Then

$$dX=F(X)dt+\sqrt{2D}dB$$

for some constant $D$ has drift (deterministic force) of $F(X)dt$ and a diffusion (random kicks) of $\sqrt{2D}dB$. This is time homogeneous (since $F(X)$ and $\sqrt{2D}$ do not depend on $t$) with additive noise ($\sqrt{2D}$ does not depend on $X$).

SDEs in Higher Dimensions

The vector $\underline{B}(t)=(B_1(t),\dots ,B_m(t))\in {\mathbb{R}}^m$ is Brownian Motion if all its components $B_i$ are BMs. An SDE for $\underline{X}(t)\in {\mathbb{R}}^m$ has the form

$$d\underline{X}=\underline{b}(\underline{X},t)dt+\sigma (\underline{X},t)d\underline{B}$$

where $\underline{b}(\underline{X},t)\in {\mathbb{R}}^m$ and in general, $\sigma (\underline{X},t)$ can be an $m\times m$ matrix. If $\sigma \propto I_m$ then we can treat it as a scalar.

Example 3 (Euler-Maruyama for 2D Brownian)

Suppose we had a Brownian particle on $2D$ with $\underline{X}(t)\in {\mathbb{R}}^2$ modeled as

$$d\underline{X}=\sqrt{2D}d\underline{B}\underline{X}(0)=\underline{0}\in {\mathbb{R}}^2$$

We can use the Euler-Maruyama method to discretize and simulate this.

1. Let $T,\delta t>0,n=T/(\delta t)$ with $t_k=k\delta t$ for $k=0,\dots ,n$.
2. For $i=0$ to $n-1$,
   1. Generate $Z_1,Z_2\sim \mathcal{N}(0,1)$ iid.
   2. ${\underline{X}}_{i+1}={\underline{X}}_i+\sqrt{\delta t}\left(\genfrac{}{}{0ex}{}{Z_1}{Z_2}\right)$
3. Output $\underline{X}$. Note that the second term is a 2-column vector with $Z_1,Z_2$, not the combinatoric definition.

Example 4 (Active Brownian Particle)

Suppose we had an active Brownian particle with

• speed $v$
• direction ${\underline{n}}_{\delta }=(\cos \theta ,\sin \theta )$
• $\theta$ diffuses with the SDEs

$$\begin{array}{rl}d\underline{X}& =u{\underline{n}}_{\theta }dt+\sqrt{2D_T}d{\underline{B}}_T\\ d\theta & =\sqrt{2D_K}d{B}_K\end{array}$$

The following is a diagram representing this system.

$"\\begin{document}\n\\begin{tikzpicture}\n$

source code

where

• $\underline{X}(t)\in {\mathbb{R}}^2$ and $\theta (t)\in \mathbb{R}$
• constants $v,D_K,D_T\in \mathbb{R}$
• ${\underline{B}}_T(t)\in {\mathbb{R}}^2$ is a 2D BM
• $B_K(t)\in \mathbb{R}$ is a 1D BM

We can use the Euler-Maruyama Method to simulate this.

1. Choose $T,\delta t>0,n=T/(\delta t)$ and $\underline{(}X{)}_0,{\theta }_0$.
2. For $i=0$ to $n-1$,
   1. Generate $Z_1,Z_2,Z_3\sim \mathcal{N}(0,1)$ iid.
   2. Set

$${\underline{X}}_{i+1}={\underline{X}}_i+v\left(\genfrac{}{}{0ex}{}{\cos {\theta }_i}{\sin {\theta }_i}\right)\delta t+\sqrt{2D_T\delta t}\left(\genfrac{}{}{0ex}{}{Z_1}{Z_2}\right)$$

3. Set 

$${\theta }_{i+1}={\theta }_i+\sqrt{2D_K\delta t}\cdot Z_3$$

3. Output $\underline{X},\theta$.

Applying Units

Suppose $X(t)$ has units length $[X]\sim L$. Given

$$dX=u(X)dt+\sqrt{2D}dB$$

we get

• $[x]\sim L$, length
• $[t]\sim T$ for time
• $[u]\sim L/T$, for velocity
• $[B]\sim \sqrt{T}$ since $\mathbb{E}\left[B(t{)}^2\right]=t$ from properties. This represents stochastic fluctuations.
• $\sqrt{2D}\sim L/\sqrt{T}$ and $[D]\sim L^2/T$. $[D]$ represents rate of spreading. Recall this is the diffusion constant.