Conditional Expectation

Discrete (Mass)

The conditional expectation of $X$ given $Y$ is:

$$\text{E}X\mid Y=y=\sum _i{x}_i\cdot f_{X|Y}(x_i\mid y)$$

Then, by the Law of Total Probability we get

$$\text{E}X=\sum _j\text{E}X\mid Y=y_j\cdot \mathbb{P}(Y=y_j)$$

Given $g(x):\mathbb{R}\to \mathbb{R}$ like $g(x)=x^2$, the conditional expectation is

$$\text{E}g(x)\mid Y=y=\sum _{x_i}g(x_i)f_{X|Y}(x_i\mid y)$$

Continuous (Density)

The conditional expectation of $X$ given $Y$ is:

$$\text{E}X\mid Y=y={\int }_{\mathbb{R}}x\cdot f_{X|Y}(x\mid y)dx$$

In which, by the Law of Total Probability we get

$$\begin{array}{rl}\text{E}X& ={\int }_{\mathbb{R}}\text{E}x\mid Y=y\cdot f_Y(y)dy\\ & ={\int }_{\mathbb{R}}{\int }_{\mathbb{R}}x\cdot f_{X|Y}(x\mid y)dxdy\end{array}$$

Intuitively, we are partitioning $X$ as parts of $Y$, and so finding the expectation of $X$ by conditioning on $Y$ is the same as simply finding its expectation.

Example

Suppose $X,Y$ have a joint density function

$$f_{X,Y}(x,y)=\frac{1}{y}{e}^{-\frac{x}{y}-y},\forall x,y>0$$

What is $f_{X|Y}(x\mid y)?$ So,

$$\begin{array}{rl}{f}_{X|Y}(x\mid y)& =\frac{f_{X,Y}(x,y)}{f_Y(y)}\end{array}$$

where

$$\begin{array}{rl}{f}_Y(y)& ={\int }_0^{\infty }{f}_{X,Y}(x,y)dx\\ & ={\int }_0^{\infty }\frac{1}{y}{e}^{-\frac{x}{y}-y}dx\\ & =\frac{1}{y}{e}^{-y}{\int }_0^{\infty }{e}^{\frac{-x}{y}dx}\\ & =\frac{1}{y}{e}^{-y}{\left[-y{e}^{\frac{-x}{y}}\right]}_0^{\infty }\\ & =e^{-y}\end{array}$$

In particular, this is just the exponential distribution. $Y\sim \text{Exp}(1)$. This allows us to find

$$f_{X|Y}(x\mid y)=\frac{1}{y}{e}^{-\frac{x}{y}}$$

where the conditional distribution of $X$ given $Y=y$ is $\text{Exp}\left(\frac{1}{y}\right)$.

We now can find the expectations

$$\text{E}X\mid Y=y=y\text{by mean of an exponential distribution}$$

and

$$\begin{array}{rl}\text{E}X& ={\int }_0^{\infty }\text{E}X\mid Y=y\cdot f_Y(y)dy\\ & ={\int }_0^{\infty }y{f}_Y(y)dy\\ & =\text{E}Y\\ & =1\end{array}$$

General Properties

• Let $X,Y,Z$ be RVs defined on the same probability space $(\Omega ,F,\mathbb{P})$.
• Let $g:\mathbb{R}\to \mathbb{R}$, $v:{\mathbb{R}}^2\to \mathbb{R}$ be such that

$$\text{E}|g(x)|<\infty \text{E}|v(x,y)|<\infty$$

• As a shorthand, we can use

$$\text{E}g(x)\mid Y=h(Y)$$

Linearity

We have that

$$\text{E}{c}_1{g}_1(X)+c_2{g}_2(Z)\mid Y=y=c_1\text{E}{g}_1(X)\mid Y=y+c_2\text{E}{g}_2(Z)\mid Y=y$$

Positivity

If $g(X)\ge 0$ then $\text{E}g(X)\mid Y=y\ge 0$.

Affix Random Variable

$$\text{E}v(X,Y)\mid Y=y=\text{E}v(X,y)\mid Y=y$$

Given that we know $Y=y$, we can convey that information to $v$.

Independence

If $X,Y$ are independent, then

$$\text{E}g(X)\mid Y=y=\text{E}g(X)$$

”Take out what is known”

$$\text{E}g(X)h(Y)\mid Y=y=h(y)\cdot \text{E}g(X)\mid Y=y$$

Iterated Expectations

$$\begin{array}{rl}\text{E}g(X)h(Y)& \\ (1)& =\int \text{E}g(X)h(Y)\mid Y=y\cdot f_Y(y)dy\\ (2)& ={\mathbb{E}}_Y[h(Y)\cdot \text{E}g(X)\mid Y]\end{array}$$

where $(1)$ follows from Law of Total Probability and $(2)$ follows from “Take out what is known”

This is the process of breaking down a joint expectation into conditional expectations and integrating over the distribution of the conditioning variable.