Conditional Probability

Discrete (Mass)

Let $X,Y$ be two discrete random variables taking values in

$$\begin{array}{r}\{x_1,x_2,\cdots \}\\ \{y_1,y_2,\cdots \}\end{array}$$

and let $A_i=\{Y=y_i\}$. The conditional probability mass function of $X$ given $Y$ is

$$p_{X\mid Y}(x_j\mid y_i):=\mathbb{P}(X=x_j\mid Y=y_i)=\frac{{\mathbb{P}}_{X,Y}(x_j,y_i)}{{\mathbb{P}}_Y(y_i)}$$

Note that

$${\mathbb{P}}_{X|Y}(x_j,y_i)\equiv \mathbb{P}(X=x_j\cap Y=y_i)$$

From this conditional PMF, we can derive the marginal PMF for $X$ a fixed $Y=y$ by:

$$\begin{array}{rl}{p}_{X|Y}(x\mid y)& =\frac{p_{XY}(x,y)}{p_{Y(y)}}\\ p_{XY}(x,y)& =p_{X|Y}(x\mid y)p_Y(y)\\ \mathbb{P}(X=x)& =\sum _{y=0}^{\infty }{p}_{X|Y}(x\mid y)p_Y(y)\end{array}$$

which is given by the Law of Total Probability:

Example 1

• Let $m\in \mathbb{N}$, $p,q\in (0,1)$
• Let $N\sim \text{Binom}(m,q)$
• Let $X\sim \text{Binom}(N,p)$

1. What is the marginal distribution of $X$?
2. What is $\text{E}X$?

We know that

$$P_N(k)=\left(\genfrac{}{}{0ex}{}{m}{k}\right)q^k(1-q{)}^{m-k},k\in \{0,1,\cdots ,m\}$$ $$P_{X|N}(j\mid k)=\left(\genfrac{}{}{0ex}{}{k}{j}\right)p^j(1-p{)}^{k-j},j\in \{0,1,\cdots ,k\}$$

Where

$$P_X(j)=\sum _{k=j}^m{P}_{X|N}(j\mid k)\cdot P_N(k)$$

which is equal to

$$\left(\genfrac{}{}{0ex}{}{m}{j}\right)(pq{)}^j(1-pq{)}^{m-j}$$

and $X\sim \text{Binom}(m,pq)$. Page 48 in PK.

Continuous (Density)

The continuous conditional $f$ of $X$ given $Y=y$ is

$$f_{X\mid Y}(x\mid y)=\frac{f_{X,Y}(x,y)}{f_Y(y)}$$

Using the probability measure:

$$\mathbb{P}(X=x\mid Y=y)=\frac{\mathbb{P}(X=x\cap Y=y)}{\mathbb{P}(Y=y)}$$