Transformation of Random Variables

We want to define this more generally. What if we wanted to do some transformation on the RV $X$?

Lemma (Change of Variables)

Let $X$ be a RV $X\sim f_X$. Let $\alpha :\mathbb{R}\to \mathbb{R}$ a real-valued function that is continuously differentiable(i.e. $f_X^{\prime }$ is continuous) with ${\alpha }^{\prime }>0$ (so that $\alpha$ is strictly increasing, and therefore invertible). Let $Y=\alpha (x)$, a PDF. Then

$$f_Y(y)=\frac{f_X(x)}{{\alpha }^{\prime }(x)}(x={\alpha }^{-1}(y))$$

Proof:

Given $X,Y$ as RVs and $Y=\alpha (X)$, then if we take $y\in \mathbb{R}$ with $x={\alpha }^{-1}(y)$, then

$$\begin{array}{rlrl}{F}_Y(y)& =P(Y\le y)& & \\ & =P(\alpha (X)\le \alpha (x))& & \\ & =P(X\le x)& & (\star )\\ & =F_X(x)& & \end{array}$$

we have that $F_Y(y)=F_X(x)$. We can differentiate to get the PDF.

$$\begin{array}{rl}{F}_Y(y)& =F_X(x)\\ f_Y(y)\cdot {\alpha }^{\prime }(x)& =f_X(x)\end{array}$$

We work with the CDF because the events are nicer to manipulate arithmetically.

What if $\alpha$ is strictly decreasing, where ${\alpha }^{\prime }<0$? We get the same argument, but the step $(\star )$ means we can do

$$P(\alpha (X)\le \alpha (x))⟹P(x\le X)=1-F_X(x)$$

Again, by differentiation to get the PDFs,

$$\begin{array}{rl}{F}_Y^{\prime }(y)\frac{dy}{dx}& =-F_X^{\prime }(x)\\ f_Y(y){\alpha }^{\prime }(x)& =-f_X(x)\end{array}$$

and since ${\alpha }^{\prime }<0$, then both PDFs are positive (or rather, non-negative) then we are fine. In general, for $\alpha$ decreasing, then

$$f_Y(y)=\frac{f_X(x)}{|{\alpha }^{\prime }(x)|}$$

Remark (Change Must be Monotonic)

In particular, if $\alpha$ is not Monotonic, we cannot say anything about $Y$, since it would no longer be injective. For example, consider $Y=X^2$.

Example 1

Let $X\sim N(0,1)$ the standard Normal distribution and $Y=\alpha (x)=\sigma X+\mu$. The PDFs of $f_X,f_Y$ of $X,Y$ respectively, are related by

$$\begin{array}{rl}{f}_Y(y)& =\frac{f_X(x)}{{\alpha }^{\prime }(x)}\\ & =\frac{1}{\sigma }\frac{1}{\sqrt{2\pi }}\exp \left(-\frac{1}{2}{x}^2\right)\\ & =\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \left(-\frac{1}{2{\sigma }^2}(y-\mu {)}^2\right)\\ & ⟹Y\sim N(\mu ,{\sigma }^2)\end{array}$$

In particular, the second step is achieved be seeing that $X={\alpha }^{-1}(y)=\frac{y-\mu }{\sigma }$.

Transformation of Multivariate Random Variables

Let $\underline{X}=(X_1,\dots ,X_n)\in {\mathbb{R}}^n$ be a vector valued RV. The Joint Distribution of $\underline{X}$ is defined by

$$F_{\underline{X}}(\underline{x})=P(X_1\le x_1,\dots ,X_n\le x_n)\forall \underline{x}=(x_1,\dots ,x_n)\in {\mathbb{R}}^n$$

If we assume that the joint PDF exists, then it is defined as

$$f_{\underline{X}}:{\mathbb{R}}^n\to [0,\infty )$$

is related by

$$F_{\underline{X}}(\underline{x})={\int }_{-\infty }^{x_n}\cdots {\int }_{-\infty }^{x_1}{f}_{\underline{X}}(u_1,\dots ,u_n)d{u}_1\dots d{u}_n$$

---

Let $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ be an invertible¹ transformation with “sufficiently smooth”² components.

$$\begin{array}{rl}T:{\mathbb{R}}^n& \to {\mathbb{R}}^n\\ \underline{x}& \mapsto \underline{y}=T(\underline{x})\\ & \underline{x}=T^{-1}(\underline{y})\end{array}$$

Recall these are vector valued functions. The Jacobian for $T^{-1}$ is determined by

$$J(\underline{y})=\det \left(\frac{\partial x_i}{\partial y_j}\right)$$

where $T^{-1}$ maps a vector from the codomain to $\underline{x}$ is the domain. In particular, $T^{-1}(\underline{y})=\underline{x}$. This tells us that each component $x_i$ of $\underline{x}$ is defined as a function of $y_j$ in $\underline{y}$, the prerequisite for calculating the Jacobian.

---

Theorem (Multivariate Change of Variables)

Suppose $\underline{X}\in {\mathbb{R}}^n$ is a RV with joint PDF $f_{\underline{X}}$. Then the joint density of $\underline{Y}$ is given by

$$f_{\underline{Y}}(\underline{y})=f_{\underline{x}}\left(T^{-1}(\underline{y})\right)\cdot |J(\underline{y})|$$

Proof:

The idea is to work with the CDF and then work towards the PDF by deriving $n$ times. So, consider this hyperbox in ${\mathbb{R}}^n$:

$$B=\prod _{i=1}^n(a_i,b_i]$$

Then via definition of joint probabilities,

$$\begin{array}{rl}P(\underline{X}\le B)& =P(a_1\le x_1\le b_1,\dots ,a_n\le x_n\le b_n)\\ & ={\int }_B{f}_{\underline{X}}(x)d\underline{x}\\ & ={\int }_{T(B)}{f}_{\underline{X}}(T^{-1}(\underline{y}))|J(\underline{y})|d\underline{y}\end{array}$$

And since $\underline{X}\in B⟺\underline{Y}=T(\underline{X})\in T(B)$, then we can change the domain of the integral. But also, this tells us that

$$\begin{array}{rl}{\int }_{T(B)}{f}_{\underline{Y}}(\underline{y})d\underline{y}& =P(Y\in T(B))\\ & ={\int }_{T(B)}{f}_{\underline{X}}(T^{-1}(\underline{y}))|J(\underline{y})|d\underline{y}\end{array}$$

which

$$⟹f_{\underline{Y}}(\underline{y})=f_{\underline{X}}(T^{-1}(\underline{y}))\cdot |J(\underline{y})|$$

as desired.

Footnotes

1. To guarantee surjectivity. ↩

2. This means that it’s derivative $T^{\prime }$ is continuous, or that it is of class $C^1$. See here. ↩