G-Test

Definition (G-Test)

Suppose $X$ is a random variable with finitely many values $a_1,\dots ,a_n$ and let $p_i=P(X=a_i)$. Suppose we make $N$ observations of the values $a_1,\dots ,a_n$ and that $O_i$ is the number of observations of $a_i$ that we made.

Let $E_i=N{p}_i$, and define

$$G=2\sum _{1=1}^n{O}_i\ln \left(\frac{O_i}{E_i}\right)$$

If $O_i=0$ for some $i$, then the summand is $0$. If there exists an $i$ such that $E_i=0$ but $O_i\ne 0$, then $G=\infty$.

Gibbs’ Inequality

We have that

$$G\ge 0$$

Moreover,

$$G=0⟺(\forall i\in {\mathbb{N}}_{\le n})(O_i=E_i)$$

Proof:

$(⟸)$ If $O_i=E_i$ then $\ln (O_i/E_i)=\ln 1=0$, and $G=0$.

$(⟹)$ The summand is always non-negative, so it must be equal to $0$.

What this tells us is that if $G$ is small, then it matches our expectations. Otherwise the observation is out of ordinary.

Wilks’ Theorem

Suppose the $N$ observations of the values $a_1,\dots ,a_n$ that we make are in fact independent observations of the random variable $X$. For large values of $N$, the values of $G$ are well approximated by a chi-square distribution with $n-1$ degrees of freedom.

The values of $G$ are well approximated by a chi-square distribution with $n-1$ degrees of freedom. Indeed, the probability that $G$ lands inside some interval $(a,b)$ is

$${\int }_a^b{f}_n(x)dx$$

So for our $G$, the probability of observing a value of $G^{\prime }>G$ is

$${\int }_G^{\infty }{f}_n(x)dx=1-{\int }_0^G{f}_n(x)dx$$

This is how we decide if $G$ is small or large.

Heuristic Addendum

The approximation of $G$ is good enough as long as the vast majority of the expected counts $E_1,\dots ,E_n$ are all at least $5$.