• Probability Generating Function (PGF) : related to Z-transform
• Moment Generating Function (MGF) : related to Laplace transform
• Characteristic Function (CF) : related to Fourier transform

• PGF: handling non-negative integral random variables
• MGF: handling general random variables
• CF: equally useful with MGF

• Easy to characterizing the distribution of the sum of independent random variables
• Play a central role in the study of branching processes
• Provide a bridge between complex analysis and probability
• ……

Definition (Moment Generating Function) MGF of an r.v. $X$ is $M(t) = E(e^{tX})$, as a function of $t$ (different t denote different valued moment) and this must finite on some open interval (-a,a) containing 0 or dont exist.

• MGF encodes the moments of an r.v.
• MGF of an r.v. Determines its distribution, like CDF and PMF/PDF
• MFG make it easy to find the distribution of a sum of i.r.v.s.

Theorem (Moments via Derivatives of the MGF) $E(X^n) = M^{(n)}(0)$
Using Taylor expansion of $M(t)$ at 0
$$M(t) = sum_{n=0}^{infty} M^{(n)}(0) frac{t^n}{n!}$$ Using Taylor expansion of $E(X)$
$$M(t) = E(e^{tX}) = Eleft( sum_{n=0}^{infty} X^n frac{t^n}{n!} right)=sum_{n=0}^{infty} E(X^n) frac{t^n}{n!}$$ Matching the coefficients of two expansions, we get $E(X^n) = M^{(n)}(0)$

Theorem (MGF Determines the Distribution) Two r.v. have the same MGF have the same distribution, more strictly, if there is even a tiny interval containing 0 on which the MGF are equal, the the r.v.s must have same distribution.

Example 1 (Bernoulli MGF) MGF of $Xsim Bern(p)$
$e^{tX}=e^t$ with probability $p$, and $1$ with probability $q$, so $M(t) = E(e^{tX})=pe^t + q$

Example 2 (Geometric MGF) MGF of $Xsim Geom(p)$
$$M(t) = E(e^{tX})=sum_{k=0}^{infty} e^{tk}q^kp=psum_{k=0}^{infty} (qe^t)^k=frac{p}{1-qe^t}$$ t in $(-infty, log(1/p))$

Example 3 (Uniform MGF) MGF of $Usim Unif(a,b)$
$$M(t) = E(e^{tU}) = int_a^b e^{tu}frac{1}{b-a} du = frac{e^{tb}-e^{ta}}{t(b-a)}$$ and $M(0) = 1$

Example 4 (Binomial MGF) $Bin(n,p)$
$$M(t) = (pe^t+q)^n$$ Example 5 (Negative Binomial) $NBin(r,p)$
$$M(t) = left( frac{p}{1-qe^t} right)^r$$ Theorem (MGF of Location-Scale Transformation) If $X$ has MGF $M(t)$, then MGF of $a+bX$ is
$$E(e^{t(a+bX)})=e^{at}E(e^{btX})=e^{at}M(bt)$$ Example 6 (Normal MGF) MGF of $(X = mu + sigma Z) sim N(mu,sigma^2)$
$$M_Z(t) = E(e^{tZ})=int_{-infty}^{infty}e^{tz}frac{1}{sqrt{2pi}}e^{-z^2/2}dz=e^{t^2/2}$$ Use the Theorem above then
$$M_X(t) = e^{mu t}M_Z(sigma t) = e^{mu t}e^{(sigma t)^2/2} = e^{mu t + frac{1}{2} sigma^2 t^2}$$

Theorem (MGF of A Sum of Independent R.V.s) If $X$ and $Y$ are independent, Then
$$M_{X+Y} (t) = M_X(t) M_Y(t)$$ Example 1 (Sum of Poissons) $Xsim Pois(lambda), Ysim Pois(mu)$, $X$ and $Y$ are independent. Then $X+Y sim Pois(lambda + mu)$
The MGF of $X$ is
$$E(e^{tX}) = sum_{k=0}^{infty}e^{tk}frac{e^{-lambda}lambda^k}{k!}=e^{-lambda}sum_{k=0}^{infty}frac{(lambda e^t)^k}{k!}=e^{-lambda}e^{lambda e^t}=e^{lambda(e^t-1)}$$ The MGF of $X+Y$ is
$$E(e^{tX})E(e^{tY}) = e^{lambda (e^t-1)} e^{mu (e^t-1)} = e^{(lambda + mu)(e^t-1)}$$ Which is the $Pois(lambda + mu)$, so $X+Ysum Pois(lambda+mu)$

Example 2 (Sum of Normals) $X_1sim N(mu_1,sigma_1^2)$ and $X_2 sim N(mu_2,sigma_2^2)$, $X_1+X_2 = ?$
MGF of $X_1+X_2$ is
$$M_{X_1+X_2}(t)= M_{X_1}(t)M_{X_2}(t)= e^{mu_1t+frac{1}{2}sigma^2_1t^2}cdot e^{mu_2 t+frac{1}{2}sigma_2^2 t^2}=e^{(mu_1+mu_2)t+frac{1}{2}(sigma_1^2+sigma_2^2)t^2}$$ Which is the N(mu_1 + mu_2, sigma_2^2 + sigma_1^2) MGF.

Definition (PGF) PGF of a nonnegative integer-valued r.v. $X$ with PMF $p_k = P(X=k)$ is the generating function of the PMF, By LOTUS , this is
$$E(t^X) = sum_{k=0}^{infty} p_k t^k$$ Example 1 (Generating Dice Probabilities) Let $X$ be the sum from rolling 6 pair dice, $X_1,…,X_6$ be the individual rolls, what is $P(X=18)$ ?
The PGF of $X_1$ is
$$E(t^{X_1}) = frac{1}{6}(t+t^2+dotsb+t^6)$$ The PGF of $X$ is
$$E(t^X) = E(t^{X_1}dotsb t^{X_6}) = E(t^{X_1})dotsb E(t^{X_6})=frac{t^6}{6^6}(1+t+dotsb +t^5)^6$$ The coefficient of $t^{18}$ in the PGF is $P(X=18)$, so
$$P(X=18) = frac{3421}{6^6}$$ Theorem (PMF & PGF)
$$P(X=k) = frac{g_{X}^{(k)}(0)}{k!}$$

Definition CF The Characteristic function of a random variable $X$ is the function $phi : R rightarrow C$ defined by
$$phi(t) = E(e^{itX}), i = sqrt{-1}$$