multivariate transformation

Basic probability theorem when dealing with joint distribution

Today I got a friend’s question about probability:

“ Random variables (X) and (Y) are independent where (X) and (Y) are Gamma distributions with different scale parameters but same rate parameter. Proof that (Y) and (X/Y) are independent. “

This reminds me of multivariate transformation which is used in calculating the joint density of functions of multi-random variables. So let’s get started.

Since (X) and (Y) are independent, therefore the joint density is

[
p(x, y) = frac{beta ^{alpha_{1}}}{Gamma(alpha_{1})}e^{-beta x}x^{alpha_{1}-1} frac{beta ^{alpha_{2}}}{Gamma(alpha_{2})}e^{-beta y}y^{alpha_{2}-1} = frac{beta ^{alpha_{1}+alpha_{2}}}{Gamma(alpha_{1})Gamma(alpha_{2})}e^{-beta(x+y)}x^{alpha_{1}-1}y^{alpha_{2}-1}.
]

Same reason, we know if (X+Y) and (X/Y) are independent, their joint density could be expressed like this

[
p(X+Y, X/Y) = g(X+Y)h(X/Y).
]

Therefore if we can get the joint density of (X+Y) and (X/Y) and factorize it into two functions, we are done. Here comes the Multivariate transformation theorem.

We denote (U=f_{1}(x, y) = X+Y) and (V=f_{2}(x, y) = X/Y), then we inverse the transformation:

[
X = g_{1}(u, v) = frac{UV}{1+V}, Y = g_{2}(u, v) = frac{U}{1+V}.
]

Then the joint density of (U) and (V) is

[
p_{UV}(u, v) = p_{XY}(g_{1}(u, v), g_{2}(u, v))|J|, text{ for } u, v text{ on their support},
]

where (|J|) is the absolute value of determinant of Jacobian,

[
J=
begin{vmatrix}
frac{partial g_{1}}{partial u} & frac{partial g_{1}}{partial v}\
frac{partial g_{2}}{partial u} & frac{partial g_{2}}{partial v}
end{vmatrix}.
]

Actually the above transformation is just an extension from univariate to multivariate, remember in unvariate case we have:

“ Let (X) be a continuous (while it’s easier for discrete case) random variable having density (p_{X}(x)), then the density of function of random variable (X), (Y=g(x)), is

[
p_{Y}(y) = p_{X}(g^{-1}(y))Bigg|frac{dg^{-1}(y)}{dy}Bigg|, text{ for } y = g(x).
]

“

Back to our problem, we solved that the Jacobian is

[
|J| =
begin{vmatrix}
begin{vmatrix}
frac{v}{1+v} & frac{u}{(1+v)^2} \
frac{1}{1+v} & frac{-u}{(1+v)^2}
end{vmatrix}
end{vmatrix}
= frac{u}{(1+v)^2}.
]

Then

[
begin{aligned}
p_{UV}(u, v) =& frac{beta ^{alpha_{1}+alpha_{2}}}{Gamma(alpha_{1})Gamma(alpha_{2})}e^{-beta(u)}left(frac{uv}{1+v}right)^{alpha_{1}-1}left(frac{u}{1+v}right)^{alpha_{2}-1}frac{u}{(1+v)^2} \
​ propto& u^{alpha_{1}+alpha_{2}}e^{-beta u}frac{v^{alpha_{1}}}{(1+v)^{alpha_{1}+alpha_{2}}} \
​ =& g(u)h(v).
end{aligned}
]

QED.

And since here (X) and (Y) are independent, we can also apply convolution, for example,

[
p_{U}(u) = int^{infty}_{0}p_{X}(u-y)p_{Y}(y)dy.
]

Update on 10/4/2017

There’s a theorem relate to this practice (thanks to the notes from Advanced Machine Learning).

• Luckacs’ theorem:

If (X) and (Y) are independent random variables (non-negative and not constant), then

[
frac{X}{X+Y}text{ is independent with } X+Y
]

if and only if (X) and (Y) are Gamma distribution with same shape parameter (beta) (or sometimes (lambda)).