Confidential Interval
- $theta$ parameter, $x_1,x_2,dots,x_n$ are data
- An interval $(a,b)$ is a $1-alpha$ confidence interval (CI) for $theta$
- if $p(aleq theta leq b)geq 1-alpha$, $a$ and $b$ are function of data.
Two ways to construct CI:
- Approximate CIs based on MLE (large $n$)
- Suppose $hat theta$ is the MLE, then
- $hat theta overset{text{approx}}sim N(theta, frac{1}{I_n(theta)})$
- $g(hat theta) overset{text{approx}}sim N(g(theta), frac{[g’(theta)]^2}{I_n(theta)})$
- Suppose $hat theta$ is the MLE, then
Hence, $frac{hat theta - theta}{sqrt{1/I_n(theta)}} overset{text{approx}}sim N(0,1)$, $P(-1.96 leq frac{hat theta - theta}{sqrt{1/I_n(theta)}} leq 1.96) = 0.95$
Equivalently, $frac{hat theta - theta}{sqrt{1/I_n(theta)}} geq -1.96 Rightarrow hat theta - theta geq -1.96sqrt{1/I_n (theta)} Rightarrow theta leq hat theta + 1.96sqrt{1/I_n(theta)}$
$Rightarrow p(hat theta - 1.96sqrt{1/I_n(theta)} leq theta leq hat theta + 1.96sqrt{1/I_n(theta)})$ is an approximate $95%$ CI for $hat theta(MLE)$ when $n$ is large, this can also be written as $hat theta pm 1.96sqrt{-frac{[g(hat theta)]^2}{l’’(hat theta)}}$.
For binomial odds, $frac{p}{1-p}$, $hat Var(frac{hat p}{1-hat p})=frac{hat p}{n(1-hat p)^3}$, so approximate $95%$ CI of the odds is $frac{hat p}{1-hat p}pm 1.96sqrt{frac{hat p}{n(1-hat p)^3}}$.
- Exact CIs based on pivots
Definition: A pivot is a function, $Q(theta, x_1,x_2,dots,x_n)$ whose distribution does not depend on $theta$.
Ex. $xsim N(mu, sigma^)$ with unknown $mu$ and $sigma^2$, given $frac{x-mu}{sigma}sim N(0,1))$, then $Q(mu, x_1,x_2,dots,x_n)=frac{x-mu}{sigma}$ is a pivot.
Ex. $x_1,x_2,dots,x_n overset{text{iid}}sim N(mu,1)$ with unknown $mu$, given $bar x sim N(mu, frac{1}{N})$, then $Q(mu, x_1,x_2,dots,x_n)=frac{bar x-mu}{sqrt{1/N}}$ is a pivot.
Ex. $x_1,x_2,dots,x_n overset{text{iid}}sim N(0,sigma^2)$, then $Q(mu, x_1,x_2,dots,x_n)=frac{(n-1)S^2}{sigma^2}sim chi_{n-1}^2$ is a pivot.
How to make CIs from pivots?
- Find/show $Q(mu, x_1,x_2,dots,x_n)$ is a pivot
- Find $a_1$, $a_2$ to satisfy $p(a_1leq Q(mu, x_1,x_2,dots,x_n) leq a_2) = 1-alpha leftarrow$ Confidence level
- Rearrange to form $aleq theta leq b$
Ex $x_1 sim Expo(beta)$, a) shown that $Q = frac{x}{beta}$ is a pivot, b) use the 0.025 and 0.975 quantiles of $Q$ to construct an exact $95%$ CI for $beta$.
a) $p(Qleq q)=p(frac{x_1}{beta}leq q)=p(x_1leq qbeta)=int_0^{qbeta}frac{1}{beta}e^{-x/beta}dx = 1-e^{-frac{qbeta}{beta}}=1-e^{-q}$
so $Q$ is a pivot as the CDF doesn’t depend on $beta$.
$Q$ has PDF, $f(q)=frac{d}{dq}(1-e^{-q})=e^{-q}$ for $q$, so $ Qsim Expo(1)$.
b) $p(a_1leq Q leq a_2)=0.95$
Here, the 0.025 quantiles satisfies $0.025 = int_0^{a_1}e^{-q}dq = 1-e^{-a_1}Rightarrow a_1 = -log(0.975) = 0.0253$
he 0.975 quantiles satisfies $0.975 = int_0^{a_2}e^{-q}dq = 1-e^{-a_2}Rightarrow a_2 = -log(0.025) = 3.69$
so $p(0.0253leq frac{x_1}{beta} leq 3.69)=0.95$
$Rightarrow p(frac{x_1}{3.69}leq beta frac{x_1}{0.0253})=0.95$
$Rightarrow CI = (frac{x_1}{3.69}, frac{x_1}{0.0253})$
Ex. $x_1,x_2,dots,x_n overset{text{iid}}sim Expo(beta)$, $frac{x_i}{beta}sim Expo(1)$
a) show that $Q(beta,x_1,x_2,dots,x_n)=frac{sum x_i}{beta}$ is a pivot.
We know $frac{x_1}{beta}, frac{x_2}{beta},dots, frac{x_n}{beta} overset{text{iid}}sim Expo(1)$ and so $frac{sum x_i}{beta}sim Gamma(n,1)$ (Hint: MGF), which is a pivot since it doesn’t depend on $beta$.
b) construct a pivot CI for $beta$ when $c_1$ and $c_2$ satisfy $p(c_1 leq frac{sum x_1}{beta} leq c_2)$
$$p(c_1 leq frac{sum x_1}{beta} leq c_2) = p(frac{sum x_i}{c_2}leq beta leq frac{sum x_i}{c_1})$$
so $(frac{sum x_i}{c_2}, frac{sum x_i}{c_1})$ is an exact 95% CI for $beta$.
Two famours pivots, where $x_1,x_2,dots,x_n overset{text{iid}}sim N(mu,sigma^2)$ with unkonwn $mu, sigma^2$.
1) $frac{bar x - mu}{s/sqrt{n}} sim t_{n-1}$ is a pivot
so if we choose $c_1$ and $c_2$ such that $p(c_1 leq frac{bar x - mu}{s/sqrt{n}} leq c_2)=1-alpha$
then we get a $1-alpha$ exact CI for $mu$
$$p(c_1 leq frac{bar x - mu}{s/sqrt{n}} leq c_2)=1-alpha Rightarrow p(-frac{c_2S}{sqrt{n}} leq mu leq -frac{c_1S}{sqrt{n}}) = 1-alpha$$
Usually, we choose $c_1 = frac{alpha}{2}$ quantile of $t_{n-1}$ and $c_2 = 1- frac{alpha}{2}$ quantile of $t_{n-1}$
Denote $c_2 = t_{n-1,1-frac{alpha}{2}}$ and notice that $c_1 = -c_2$, since $t$ is symmetric
So exact $1-alpha$ CI for $mu$ is $(bar x - t_{n-1,1-frac{alpha}{2}}cdot frac{S}{sqrt{n}}, bar x + t_{n-1,1-frac{alpha}{2}}cdot frac{S}{sqrt{n}})$ or $bar x pm t_{n-1,1-frac{alpha}{2}}cdot frac{S}{sqrt{n}}$, whic is the t-based CI for $mu$.
2) $frac{(n-1)S^2}{sigma^2}simchi_{n-1}^2$ is a pivot, so if we choose $c_1$, $c_2$ such that $p(c_1 leq frac{(n-1)S^2}{sigma^2} leq c_2) = 1-alpha$, then we get the $1-alpha$ exact CI for $sigma^2$
$p(frac{(n-1)S^2}{c_1}leq sigma^2 leq frac{(n-1)S^2}{c_2}) = 1-alpha$
近期评论