fundamental statistics learning note(19)

Intepretation of p-value

p-vale represent $p(reject H_O|H_O is true)$, calculated on your data. It is Not $p(H_O is true)$.

E.X. There are 1000 new cancer drugs developed. To test the effectiveness of each, suppose we have a hypothesis test that rejects $H_O$: there is no treatment effect at level $alpha=0.5$. Suppose we can achieve $90%$ power for their test if $H_O$ is false.
Case I: none of drugs actually work

1. For how many drugs do we expect $H_O$ to be rejected?
   Ans: according to $alpha = 0.5$, though none of drugs acutually work, there is $5%$ probability we could incorrectly reject $H_O$. Thus $50$ drugs are expected to be rejected.
2. What is $p(H_O is true)$ among drugs where $H_O$ is rejected?
   Ans: It is $frac{50}{50}=1$ because the rejected drugs are incorrectly rejected, they don’t have effects.

Case II: 100 of them actually work

1. For how many drugs do we expect $H_O$ to be rejected?
   Ans: Here we use the fact that we can achive $90%$ power for the test if $H_O$ is false, which means we could correctly reject $H_O$ by $90%$ when $H_O$ is false.
   $$begin{matrix}
   text{} & text{reject } H_O & text{accept } H_O & text{Total}
   text{work} & 90 & 10 & 100
   text{doesn’t wrok} & 45 & 855 & 900
   text{} & 135 & 865 & text{}
   end{matrix}$$
2. What is $p(H_O is true)$ among drugs where $H_O$ is rejected?
   Ans: It is $frac{45}{135}=frac{1}{3}$