Mehmet Emre Tiryaki
Back to Notes

Hypothesis Testing

Definitions

  • Type I error Rejecting H0H_0 when it is true. The significance level is α=P(type I error)\alpha = P(\text{type I error}).
  • Type II error Accepting H0H_0 when it is false, β=P(type II error)\beta = P(\text{type II error}). The power is P(rejecting H0 when false)=1−βP(\text{rejecting } H_0 \text{ when false}) = 1 - \beta.
  • The test statistic and the rejection region (the values for which H0H_0 is rejected) define the test. The distribution of the test statistic under H0H_0 is the null distribution.
  • The p-value is the smallest significance level at which H0H_0 would be rejected for the observed TobsT_{obs}, e.g. P(Xˉ≥Tobs∣H0)P(\bar{X} \ge T_{obs} | H_0) for a rejection region Xˉ>c\bar{X} > c.
  • A hypothesis is simple if it specifies the distribution completely.

Neyman-Pearson

For simple H0H_0 and HAH_A, the likelihood ratio test rejects H0H_0 when

Λ=f(X1,…,Xn∣H0)f(X1,…,Xn∣HA)<c\Lambda = \frac{f(X_1, \ldots, X_n | H_0)}{f(X_1, \ldots, X_n | H_A)} < c

with cc chosen for significance level α\alpha. Any other test with significance level ≤α\le \alpha has power ≤\le that of this test. For a composite HAH_A, a test that is most powerful against every simple hypothesis in HAH_A is uniformly most powerful.

Normal mean with known variance H0:μ=μ0H_0 : \mu = \mu_0 against HA:μ=μa>μ0H_A : \mu = \mu_a > \mu_0. The likelihood ratio test rejects for Xˉ>C~\bar{X} > \tilde{C}, and under H0H_0, Xˉ∼N(μ0,σ2n)\bar{X} \sim \mathcal{N}(\mu_0, \frac{\sigma^2}{n}), so

P[Xˉ−μ0σ/n>C~−μ0σ/n ∣ H0]=α  ⟹  reject if Xˉ>μ0+zασnP\Big[ \frac{\bar{X} - \mu_0}{\sigma/\sqrt{n}} > \frac{\tilde{C} - \mu_0}{\sigma/\sqrt{n}} \,\Big|\, H_0 \Big] = \alpha \implies \text{reject if } \bar{X} > \mu_0 + z_\alpha \frac{\sigma}{\sqrt{n}}

with P(Z≥zα)=αP(Z \ge z_\alpha) = \alpha. This does not depend on μa\mu_a, only on μa>μ0\mu_a > \mu_0, so the test is uniformly most powerful for the composite HA:μ>μ0H_A : \mu > \mu_0. Its power at μa\mu_a is

P(Xˉ>μ0+zασn ∣ μa)=P(Z>zα−(μa−μ0)nσ)→1 as n→∞P\Big( \bar{X} > \mu_0 + z_\alpha \frac{\sigma}{\sqrt{n}} \,\Big|\, \mu_a \Big) = P\Big( Z > z_\alpha - \frac{(\mu_a - \mu_0)\sqrt{n}}{\sigma} \Big) \to 1 \text{ as } n \to \infty

Generalized Likelihood Ratio

Λ=max⁡θ∈ω0lik(θ)max⁡θ∈Ωlik(θ)\Lambda = \frac{\max_{\theta \in \omega_0} \mathrm{lik}(\theta)}{\max_{\theta \in \Omega} \mathrm{lik}(\theta)}

and reject H0H_0 for Λ<c\Lambda < c. Let dim⁡\dim be the number of free parameters.

Wilks For regular f(x1,…,xn∣θ)f(x_1, \ldots, x_n | \theta) and dim⁡Ω−dim⁡ω0=k\dim \Omega - \dim \omega_0 = k, under H0H_0

−2log⁡Λ→dχk2-2\log\Lambda \xrightarrow{d} \chi^2_k

Two-sided normal mean with known variance H0:μ=μ0H_0 : \mu = \mu_0 against μ≠μ0\mu \neq \mu_0. Here

−2log⁡Λ=[Xˉ−μ0σ/n]2-2\log\Lambda = \Big[ \frac{\bar{X} - \mu_0}{\sigma/\sqrt{n}} \Big]^2

so rejecting for −2log⁡Λ>c-2\log\Lambda > c means P[Xˉ−μ0σ/n>c ∣ H0]+P[Xˉ−μ0σ/n<−c ∣ H0]=αP\big[ \frac{\bar{X} - \mu_0}{\sigma/\sqrt{n}} > \sqrt{c} \,|\, H_0 \big] + P\big[ \frac{\bar{X} - \mu_0}{\sigma/\sqrt{n}} < -\sqrt{c} \,|\, H_0 \big] = \alpha, i.e. reject if

Xˉ<μ0−zα/2σnorXˉ>μ0+zα/2σn\bar{X} < \mu_0 - z_{\alpha/2}\frac{\sigma}{\sqrt{n}} \quad \text{or} \quad \bar{X} > \mu_0 + z_{\alpha/2}\frac{\sigma}{\sqrt{n}}

For normal data, μ^MLE=Xˉ\hat{\mu}_{MLE} = \bar{X} and σ^MLE2=1n∑i=1n(Xi−Xˉ)2\hat{\sigma}^2_{MLE} = \frac{1}{n}\sum_{i=1}^n (X_i - \bar{X})^2, and σ^MLE2=1n∑i=1n(Xi−μ0)2\hat{\sigma}^2_{MLE} = \frac{1}{n}\sum_{i=1}^n (X_i - \mu_0)^2 when μ=μ0\mu = \mu_0 is known.