Survey Sampling

Population

A population of size $N$ with values $x_1, \ldots, x_N$ (or pairs $(x_i, y_i)$),

$$\mu = \frac{1}{N}\sum_{i=1}^N x_i, \qquad \tau = \sum_{i=1}^N x_i = N\mu, \qquad \sigma^2 = \frac{1}{N}\sum_{i=1}^N (x_i - \mu)^2 = \frac{1}{N}\sum_{i=1}^N x_i^2 - \mu^2, \qquad \sigma_{xy} = \frac{1}{N}\sum_{i=1}^N (x_i - \mu_x)(y_i - \mu_y)$$

For a dichotomous population ($x_i \in \{0, 1\}$), $\mu = p$ and $\sigma^2 = p(1 - p)$.

Simple Random Sampling

Draw $X_1, \ldots, X_n$. With replacement they are IID. Without replacement they are not independent, but they are identically distributed and exchangeable: if $\xi$ is a value taken by $n_\xi$ members of the population, $P(X_i = \xi) = \frac{n_\xi}{N}$. A particular ordered sample has probability $\frac{1}{N^n}$ with replacement and $\frac{(N - n)!}{N!}$ without.

Sample Mean and Total

$$\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i, \qquad T = N\bar{X}, \qquad E[\bar{X}] = \mu, \qquad E[T] = \tau$$

with $\bar{X} = \hat{p}$ in the dichotomous case. Without replacement, $\mathrm{Cov}(X_i, X_j) = -\frac{\sigma^2}{N - 1}$ for $i \neq j$, so

$$\mathrm{Var}(\bar{X}) = \frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \mathrm{Cov}(X_i, X_j) = \begin{cases} \frac{\sigma^2}{n} & \text{with replacement} \\ \frac{\sigma^2}{n}\Big(1 - \frac{n - 1}{N - 1}\Big) & \text{without replacement} \end{cases}$$

with $\frac{n}{N}$ the sampling fraction. $\mathrm{Var}(T) = N^2\mathrm{Var}(\bar{X})$, and $\mathrm{Var}(\hat{p}) \le \frac{1}{4n}$. For pairs, $\mathrm{Cov}(X_i, Y_i) = \sigma_{xy}$, $\mathrm{Cov}(X_i, Y_j) = -\frac{\sigma_{xy}}{N - 1}$ for $i \neq j$ without replacement, and

$$\mathrm{Cov}(\bar{X}, \bar{Y}) = \frac{\sigma_{xy}}{n}\Big(1 - \frac{n - 1}{N - 1}\Big)$$

Consistency By Chebyshev, $P(|\bar{X} - \mu| > \delta) \le \frac{\mathrm{Var}(\bar{X})}{\delta^2} \to 0$ as $n \to \infty$.

Estimating the Variance

$\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n (X_i - \bar{X})^2 = \frac{1}{n}\sum_{i=1}^n X_i^2 - \bar{X}^2$ is biased. Without replacement,

$$E[\hat{\sigma}^2] = \frac{1}{n}\sum_{i=1}^n E[X_i^2] - E[\bar{X}^2] = \frac{1}{n}\sum_{i=1}^n (\sigma^2 + \mu^2) - \frac{\sigma^2}{n}\Big(1 - \frac{n - 1}{N - 1}\Big) - \mu^2 = \sigma^2\Big(\frac{n - 1}{n}\Big)\frac{N}{N - 1}$$

i.e. $E[\hat{\sigma}^2] = \sigma^2 - \mathrm{Var}(\bar{X})$. Multiplying by the reciprocal of the bias factor gives unbiased estimates. With $s^2 = \frac{1}{n - 1}\sum_{i=1}^n (X_i - \bar{X})^2 = \frac{n}{n - 1}\hat{\sigma}^2$, $E[s^2] = \frac{N}{N - 1}\sigma^2$, so $\big(1 - \frac{1}{N}\big)s^2$ is unbiased for $\sigma^2$, and

$$s_{\bar{X}}^2 = \frac{s^2}{n}\Big(1 - \frac{n}{N}\Big), \qquad s_T^2 = N^2 s_{\bar{X}}^2, \qquad s_{\hat{p}}^2 = \frac{\hat{p}(1 - \hat{p})}{n - 1}\Big(1 - \frac{n}{N}\Big)$$

are unbiased for $\mathrm{Var}(\bar{X})$, $\mathrm{Var}(T)$ and $\mathrm{Var}(\hat{p})$. In the dichotomous case $s^2 = \frac{n}{n - 1}\hat{p}(1 - \hat{p})$.

Normal Approximation

$$P\Big( \frac{\bar{X}_n - \mu}{\sigma_{\bar{X}}} \le z \Big) \to \Phi(z), \qquad P(|\bar{X} - \mu| \le \delta) \approx 2\Phi\Big( \frac{\delta}{\sigma_{\bar{X}}} \Big) - 1$$

since $\Phi(-z) = 1 - \Phi(z)$. A $100(1 - \alpha)\%$ confidence interval is $\bar{X} \pm z(\alpha/2)\, s_{\bar{X}}$, with $z(\alpha/2)$ the point with area $\alpha/2$ of the standard normal to its right.

Ratio Estimation

Expanding $g$ around the means,

$$g(X, Y) \approx g(\mu_x, \mu_y) + \frac{\partial g}{\partial x}(X - \mu_x) + \frac{\partial g}{\partial y}(Y - \mu_y) + \frac{1}{2}\frac{\partial^2 g}{\partial x^2}(X - \mu_x)^2 + \frac{1}{2}\frac{\partial^2 g}{\partial y^2}(Y - \mu_y)^2 + \frac{\partial^2 g}{\partial x \partial y}(X - \mu_x)(Y - \mu_y)$$

(derivatives at $(\mu_x, \mu_y)$). In one variable, $E[g(X)] \approx g(\mu) + \frac{g''(\mu)}{2}\mathrm{Var}(X)$. For the ratio $R = \bar{Y}/\bar{X}$ estimating $r = \mu_y/\mu_x$,

$$E\Big[ \frac{\bar{Y}}{\bar{X}} \Big] \approx \frac{\mu_y}{\mu_x} + \frac{\mu_y}{\mu_x^3}\mathrm{Var}(\bar{X}) - \frac{1}{\mu_x^2}\sigma_{\bar{X}\bar{Y}}, \qquad \mathrm{Var}\Big( \frac{\bar{Y}}{\bar{X}} \Big) \approx \frac{1}{\mu_x^2}\big( r^2\sigma_{\bar{X}}^2 + \sigma_{\bar{Y}}^2 - 2r\sigma_{\bar{X}\bar{Y}} \big)$$