Mehmet Emre Tiryaki
Back to Notes

Common Distributions

M(θ)=E[eθX]M(\theta) = E[e^{\theta X}] is the moment generating function, with M(k)(0)=E[Xk]M^{(k)}(0) = E[X^k].

Discrete

p(x)E[X]Var(X)M(θ)Bernoulli(p)px(1−p)1−x,  x∈{0,1}pp(1−p)1−p+peθBinomial(n,p)(nx)px(1−p)n−xnpnp(1−p)(1−p+peθ)nGeometric(p)p(1−p)x−1,  x≥11p1−pp2peθ1−(1−p)eθNeg. Binomial(r,p)(x−1r−1)pr(1−p)x−r,  x≥rrpr(1−p)p2(peθ1−(1−p)eθ)rPoisson(λ)λxe−λx!λλeλ(eθ−1)\begin{array}{l|l|l|l|l} & p(x) & E[X] & \mathrm{Var}(X) & M(\theta) \\ \hline \text{Bernoulli}(p) & p^x(1 - p)^{1 - x}, \; x \in \{0, 1\} & p & p(1 - p) & 1 - p + pe^\theta \\ \text{Binomial}(n, p) & \binom{n}{x}p^x(1 - p)^{n - x} & np & np(1 - p) & (1 - p + pe^\theta)^n \\ \text{Geometric}(p) & p(1 - p)^{x - 1}, \; x \ge 1 & \frac{1}{p} & \frac{1 - p}{p^2} & \frac{pe^\theta}{1 - (1 - p)e^\theta} \\ \text{Neg. Binomial}(r, p) & \binom{x - 1}{r - 1}p^r(1 - p)^{x - r}, \; x \ge r & \frac{r}{p} & \frac{r(1 - p)}{p^2} & \Big(\frac{pe^\theta}{1 - (1 - p)e^\theta}\Big)^r \\ \text{Poisson}(\lambda) & \frac{\lambda^x e^{-\lambda}}{x!} & \lambda & \lambda & e^{\lambda(e^\theta - 1)} \end{array}
  • Binomial: number of successes in nn independent Bernoulli trials.
  • Geometric: the trial of the first success. Negative binomial: the trial of the rr-th success (geometric is r=1r = 1).
  • Hypergeometric(N,M,n)(N, M, n): number of type 1 objects in nn draws without replacement from NN type 1 and MM type 2 objects, p(k)=(Nk)(Mn−k)/(N+Mn)p(k) = \binom{N}{k}\binom{M}{n - k} / \binom{N + M}{n}.
  • Poisson approximates the binomial for large nn and small pp, with λ=np\lambda = np.

Continuous

f(x)E[X]Var(X)M(θ)Uniform(a,b)1b−a,  a<x<ba+b2(b−a)212eθb−eθaθ(b−a)Exponential(λ)λe−λx,  x>01λ1λ2λλ−θ,  θ<λGamma(a,b)xa−1e−x/bΓ(a)ba,  x>0abab2(1−bθ)−a,  θ<1bN(μ,σ2)1σ2πe−(x−μ)22σ2μσ2eμθ+12σ2θ2\begin{array}{l|l|l|l|l} & f(x) & E[X] & \mathrm{Var}(X) & M(\theta) \\ \hline \text{Uniform}(a, b) & \frac{1}{b - a}, \; a < x < b & \frac{a + b}{2} & \frac{(b - a)^2}{12} & \frac{e^{\theta b} - e^{\theta a}}{\theta(b - a)} \\ \text{Exponential}(\lambda) & \lambda e^{-\lambda x}, \; x > 0 & \frac{1}{\lambda} & \frac{1}{\lambda^2} & \frac{\lambda}{\lambda - \theta}, \; \theta < \lambda \\ \text{Gamma}(a, b) & \frac{x^{a - 1}e^{-x/b}}{\Gamma(a)b^a}, \; x > 0 & ab & ab^2 & (1 - b\theta)^{-a}, \; \theta < \frac{1}{b} \\ \mathcal{N}(\mu, \sigma^2) & \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x - \mu)^2}{2\sigma^2}} & \mu & \sigma^2 & e^{\mu\theta + \frac{1}{2}\sigma^2\theta^2} \end{array}

The uniform CDF is x−ab−a\frac{x - a}{b - a} on (a,b)(a, b) and the exponential CDF is 1−e−λx1 - e^{-\lambda x}. The gamma here has shape aa and scale bb.