Common Statistical Distributions by jau1990

Common Statistical Distributions Johar M. Ashfaque This note contains some of the key facts about various common statistical distributions. In the case if continuous distributions, the probability density function p(x) equals the derivative of the cumulative distribution function F (x) = P (X ≤ x). In the case of discrete distributions, the probability density function p(x) equals the probability that the random variable X takes the value of x. The mean or expectation is defined by Z E(X) =

xp(x)dx

or E(X) =

xp(x)

depending on whether the random variable is discrete or continuous. The variance is defined as Z V (X) = (x − E(X))2 p(x)dx or V (X) =

X (x − E(X))2 p(x)

depending on whether the random variable is discrete or continuous. A mode is any value for which p(x) is a maximum. The most common distributions have only one mode and therefore are called unimodal. A median is any value say m such that both P (X ≤ m) ≥

1 , 2

and

P (X ≥ m) ≥

1 . 2

In the case of most continuous distributions, there is a unique median say m and F (x) = P (X ≤ m) =

1 . 2

Normal Distribution

X is normal with mean θ and variance φ, denoted X ∼ N (θ, φ) if it has the density 1 exp p(X) = √ 2πφ

(X − θ)2 − , 2φ

−∞ < X < ∞.

The mean and variance are E(X) = θ, V ar(X) = φ. Since the distribution is symmetrical and unimodal, the median and mode both equal the mean median(X) = mode(X) = θ. If θ = 0 and φ = 1, X is said to have a standard normal distribution.

Chi-Square Distribution

X has a chi-squared distribution on ν degrees of freedom denoted X ∼ χ2nu if it has the same distribution as Z12 + Z22 + ... + Zν2 where Z1 , Z2 , ..., Zν are independent standard normal variates or equivalently X 1 X ν/2−1 exp − p(X) = ν/2 , 0 < X < ∞. 2 2 Γ(ν/2) If Y = X/S where S is a constant then Y is a chi-squared variate on ν degrees of freedom divided by S denoted Y ∼ S −1 χ2ν with density p(Y ) =

S ν/2 Y ν/2−1 exp ν/2 2 Γ(ν/2)

The mean and variance are

−

SY 2

0 < Y < ∞.

ν , S 2ν V ar(Y ) = 2 . S (Y ) =

The mode is mode(Y ) =

(ν − 2) S

provided ν ≥ 2.

Gamma Distribution

X has a one-parameter gamma distribution with parameter α denoted by X ∼ G(α) if it has density 1 X α−1 exp(−X), Γ(α) This is simply another name for the distribution p(X) =

0) < X < ∞.

1 2 χ . 2 2α If Y = βX then Y has two-parameter distribution with parameters α and β denoted Y ∼ G(α, β) and it has density 1 Y α−1 Y e exp − , p(Y ) = α β Γ(α) β so the mean and variance are E(Y ) = αβ,

0<Y <∞

V ar(Y ) = αβ 2 . This is another name for the distribution

1 2 βχ . 2 2α

If α = 1 so that the density is p(Y ) = β −1 exp(−Y /β) which is sometimes called the negative exponential distribution denoted Y ∼ E(β). 2

Student’s t Distribution

X has a Student’s t distribution on ν degrees of freedom denoted X ∼ tν if it has the same distribution as

Z p

where Z ∼ N (0, 1) and W ∼

χ2ν

W/ν

are independent or equivalently if X has the density

ν 1 p(X) = B , 2 2

−1

X2 1+ ν

−(ν+1)/2 .

It follows that is X1 , X2 , ..., Xn are independently N (µ, σ 2 ) and X = ΣXi /n S = Σ(Xi − X)2 s2 = S/(n − 1) then X −µ √ ∼ tn−1 . s/ n The mean and variance are E(X) = 0, ν V ar(X) = . ν−2 Since distribution is symmetrical and unimodal, the median and mode both equal the mean that is median(X) = mode(X) = 0.

Beta Distribution

X has a beta distribution with parameters α and β denoted X ∼ Be(α, β) if it has density p(X) =

1 X α−1 (1 − X)β−1 , B(α, β)

0<X<1

where the beta function B(α, β) is the well-known Euler beta function. The mean and variance are E(X) = α/(α + β), V ar(X) =

αβ . (α + β)2 (α + β + 1)

The mode is mode(X) = (α − 1)/(α + β − 2).

Binomial Distribution

X has a binomial distribution of index n and parameter π denoted X ∼ B(n, π) if it has a discrete distribution with density n X p(X) = π (1 − π)n−X , X

X = 0, 1, 2, ..., n.

The mean and variance are E(X) = nπ, V ar(X) = nπ(1 − π). Since

p(X + 1) (n − X)π = p(X) (X + 1)(1 − π)

it can be seen that p(X + 1) > p(X) if and only if X < (n + 1)π − 1 and therefore the mode occurs at mode(X) = [(n + 1)π].

Poisson Distribution

X has a Poisson distribution of mean λ denoted X ∼ P (λ) if it has a discrete distribution with density p(X) =

λX exp(−λ), X!

X = 0, 1, 2, ....

The mean and variance are E(X) = V ar(X) = λ. Since p(X + 1)/p(X) = λ/(X + 1) it can be seen that p(X + 1) > p(X) if and only if X <λ−1 and hence the mode occurs at mode(X) = [λ].

Hypergeometric Distribution

X has a hypergeometric distribution of population size N , index n and parameter π denoted X ∼ H(N, n, π) if it has a discrete distribution with density Nπ p(X) =

N (1−π) n−X N n

X = 0, 1, 2, ....

The mean and variance are E(X) = nπ, V ar(X) = nπ(1 − π)(N − n)/(N − 1). Since

p(X + 1) (n − X)(N π − X) = p(X) (X + 1)[N (1 − π) − n + X + 1]

it can be seen that p(X + 1) > p(X) if and only if X < (n + 1)π − 1 + (n + 1)(1 − 2π)/(N + 2) and is usually the case for N if fairly large if and only if X < (n + 1)π − 1 and therefore the mode occurs very close to the binomial value. As N → ∞, this distribution approaches the binomial distribution.

Uniform Distribution

X has uniform distribution on the interval (a, b) denoted X ∼ U (a, b) if it has the density p(X) = where I(a,b)

1 I(a,b) (X) b−a

( 1, X ∈ (a, b) = 0, X ∈ / (a, b)

is the indicator function of the set (a, b). The mean and variance are 1 (a + b), 2

E(X) =

(b − a)2 . 12 There is no unique mode but the distribution is symmetrical and therefore V ar(X) =

median(X) = E(X).

Pareto Distribution

X has a Pareto distribution with parameters ξ and γ denoted X ∼ P a(ξ, γ) if it has density p(X) = γξ γ X −γ−1 I(ξ,∞) (X) where I(ξ,∞)

( 1, = 0,

X > xi otherwise

is the indicator function of the set (ξ, ∞). The mean and variance are E(X) =

γξ , γ−1

provided γ > 1 5

V ar(X) =

γξ 2 , (γ − 1)2 (γ − 2)

provided γ > 2.

The distribution function is given by F (x) = [1 − (ξ/x)γ ]I(ξ,∞) (x) and the median is median(X) = 21/γ ξ and the mode occurs at mode(X) = ξ.

Circular Normal Distribution

X has a circular normal or von Mises’ distribution with mean µ and concentration parameter κ denoted X ∼ M (µ, κ) if it has density p(X) =

1 exp(κ cos(X − µ)) 2πI0 (κ)

where X is any angle so 0 ≤ X < 2π and I0 (κ) is the modified Bessel function of the first kind and of order zero. It turns out that 2r ∞ X 1 κ I0 (κ) = 2 (r!) 2 r=0 and for large κ asymptotically I0 (κ) ∼ √

1 exp(κ). 2πκ

Approximately, for large κ X ∼ N (µ, 1/κ) while for small kappa 1 κ {1 + cos(X − µ)} 2π 2 which is sometimes to referred to as the cardioid distribution. p(X) =

Behren’s Distribution

X is said to have Behren’s distribution with degrees of freedom ν1 and ν2 and angle φ denoted X ∼ BF (ν1 , ν2 , φ) if X has the same distribution as T2 cos φ − T1 sin φ where T1 and T2 are independent and T1 ∼ tν1

and

T2 ∼ tν2 .

Snedecor’s F Distribution

X has an F distribution on ν1 and ν2 degrees of freedom denoted X ∼ Fν1 ,ν2 if X has the same distribution as

W1 /ν1 W2 /ν2

where W1 and W2 are independent and W1 ∼ χ2ν1

and

The mean and variance are E(X) = V ar(X) =

W2 ∼ χ2ν2 .

ν2 , ν2 − 2

2ν22 (ν1 + ν2 − 2) . ν1 (ν2 − 2)2 (ν2 − 4)

The mode is mode(X) =

ν2 ν1 − 2 . ν2 + 1 ν1