Multinomial distribution: Definition from Answers.com

Multinomial
parameters:	$n > 0$ number of trials (integer) $p_1, \ldots p_k$ event probabilities ( $Σ p i = 1$ )
support:	$X_i \in \{0,\dots,n\}$ $\Sigma X_i = n\!$
pmf:	$\frac{n!}{x_1!\cdots x_k!} p_1^{x_1} \cdots p_k^{x_k}$
cdf:
mean:	$E {X i} = n p i$
median:
mode:
variance:	$\textstyle{\mathrm{Var}}(X_i) = n p_i (1-p_i)$ $\textstyle {\mathrm{Cov}}(X_i,X_j) = - n p_i p_j~~(i\neq j)$
skewness:
kurtosis:
entropy:
mgf:	$\left( \sum_{i=1}^k p_i e^{t_i} \right)^n$
cf:

In probability theory, the multinomial distribution is a generalization of the binomial distribution.

The binomial distribution is the probability distribution of the number of "successes" in n independent Bernoulli trials, with the same probability of "success" on each trial. In a multinomial distribution, the analog of the Bernoulli distribution is the categorical distribution, where each trial results in exactly one of some fixed finite number k of possible outcomes, with probabilities p₁, ..., p_k (so that p_i ≥ 0 for i = 1, ..., k and $\sum_{i=1}^k p_i = 1$ ), and there are n independent trials. Then let the random variables X_i indicate the number of times outcome number i was observed over the n trials. The vector X = (X₁, ..., X_k) follows a multinomial distribution with parameters n and p, where p = (p₁, ..., p_k).

1 Specification
- 1.1 Probability mass function
2 Properties
3 Example
4 Sampling from a multinomial distribution
5 Related distributions
6 See also
7 External links
8 References

Specification

Probability mass function

The probability mass function of the multinomial distribution is:

$\begin{align} f(x_1,\ldots,x_k;n,p_1,\ldots,p_k) & {} = \Pr(X_1 = x_1\mbox{ and }\dots\mbox{ and }X_k = x_k) \\ \\ & {} = \begin{cases} { \displaystyle {n! \over x_1!\cdots x_k!}p_1^{x_1}\cdots p_k^{x_k}}, \quad & \mbox{when } \sum_{i=1}^k x_i=n \\ \\ 0 & \mbox{otherwise,} \end{cases} \end{align}$

for non-negative integers x₁, ..., x_k.

Properties

The expected number of times the outcome i was observed over n trials is

$\operatorname{E}(X_i) = n p_i.\,$

The covariance matrix is as follows. Each diagonal entry is the variance of a binomially distributed random variable, and is therefore

$\operatorname{var}(X_i)=np_i(1-p_i).\,$

The off-diagonal entries are the covariances:

$\operatorname{cov}(X_i,X_j)=-np_i p_j\,$

for i, j distinct.

All covariances are negative because for fixed n, an increase in one component of a multinomial vector requires a decrease in another component.

This is a k × k positive-semidefinite matrix of rank k − 1.

The off-diagonal entries of the corresponding correlation matrix are

$\rho(X_i,X_j) = -\sqrt{\frac{p_i p_j}{ (1-p_i)(1-p_j)}}.$

Note that the sample size drops out of this expression.

Each of the k components separately has a binomial distribution with parameters n and p_i, for the appropriate value of the subscript i.

The support of the multinomial distribution is the set

$\{(n_1,\dots,n_k)\in \mathbb{N}^{k}| n_1+\cdots+n_k=n\}.\,$

Its number of elements is

${n+k-1 \choose k-1} = \left\langle \begin{matrix}n \\ k \end{matrix}\right\rangle,$

the number of n-combinations of a multiset with k types, or multiset coefficient.

Example

In a recent three-way election for a large country, candidate A received 20% of the votes, candidate B received 30% of the votes, and candidate C received 50% of the votes. If six voters are selected randomly, what is the probability that there will be exactly one supporter for candidate A, two supporters for candidate B and three supporters for candidate C in the sample?

Note: Since we’re assuming that the voting population is large, it is reasonable and permissible to think of the probabilities as unchanging once a voter is selected for the sample. Technically speaking this is sampling without replacement, so the correct distribution is the multivariate hypergeometric distribution, but the distributions converge as the population grows large.

$\Pr(A=1,B=2,C=3) = \frac{6!}{1! 2! 3!}(.2^1) (.3^2) (.5^3) = .135$

Sampling from a multinomial distribution

First, reorder the parameters $p_1, \ldots p_k$ such that they are sorted descendingly (this is only to speed up computation and not strictly necessary). Now, for each trial, draw an auxiliary variable X from a uniform (0, 1) distribution. The resulting outcome is the component

$j = \arg \min_{j'=1}^{k} \left( \sum_{i=1}^{j'} p_i \ge X \right).$

This is a sample for the multinomial distribution with n = 1. A sum of independent repetitions of this experiment is a sample from a multinomial distribution with n equal to the number of such repetitions.

Related distributions

When k = 2, the multinomial distribution is the binomial distribution.
Categorical distribution, the distribution of each trial; for k = 2, this is the Bernoulli distribution
The Dirichlet distribution is the conjugate prior of the multinomial in Bayesian statistics.
Multivariate Polya distribution
Beta-binomial model

External links

References

Evans, Merran; Nicholas Hastings, Brian Peacock (2000). Statistical Distributions. New York: Wiley. pp. 134–136. 3rd ed.. ISBN 0-471-37124-6.

Probability distributions

Discrete univariate with finite support

Benford · Bernoulli · binomial · categorical · hypergeometric · Rademacher · discrete uniform · Zipf · Zipf-Mandelbrot

Discrete univariate with infinite support

Boltzmann · Conway–Maxwell–Poisson · compound Poisson · discrete phase-type · extended negative binomial · Gauss–Kuzmin · geometric · logarithmic · negative binomial · parabolic fractal · Poisson · Skellam · Yule–Simon · zeta

Continuous univariate supported on a bounded interval, e.g. [0,1]

Beta · Irwin–Hall · Kumaraswamy · raised cosine · triangular · U-quadratic · uniform · Wigner semicircle

Continuous univariate supported on a semi-infinite interval, usually [0,∞)

Beta prime · Bose–Einstein · Burr · chi-square · chi · Coxian · Erlang · exponential · F · Fermi–Dirac · folded normal · Fréchet · Gamma · generalized extreme value · generalized inverse Gaussian · half-logistic · half-normal · Hotelling's T-square · hyper-exponential · hypoexponential · inverse chi-square (scaled inverse chi-square) · inverse Gaussian · inverse gamma · Lévy · log-normal · log-logistic · Maxwell–Boltzmann · Maxwell speed · Nakagami · noncentral chi-square · Pareto · phase-type · Rayleigh · relativistic Breit–Wigner · Rice · Rosin–Rammler · shifted Gompertz · truncated normal · type-2 Gumbel · Weibull · Wilks' lambda

Continuous univariate supported on the whole real line (−∞, ∞)

Cauchy · extreme value · exponential power · Fisher's z · generalized normal · generalized hyperbolic · Gumbel · hyperbolic secant · Landau · Laplace · logistic · normal (Gaussian) · normal inverse Gaussian · skew normal · slash · Student's t · type-1 Gumbel · Variance-Gamma · Voigt

Multivariate (joint)

Discrete: Ewens · Beta-binomial · multinomial · multivariate Polya Continuous: Dirichlet · Generalized Dirichlet · multivariate normal · multivariate Student · normal-scaled inverse gamma · normal-gamma Matrix-valued: inverse-Wishart · matrix normal · Wishart

Directional, degenerate, and singular

Directional: Kent · von Mises · von Mises–Fisher Degenerate: discrete degenerate · Dirac delta function Singular: Cantor

Families

exponential · natural exponential · location-scale · maximum entropy · mixture · Pearson · stable · Tweedie

This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer)

Donate to Wikimedia

	quadrinomial distribution (statistics)
	trinomial distribution (statistics)
	Dirichlet distribution

	Is the F distribution a symmetric distribution? Read answer...
	What is the trustees distribution? Read answer...
	How is money distributed? Read answer...

	What is a multinomial logit model?
	What is the difference between plynomial and multinomial?
	What is the multinomial logit model of transportation?

		Sci-Tech Dictionary. McGraw-Hill Dictionary of Scientific and Technical Terms. Copyright © 2003, 1994, 1989, 1984, 1978, 1976, 1974 by McGraw-Hill Companies, Inc. All rights reserved. Read more
		Wikipedia. This article is licensed under the Creative Commons Attribution/Share-Alike License. It uses material from the Wikipedia article "Multinomial distribution". Read more

Multinomial distribution

Contents