Beta Distribution
Introduction to the beta distribution
The beta distribution is a continuous probability distribution defined over the open interval $( 0 , 1 )$. It depends on two positive numbers, $\alpha$ and $\beta$, which determine how the curve bends and how its mass is distributed along the interval. Because it only takes values between $0$ and $1$, it is often used to describe random quantities that represent proportions, ratios, or probabilities, situations where the outcomes are naturally limited within these bounds. In formal terms, the beta distribution is defined by the following probability density function:
\[B ( x ; \alpha , \beta ) = \frac{x^{\alpha - 1} ( 1 - x )^{\beta - 1}}{B ( \alpha , \beta )} 0 < x < 1\]where $B ( \alpha , \beta )$ is the beta function, related to the Gamma function by:
\[B ( \alpha , \beta ) = \frac{\Gamma ( \alpha ) \Gamma ( \beta )}{\Gamma ( \alpha + \beta )}\]Therefore, the probability density function can also be written explicitly in terms of the Gamma function as:
\[B ( x ; \alpha , \beta ) = \frac{\Gamma ( \alpha + \beta )}{\Gamma ( \alpha ) \Gamma ( \beta )} x^{\alpha - 1} ( 1 - x )^{\beta - 1}\]with $B ( x ; \alpha , \beta ) = 0$ for $x \notin ( 0 , 1 )$. The gamma function $\Gamma ( c )$ itself is defined, for every $c \in \mathbb{R}^{+} ,$ by the following integral representation:
\[\Gamma ( c ) = \int_{0}^{+ \infty} x^{c - 1} e^{- x} d x\]The gamma function can be regarded as a continuous extension of the factorial, which is defined only for natural numbers, to all positive real values.
The shape of the beta distribution
The shape of the beta distribution depends on the values of its parameters $\alpha$ and $\beta$. Depending on their magnitude, the distribution can take various forms: unimodal, U-shaped, or monotonic. The distribution reaches its mode at the point
\[x_{0} = \frac{\alpha - 1}{\alpha + \beta - 2}\]- If $\alpha > 1$ and $\beta > 1$, the distribution has a mode at $x_{0}$, corresponding to a maximum point of the density function.
- If $\alpha < 1$ and $\beta < 1$, the function has a minimum at the same point.
- In all other parameter combinations, the distribution is monotonic.
- When $\alpha = \beta$, the distribution is symmetric with respect to the vertical line $x = x_{0} = \frac{1}{2}$.
The figure illustrates one of the possible shapes of the beta distribution when both parameters $\alpha$ and $\beta$ are less than 1 and equal to each other. In this configuration, the distribution takes on a characteristic U-shaped form, with the density approaching infinity near the boundaries of the interval $( 0 , 1 )$. It is possible to observe a minimum point located at $x = x_{0}$, corresponding to the lowest value of the probability density within the domain.
An interesting case occurs when the two parameters are equal, that is $\alpha = \beta$, and both are greater than $1$. In this situation, the beta distribution becomes symmetric with respect to the vertical line $x = \frac{1}{2}$ and takes on a unimodal (that is, a single-peaked curve), bell-shaped form with a single central peak. As the values of $\alpha$ and $\beta$ increase, the curve becomes progressively narrower and increasingly similar to a normal distribution centered around $x = 0.5$.
To be more precise, this is an asymptotic approximation that holds for large values of $\alpha$ and $\beta$. For sufficiently large parameters, the Beta distribution can be approximated by a normal distribution with:
\(\mu = \frac{\alpha}{\alpha + \beta}\) \(\sigma^{2} = \frac{\alpha \beta}{( \alpha + \beta )^{2} ( \alpha + \beta + 1 )}\)
In the symmetric case, where $\alpha = \beta = k$ we have:
\[\mu = \frac{1}{2} \sigma^{2} = \frac{1}{8 ( 2 k + 1 )} \approx \frac{1}{16 k}\]Therefore, as $k \rightarrow \infty$:
\[B ( x ; k , k ) \approx \mathcal{N} ( \frac{1}{2} , \frac{1}{8 ( k + 1 )} )\]Key features
- \[\text{1}. f ( x ) = \frac{x^{\alpha - 1} ( 1 - x )^{\beta - 1}}{B ( \alpha , \beta )} 0 \leq x \leq 1\]
- \[\text{2}. \mu = E ( X ) = \frac{\alpha}{\alpha + \beta}\]
- \[\text{3}. \sigma^{2} = Var ( X ) = \frac{\alpha \beta}{( \alpha + \beta )^{2} ( \alpha + \beta + 1 )}\]
- \[\text{4}. \sigma = \sqrt{\frac{\alpha \beta}{( \alpha + \beta )^{2} ( \alpha + \beta + 1 )}}\]
Each expression highlights a key property of the Beta distribution, showing how its shape depends on the parameters $\alpha$ and $\beta$, and how its mean and variability reflect the balance between these two shape parameters.
Mean of the beta distribution
The mean, or expected value, of a beta distribution represents the average value of a random variable defined on the interval $( 0 , 1 )$, depending on the shape parameters $\alpha$ and $\beta$. Formally, the mean is obtained from the general definition of the expected value:
\[\mu = E ( X ) = \int_{0}^{1} x B ( x ; \alpha , \beta ) d x\]Substituting the probability density function of the Beta distribution we have:
\[E ( X ) = \int_{0}^{1} x \frac{x^{\alpha - 1} ( 1 - x )^{\beta - 1}}{B ( \alpha , \beta )} d x\]which simplifies to:
\[E ( X ) = \frac{1}{B ( \alpha , \beta )} \int_{0}^{1} x^{\alpha} ( 1 - x )^{\beta - 1} d x\]Recognizing that the integral on the right-hand side is itself the Beta function $B ( \alpha + 1 , \beta )$, we obtain:
\[E ( X ) = \frac{B ( \alpha + 1 , \beta )}{B ( \alpha , \beta )}\]Using the identity that relates the Beta and Gamma functions we obtain:
\[B ( \alpha , \beta ) = \frac{\Gamma ( \alpha ) \Gamma ( \beta )}{\Gamma ( \alpha + \beta )}\]Therefore, the mean can be expressed as:
\[E ( X ) = \frac{\alpha}{\alpha + \beta}\]Hence, the mean of the Beta distribution depends only on the two shape parameters and expresses the balance between them.
Variance of the beta distribution
The variance of the beta distribution measures how much the random variable is expected to vary around its mean value. While the mean describes the central tendency of the distribution, the variance quantifies its spread — that is, how concentrated or dispersed the possible outcomes are within the interval $( 0 , 1 )$. Formally, the variance is defined as:
\[\sigma^{2} = Var ( X ) = E ( X^{2} ) - [ E ( X ) ]^{2}\]Starting from the probability density function of the beta distribution:
\[B ( x ; \alpha , \beta ) = \frac{x^{\alpha - 1} ( 1 - x )^{\beta - 1}}{B ( \alpha , \beta )}\]the expression can be rewritten as:
\[E ( X^{2} ) & = \int_{0}^{1} x^{2} f ( x ; \alpha , \beta ) d x \\ & = \frac{1}{B ( \alpha , \beta )} \int_{0}^{1} x^{\alpha + 1} ( 1 - x )^{\beta - 1} d x \\ & = \frac{B ( \alpha + 2 , \beta )}{B ( \alpha , \beta )}\]Substituting this expression and the mean into the formula gives:
\[\sigma^{2} = \frac{B ( \alpha + 2 , \beta )}{B ( \alpha , \beta )} - (( \frac{\alpha}{\alpha + \beta} ))^{2}\]Using the relationship between the beta and gamma functions:
\[B ( \alpha , \beta ) = \frac{\Gamma ( \alpha ) \Gamma ( \beta )}{\Gamma ( \alpha + \beta )}\]we obtain the simplified expression for the variance:
\[\sigma^{2} = \frac{\alpha \beta}{( \alpha + \beta )^{2} ( \alpha + \beta + 1 )}\]When $\alpha$ and $\beta$ increase together, the variance decreases, causing the distribution to become more concentrated around its mean.
Relationship between the beta and uniform distribution
The uniform distribution can be regarded as a special case of the beta distribution. When both parameters are equal to one, that is $\alpha = \beta = 1$, the probability density function of the beta distribution becomes constant over the interval $( 0 , 1 )$. In the general case, the continuous uniform distribution defined over an interval $( a , b )$ is given by:
\[f ( x ) = \{ \frac{1}{b - a} & a < x < b \\ 0 & \text{otherwise}\]For $a = 0$ and $b = 1$, this expression reduces to $f ( x ) = 1$, which corresponds exactly to the $B ( 1 , 1 )$ distribution. In this case, the two parameters of the beta distribution take the values $\alpha = 1$ and $\beta = 1$, producing a constant probability density across the interval $0 , 1$.