4.3 - Exponent of Multivariate Normal Distribution

Printer-friendly versionPrinter-friendly version

Recall the Multivariate Normal Density function below:

\(\phi(\textbf{x}) = \left(\frac{1}{2\pi}\right)^{p/2}|\Sigma|^{-1/2}\exp\{-\frac{1}{2}(\textbf{x}-\mathbf{\mu})'\Sigma^{-1}(\textbf{x}-\mathbf{\mu})\}\)

You will note that this density function, φ(x), only depends on x through the squared Mahalanobis distance:

\((\textbf{x}-\mathbf{\mu})'\Sigma^{-1}(\textbf{x}-\mathbf{\mu})\)

This is the equation for a hyper-ellipse centered at μ.

For a bivariate normal, where p = 2 variables, we have an ellipse as shown in the plot below:

Useful Facts about the Exponent Component    \( (\textbf{x}-\mathbf{\mu})'\Sigma^{-1}(\textbf{x}-\mathbf{\mu})\) 

  • All values of x such that \( (\textbf{x}-\mathbf{\mu})'\Sigma^{-1}(\textbf{x}-\mathbf{\mu})=c\)  for any specified constant value c have the same value of the density f(x) and thus have equal likelihood.
  • As the value of \((\textbf{x}-\mathbf{\mu})'\Sigma^{-1}(\textbf{x}-\mathbf{\mu})\)  increases, the value of the density function decreases. The value of \((\textbf{x}-\mathbf{\mu})'\Sigma^{-1}(\textbf{x}-\mathbf{\mu})\)  increases as the distance between x and μ increases.
  • The variable \(d^2=(\textbf{x}-\mathbf{\mu})'\Sigma^{-1}(\textbf{x}-\mathbf{\mu})\) has a chi-square distribution with p degrees of freedom.
  • The value of d 2 for a specific observation \(\textbf{x}_j\) is called a squared Mahalanobis distance. It is calculated as \(d^2_j = (\textbf{x}_j-\mathbf{\bar{x}})'\Sigma^{-1}(\textbf{x}_j-\mathbf{\bar{x}})\).

If we define a specific hyper-ellipse by taking the squared Mahalanobis distance equal to a critical value of the chi-square distribution with p degrees of freedom and evaluate this at α, then the probability that the random value X will fall inside the ellipse is going to be equal to 1 - α.

\(\text{Pr}\{(\textbf{x}-\mathbf{\mu})'\Sigma^{-1}(\textbf{x}-\mathbf{\mu}) \le \chi^2_{p,\alpha}\}=1-\alpha\)

This particular ellipse is called the (1 - α) x 100% prediction ellipse for a multivariate normal random vector with mean vector μ and variance-covariance matrix Σ.

Calculating Mahalanobis Distance With SAS

SAS does not provide Mahalanobis distance directly, but we can compute them using principal components. The steps are:

  1. Determine principal components for the correlation matrix of the x-variables.
  2. Standardize the principal component scores so that each principal component has standard deviation = 1. For each component, this is done by dividing the scores by the square root of the eigenvalue. In SAS, use the STD option as part of the PROC PRINCOMP command to automate this standard deviation.
  3. For each observation, calculate d2 = sum of squared standardized principal components scores. This will equal the squared Mahalanobis distance.

Example - Calculating and Printing Mahalonobis Distances in SAS

Suppose we have four x-variables, called x1, x2, x3, x4, and they have already been read into SAS. The following SAS code (mahalonobis.sas) will determine standardized principal components and calculate Mahalanobis distances (the printout will include observation numbers). Within the DATA step, the “uss(of prin1-prin4)” function calculates the uncorrected sum of squares for the variables prin1-prin4. This value will be computed for each observation in the “pcout” data set. The result of the DATA step will be a SAS data set named “mahal” that will include the original variables, the standardized principal component scores (named prin1-prin4) and the Mahalanobis distance (named dist2).

SAS program

-

Click on the graphic or the link below to walk through how to find Mahalonobis distances using Minitab.