9.7 - Maximum Likelihood Estimation Method


Maximum Likelihood Estimation requires that the data are sampled from a multivariate normal distribution. This is going to be a drawback to this method. If you have data collected on a Likert scale, which is most often the case in the social sciences, these kinds of data can not really be normally distributed, since they are discrete and bounded.

Using the Maximum Likelihood Estimation Method we must assume that the data are independently sampled from a multivariate normal distribution with mean vector µ and variance-covariance matrix that takes this particular form:

\(\Sigma = \mathbf{LL' +\Psi}\)

where L is the matrix of factor loadings and ψ is the diagonal matrix of specific variances.

To use this, we are going to have to use some extra notation: As usual, the data vectors for n subject will be represented as shown:

\(\mathbf{X_1},\mathbf{X_2}, \dots, \mathbf{X_n}\)

Maximum likelihood estimation involves estimating the mean, the matrix of factor loadings, and the specific variance.

The maximum likelihood estimator for the mean vector μ, the factor loadings L and the specific variances ψ are obtained by finding \(\hat{\mathbf{\mu}}\), \(\hat{\mathbf{L}}\), and \(\hat{\mathbf{\Psi}}\) that maximizes the log likelihood, which is given by the following expression:

\[l(\mathbf{\mu, L, \Psi}) =  - \frac{np}{2}\log{2\pi}- \frac{n}{2}\log{|\mathbf{LL' + \Psi}|} - \frac{1}{2}\sum_{i=1}^{n}\mathbf{(X_i-\mu)'(LL'+\Psi)(X_i-\mu)}\]

The log of the joint probability distribution of the data is to be maximized. We want to find the values of the parameters, (μ, L, and ψ), that is most compatible with what we see in the data. As was noted earlier the solutions for these factor models are not unique. Equivalent models can be obtained by rotation. To obtain a unique solution an additional constraint to be imposed is that \(\mathbf{L'\Psi^{-1}L}\) is a diagonal matrix.

Computationally this process is complex. In general, there is no closed-form solution to this maximization problem. So iterative methods must be applied. Implementation of iterative methods can run into problems, as we will see later.