Factor Analysis and Alternating Minimization

Factor analysis, in its original formulation, deals with the linear statistical model Y=HX+w (1) where H is a deterministic matrix, X and w independent random vectors, the first with dimension smaller than Y, the second with independent components. What makes this model attractive in applied research is the data reduction mechanism built in it. A large number of observed variables Y are explained in terms of a small number of unobserved (latent) variables X perturbed by the independent noise w. Under normality assumptions, which are the rule in the standard theory, all the laws of the model are specified by covariance matrices. More precisely, assume that X and ge are zero mean independent normal vectors with Cov(X) = P and Cov(w) = D, where D is diagonal. It follows from (1) that Cov(Y) = HPH T + D.