Learning From Hidden Traits: Joint Factor Analysis and Latent Clustering
Abstract: Dimensionality reduction techniques play an essential role in data analytics, signal processing, and machine learning. Dimensionality reduction is usually performed in a preprocessing stage that is separate from subsequent data analysis, such as clustering or classification. Finding reduced reduced-dimension dimension representations that are well-suited suited for the intended task is more appealing. This paper proposes a joint factor analysis and latent clustering framework, which aims at learning clustercluster aware low-dimensional representations presentations of matrix and tensor data. The proposed approach leverages matrix and tensor factorization models that produce essentially unique latent representations of the data to unravel latent cluster structure-which which is otherwise obscured because of th the e freedom to apply an oblique transformation in latent space. At the same time, latent cluster structure is used as prior information to enhance the performance of factorization. Specific contributions include several custom custom-built built problem formulations, corresponding cor algorithms, and discussion of associated convergence properties. Besides extensive simulations, real--world world datasets such as Reuters document data and MNIST image data are also employed to showcase the effectiveness of the proposed approaches.