I've already read the answers to this , but they were written so long ago that I wonder if some progress has been made on the subject since then.  I would like to know if there's some more objective way to choose the correlation or covariance matrix to do PCA than «careful thought and some experience» or «similar scales»?  Any help would be appreciated.  Edit: I asked this because there could be maybe some measure, and a respective scale of interpretation for when to use a covariance or correlation matrix. In statistics, we can observe some examples of this like for bayes factors, p-values, or at least an interpretative comparison like what happens with AIC or BIC, albeit these measures are for different objectives than the one I'm searching.
4429
What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
15875
Given a covariance matrix $\boldsymbol \Sigma_s$, how to generate data such that it would have the sample covariance matrix $\hat{\boldsymbol \Sigma} = \boldsymbol \Sigma_s$?    More generally: we are often interested in generating data from a density $ f(x \vert \boldsymbol\theta) $, with data $x$ given some parameter vector $\boldsymbol\theta$. This results in a sample, from which we may then again estimate a value $\boldsymbol{\hat\theta}$. What I am interested in is the reverse problem: What if we are given a set of parameters $\boldsymbol\theta_{s}$, and we would like to generate a sample $x$ such, that $ \boldsymbol{\hat\theta} = \boldsymbol\theta_{s}$.   Is this a known problem? Is such a method useful? Are algorithms available?
2564
Robust PCA (as developed by  or better yet ) is , but Mahalanobis distance can also be used for outlier detection given a . I'm curious about the (dis)advantages of using one method over the other.  My intuition tells me that the greatest distinction between the two is the following: When the data set is "small" (in a statistical sense), robust PCA will give a lower-rank covariance while robust covariance matrix estimation will instead give a full-rank covariance due to the Ledoit-Wolf regularization. How does this in turn affect outlier detection?
76076
We want to apply PCA to monitor our process testing data. After we plot the density of the testing data and do normality tests, we found them to be definitely not normal, but still symmetrical and long tailed.  What distribution can these be?  If they are not normal, is PCA still be applicable? If not, what other monitoring technique could be used?   We have too many of those testing data, so eyeballing them 1 by 1 is definitely not a good choice.    shapiro test   ARM_Z  RAMP_Z  DISC_Z RAMP_Z8  ARM_Z1  ARM_Z2   0.0000  0.0032  0.0000  0.0724  0.0246  0.0000   adf test   ARM_Z  RAMP_Z  DISC_Z RAMP_Z8  ARM_Z1  ARM_Z2     0.01    0.01    0.01    0.01    0.01    0.01   jarque-bera test   ARM_Z.X-squared  RAMP_Z.X-squared  DISC_Z.X-squared RAMP_Z8.X-squared  ARM_Z1.X-squared  ARM_Z2.X-squared             0.0000            0.4580            0.0003            0.8591            0.0029            0.0000    They are more peaked than corresponding fitted normal distributions.
34154
I am always confused by the difference between the use of for and of in cases like these:        Principal component analysis of microarray data.   Principal component analysis for microarray data.      Which of these is correct? Is it OK to use both? If yes, when should I use 1. and 2., respectively?
24039
How are PCA and classical MDS different? How about MDS versus nonmetric MDS? Is there a time when you would prefer one over the other? How do the interpretations differ?
32558
I have a dataset with 11 variables and PCA (orthogonal) was done to reduce the data. Deciding on the number of components to keep it was evident for me from my knowledge about the subject and the scree plot (see below) that two principal components (PCs) were enough to explain the data and the remaining components were only less informative.   Scree plot with parallel analysis:  observed eigenvalues (green) and simulated eigenvalues based on 100 simulations (red). Scree plot suggests 3 PCs, whereas parallel test suggests only the first two PCs.      As you can see only 48% of the variance could be captured by the first two PCs.    Plotting observations on the first plane made by the first 2 PCs revealed three different clusters using hierarchical agglomerative clustering (HAC) and K-means clustering. These  3 clusters turned out to be very relevant to the problem in question and were consistent with other findings as well. So except the fact that only 48% of variance was captured everything else was tremendously fine.      One of my two reviewers said: one cannot rely much on these findings as only 48% of variance could be explained and it is less than required.      Question Is there any required value of how much variance should be captured by PCA to be valid? Is it not dependent on the domain knowledge and methodology in use? Can anybody judge on the merit of the whole analysis just based on the mere value of the explained variance?    Notes     Data are 11 variables of genes measured by a very sensitive methodology in molecular biology called Real-Time Quantitative Polymerase Chain Reaction (RT-qPCR).   Analyses were done using R.   Answers from data analysts based on their personal experience working on real-life problems in the fields of microarray analysis, chemometrics, spectometric analyses or alike are much appreciated.   Please consider supporting you answer with references as much as possible.
78311
What are principal component scores (PC scores, PCA scores)?
97919
I have carried out a principal components analysis of six variables $A$, $B$, $C$, $D$, $E$ and $F$. If I understand correctly, unrotated PC1 tells me what linear combination of these variables describes/explains the most variance in the data and PC2 tells me what linear combination of these variables describes the next most variance in the data and so on.   I'm just curious -- is there any way of doing this "backwards"? Let's say I choose some linear combination of these variables -- e.g. $A+2B+5C$, could I work out how much variance in the data this describes?
59591
Suppose we have a covariance matrix $C$. Define the eigendecomposition, $C = Q^{-1} \Lambda Q$ and some other arbitrary basis $C = B^{-1} D B$.  Define $V_\text{PCA} = \text{diag}(QCQ^{-1})$ and $V_B = diag(BCB^{-1})$, and assume they're both sorted by magnitude. Define $T_\text{PCA} = \text{sum}(V_\text{PCA})$ and $T_B = \text{sum}(V_B)$. Is it possible that $V_\text{PCA}[i] / T_\text{PCA} &lt; V_B[i] / T_B$ for $i = 0$? If so, how is this possible? Doesn't it violate the idea that PCA initially finds the basis vector with maximum variance?  I can understand how the inequality holds for $i &gt; 0$, due to the orthogonality constraint of PCA which may not result in the "optimal" vectors being found in the variance-explained sense.  I'm trying to figure out if there's an error in my code, so please forgive the convoluted notation, but I want to be as clear as possible about what I'm seeing.
102747
While trying Gaussian Mixture Models , I found these 4 types of covariances.   'full' (each component has its own general covariance matrix), 'tied' (all components share the same general covariance matrix), 'diag' (each component has its own diagonal covariance matrix), 'spherical' (each component has its own single variance).   I googled a lot to find more details about each of these types but found very high-level descriptions (such as ) only.   Appreciate if someone can help me understand these, or at least direct me to somewhere I can read about these.
79192
Let's say I have a $p$-dimensional multivariate Gaussian distribution. And I take $n$ observations (each of them a $p$-vector) from this distribution and calculate the sample covariance matrix $S$. In this , the authors state that the sample covariance matrix calculated with $p &gt; n$ is singular.    How is it true or derived? Any explanations?
33215
I have a dataset with a large number of Yes/No responses. Can I use principal components (PCA) or any other data reduction analyses (such as factor analysis) for this type of data? Please advise how I go about doing this using SPSS.
94976
I've been testing PCA via SVD to decompose a simple time series data matrix, $X$.  I have two signals $x_1(t)$ and $x_2(t)$ in a data matrix where $M$ rows represents each timepoint sample and each column represents $x_1$ and $x_2$.  The mean signal, $\hat{x}$, is defined as the mean along the row axis (average of $x_1$ and $x_2$ along each timepoint).  I normalize each column of $X$ by subtracting its mean and dividing by the standard deviation.  When I use [U S V] = svd(Xz) in matlab, regardless of how the variables are distributed (whether they are correlated or uncorrelated), one of the columns of the right singular matrix, V, always points in the same direction (to a multiplicative constant) as the mean vector $(1/2, 1/2)$.  But when add an additional third time series, this is never the case (where the mean vector is $(1/3, 1/3, 1/3)$.  Because I normalize the standard deviation for each vector, it does make sense that the direction of most variance given by PCA would be the diagonal 45 degree line.  But if both variables $x_1$ and $x_2$ are independent gaussians, couldn't the PCA direction be any direction since the distribution is radially symmetric?  MATLAB Code:  s = RandStream('mcg16807', 'Seed', 0); RandStream.setDefaultStream(s);  G = zeros(1000,2); G(:,1) = 40*randn(1000,1)-100; G(:,2) = tan(G(:,1)) + randn(1000,1); X = G - repmat(mean(G),[size(G,1), 1]); Xz = X./repmat(std(G),[size(G,1), 1]); [U S V] = svd(Xz); G_mean = mean(Xz,2); corrcoef(G_mean, U(:,1)) corrcoef(G_mean, U(:,2))
38574
I am using Principal Component Analysis (PCA) to create an index required for my research. My question is how I should create a single index by using the retained principal components calculated through PCA.  For instance, I decided to retain 3 principal components after using PCA and I computed scores for these 3 principal components. What is the appropriate ways to create, for each respondent, a single index out of these 3 scores?   Is it relevant to add the 3 computed scores to have a composite value? Or to average the 3 scores to have such a value?  Or should I just keep the first principal component (the strongest) only and use its score as the index?   Alternatively, one could use Factor Analysis (FA) but the same question remains: how to create a single index based on several factor scores?
18745
What is the motivation for demeaning the data when doing PCA. I've been told to do it, but I've never heard a good and/or intuitive reason for it. Is this a case where doing it just makes the math easier, or is it more like: if you don't do it, you'll get the wrong answer.  This is something I've been wondering about for a while.  One other question: when you demean the data, do you also need to normalize it (i.e., divide by standard deviation)?
67276
I have a big dataset and I want to perform a dimensionality reduction.   Now everywhere I read that I can use PCA for this. However, I still don't seem to get what to do after calculating/performing the PCA. In R this is easily done with the command princomp.  But what to do after calculating the PCA? If I decided I want to use the first $100$ principal components, how do I reduce my dataset exactly?