What are the mathematical steps to get loadings and scores matrices of a 3x3 matrix basing of PCA and what is the relationship relating eigenvalues eigenvectors with loadings and score?
1306
In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues.   I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like a machine. But I didn't understand it. I didn't get the purpose of it. I didn't get the feel of it.        I strongly believe in the following quote:     You do not really understand something unless you can explain it to your grandmother. -- Albert Einstein   Well, I can't explain these concepts to a layman or grandma.   Why PCA, eigenvectors &amp; eigenvalues? What was the need for these concepts? How would you explain these to a layman?
21424
I perform Principal Component Analysis using two variables that are standardized. This is done by applying a SVD on the correlation matrix of the concerned variates. However, the SVD gives me the same eigenvector (weights) irrespective of what the two variables are. It's always [.70710678, .70710678]. I find this strange. Of course, the eigenvalues differ.  My question is: How to interpret this?    PS. I wanted to conduct a total least squares regression on two variables. My statistical programme does not provide TLS, but TLS luckily equals Principal Component Analysis, as far as I know. Hence my question. The question is not about TLS directly, but why I get the same eigenvectors irrespective of which variables I use (as long as they are exactly 2).
67276
I have a big dataset and I want to perform a dimensionality reduction.   Now everywhere I read that I can use PCA for this. However, I still don't seem to get what to do after calculating/performing the PCA. In R this is easily done with the command princomp.  But what to do after calculating the PCA? If I decided I want to use the first $100$ principal components, how do I reduce my dataset exactly?
94529
I have a question on determining eigenvalues for a given matrix A:  $$ A=  \begin{bmatrix}  2 &amp; 1 &amp; 2 \\  0 &amp; 2 &amp; -1 \\  0 &amp; 1 &amp; 0 \\  \end{bmatrix} $$  Here's what I have so far:  \begin{align} &amp; det(A-\lambda I_3) = 0\\ &amp; =det \begin{bmatrix}  2-\lambda &amp; 1 &amp; 2 \\  0 &amp; 2-\lambda &amp; -1 \\  0 &amp; 1 &amp; 0-\lambda \\  \end{bmatrix} = 0 \\ &amp; = (2-\lambda) \begin{vmatrix}  2-\lambda &amp; -1 \\  1 &amp; 0-\lambda \\  \end{vmatrix} \\ &amp; = (2-\lambda)(-2\lambda+\lambda^2+1) \\ &amp; = (2-\lambda)(\lambda-1)^2 = 0\end{align}  So my eigenvalues are $2$ and $1$. I am almost postitive this is correct. Here's my confusion/question. I know that the determinant of an upper triangular matrix is the product of the terms along the diagonal.  Some of my classmates used a different approach where from the beginning they row reduced $A$ into an upper triangular matrix by eliminating the $1$ from the $3$rd row, then subtracted $\lambda I$ and then instead of solving the determinant like I did they just multiplied the diagonal terms together.  The problem I see with this is that you can obtain different matrices depending on how you choose to row reduce, which means you can get different eigenvalues. Is it possible to solve for the eigenvalues by using the approach of making $A$ into an upper triangular matrix first? (So that you can solve the determinant by multiplying the diagonal entries) I hope that makes sense. Thanks!
95482
Suppose A is a 3*3  symmetric matrix s.t.  $$\begin{pmatrix} x &amp; y &amp; 1 \\   \end{pmatrix} A  \begin{pmatrix} x  \\  y\\ 1\end{pmatrix}  = xy -1 $$  let p be the no. of positive eigen value of A and q = rank(A)-p, then p=? q=?  what matrix A should be I can't find??
49275
I have a matrix $A \in M_3(R)$ and it is known that $\sigma (A)=\{-1, 0, 1\}$, where $\sigma (A)$ is a set of eigenvalues of matrix $A$. I am now supposed to calculate $\det(I + A^{100})$.  I know that $A^{100}$ could be calculated using a diagonal matrix which has the eigenvalues of $A$ on it's diagonal and using matrices which are formed using the eigenvectors of $A$, but I am not sure how to get there. Or it might not even be the right approach.  I know there is a similar question, but I don't really understand the answer given there. It's not fully explained. So if anyone could help, that would be great. Thanks
14666
I've read a lot about PCA, including various tutorials and questions (such as , , , and ).   The geometric problem that PCA is trying to optimize is clear to me: PCA tries to find the first principal component by minimizing the reconstruction (projection) error, which simultaneously maximizes the variance of the projected data.    When I first read that, I immediately thought of something like linear regression; maybe you can solve it using gradient descent if needed.  However, then my mind was blown when I read that the optimization problem is solved by using linear algebra and finding eigenvectors and eigenvalues. I simply do not understand how this use of linear algebra comes into play.  So my question is: How can PCA turn from a geometric optimization problem to a linear algebra problem? Can someone provide an intuitive explanation?  I am not looking for an answer like  that says "When you solve the mathematical problem of PCA, it ends up being equivalent to finding the eigenvalues and eigenvectors of the covariance matrix." Please explain why eigenvectors come out to be the principal components and why the eigenvalues come out to be variance of the data projected onto them  I am a software engineer and not a mathematician, by the way.  Note: the figure above was taken and modified from .
52271
I have stumbled across a very old exam question from my linear algebra course and the solutions are not available. I was wondering if my working/logic is correct and if any improvements can be made.     Let  $\ B=\begin{pmatrix}     1 &amp; -1 &amp; 4 \\     6 &amp; -7 &amp; 2 \\     -3 &amp; 1 &amp; -6       \end{pmatrix}\ $ and $\ \vec{v_1}=\begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix}$.      Part (i): Calculate $(B+5I)\vec{v_1}$ and $(B+5I)^2\vec{v_1}$.   Using simply matrix multiplication, I calculated $(B+5I)\vec{v_1}=\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}$ and $(B+5I)^2\vec{v_1}=\vec{0}$.     Part (ii): Find all eigenvalues and eigenvectors of B.   From our calculations in part (i), we can see that $\ \lambda=-5,-5$ are two of the eigenvalues, with corresponding eigenvector $\ \begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}$ and generalised eigenvector $\ \begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix}$. From the trace of B, the remaining eigenvalue is $\lambda=-2$ with corresponding eigenvector $\begin{pmatrix} -2 \\ -2 \\ 1 \end{pmatrix}$.     Part (iii): Giving reasons, write down a basis for the generalised eigenspace $GE_{-5}$ of B.   I am unsure of this part of the question. If I had to guess, I would say a basis for $GE_{-5}=\left\{\begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix},\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}\right\}$. Is this correct?     Part (iv): Solve the initial value problem $\vec{y'}=B\vec{y}$ where $\ B=\begin{pmatrix}     1 &amp; -1 &amp; 4 \\     6 &amp; -7 &amp; 2 \\     -3 &amp; 1 &amp; -6       \end{pmatrix}\ $ and $\vec{y}(0)=\begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix}$.   Well, the solution to this IVP will take the form $\vec{y}=e^{tB}\begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix}$. I noticed that $\begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix}$ was a generalised eigenvector and hence we can write $\vec{y}=e^{-5t}\begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix}+e^{-5t}\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}t$. Thus $\vec{y}=e^{-5t}\begin{pmatrix} 1+t \\ 1+2t \\ -1-t \end{pmatrix}$
69130
I'm learning PCA in R language. I met two problems right now that I don't understand.  I am performing a PCA analysis in R on a 318×17 dataset using some custom code.  I take eigen function in R  to find eigenvalues and eigenvectors.  But my 1st and 3rd eigenvectors are of the opposite sign to my handbook.  My second eigenvectors is almost the same.  I know that given a square matrix A, the condition that characterizes an eigenvalue, $\lambda$, is the existence of a nonzero vector $x$ such that $Ax=\lambda x$; this equation can be rewritten as follows: $(A - \lambda)x=0$. Now I calculate covariance of my data and have eigenvalues. I want to solve this linear combination equation to find $x$ and compare with initial eigenvectors. When I take solve function in R, my $x$ vector is always zero.   Here are my questions: Why the sign is different? How to use solve function in R  to find a non-zero vector $x$?
83153
Let $A$ be a $n\times n$ matrix such that $a_{ij}=ij$.  I want to find the eigenvalues of the matrix $A$.   Efforts: $n=2$, I found that eigenvalues are equal to $0,5$.  For $n=3$, I found eigenvalues are equal to $0,0,14$. For $n=4$ I found eigenvalues are equal to $0,0,0,30$.  For $n=5$ I found eigenvalues are equal to $0,0,0,0,0,55$.  It looks like there is a pattern here. Eigenvalues of $n\times n$ matrix seems to be $\sum_{i=1}^n i^2, \underbrace{0,0,0,..0}_{n-1\text{ times}}$.  But how can I prove it? I mean whether there are some beautiful arguments that makes things easy, and ideas clear.   Thanks for reading and helping out.
34939
$$A=\begin{array}{ccc} 0 &amp; 1/2 &amp; 0 &amp; \cdots &amp; 0 \\ 1/2 &amp; 0 &amp; 1/2 &amp;\cdots&amp; 0\\ 0 &amp; 1/2 &amp; 0 &amp; \cdots &amp; 0\\ \vdots &amp; \vdots &amp; \vdots &amp;\ddots &amp; 1/2\\ 0 &amp; 0 &amp; 0 &amp; 1/2 &amp; 0 \end{array}$$  I need to know the spectral radius of $A$ in function of the dimension of the matrix $n$. I know only one way to know the spectral radius, using the definition I find all eigenvalues and then find the maximum. but it very hard if $n$ is changing. Some help please!  I did some iteration by   $$det(\lambda I-A)$$  $$n=2 \quad r=\frac{\sqrt{1}}{2}$$ $$n=3 \quad r=\frac{\sqrt{2}}{2}$$  for $n=2$ and $n=3$ are correct.  and if we take this rule, for   $$n=5\quad \quad r=\frac{\sqrt{4}}{2}$$  but this isn´t true since I know radius for this matrix should be $&lt; 1$ and if we take this rule  for $n=5$ we have  $$\frac{\sqrt4}{2}=1$$   Some hint?
67460
For t∈R, let $A_t = \left( \begin{array}{ccc} t &amp; 1 &amp; 1 \\ 1 &amp; t &amp; 1 \\ 1 &amp; 1 &amp; t \end{array} \right) $. Find the Eigenvalues and Eigenvectors. Is $A_t$ diagonalizable for all t?  So I am thinking of approaching it like this. First, I find the characteristic polynomial of $A_t$ and use that to find the eigenvalues and their corresponding eigenvectors. Then I check if those eigenvectors are linearly independent for all t. If they are linearly independent, then $A_t$ is diagonalizable for all t, and if they're not, $A_t$ is not diagonalizable for all t.  Should I pursue this problem like this? Or is there a more efficient method? Thanks in advance!
2545
(a) Given the column vector $a$ as an $n\times 1$ matrix. Write out its singular value decomposition, showing the matrices $U, \sum$, and $V$ explicitly.  (b) Given the row vector $a^T$ as an $1\times n$ matrix. Write out its singular value decomposition, showing the matrices $U, \sum$, and $V$ explicitly.  I currently get stucked on this question, since I couldn't find the eigenvalues of the $n\times n$ matrix $aa^T$ in part (a), and $a^Ta$ in part (b) to find the columns of $U$ and $V$, respectively. Can someone please help me on this problem?
32991
what is the signature of this matrix:  $\begin{pmatrix} -3&amp;0&amp;-1 \\0&amp;-3&amp;0 \\ -1&amp;0&amp;-1 \end{pmatrix}$ ?  I tried calculating them without eigenvalues; this should be done via using Gauß, right?  However, I don't see a way to get the signature by using simultaneously raw and line-transformations.. ?
97919
I have carried out a principal components analysis of six variables $A$, $B$, $C$, $D$, $E$ and $F$. If I understand correctly, unrotated PC1 tells me what linear combination of these variables describes/explains the most variance in the data and PC2 tells me what linear combination of these variables describes the next most variance in the data and so on.   I'm just curious -- is there any way of doing this "backwards"? Let's say I choose some linear combination of these variables -- e.g. $A+2B+5C$, could I work out how much variance in the data this describes?
87268
Our teacher didn't explain us how to find it so I've had to look up a bit by myself.  I have this matrix :  $$A = \begin{pmatrix} 9 &amp; 4 &amp; 5 \\ -4 &amp; 0 &amp; -3 \\ -6 &amp; -4 &amp; -2 \end{pmatrix}$$  I've found its characteristic polynomial $p_A(\lambda) = -(\lambda -3)(\lambda - 2)^2$.  I've found its eigenspaces : $E_2 = span(\begin{pmatrix}-2 &amp; 1 &amp; 2\end{pmatrix}^T)$, $E_3 = span(\begin{pmatrix}-3 &amp; 2 &amp; 2\end{pmatrix}^T)$.  And I've found the generalized eigenspace of $\lambda = 2$ :  $\ker((A-2I_3)^2) = span(\begin{pmatrix}0 &amp; 1 &amp; 0\end{pmatrix}^T, \begin{pmatrix}1 &amp; 0 &amp; -1\end{pmatrix}^T)$  But I don't know what to do now. I've tried to see what result I would get with $$M = \begin{pmatrix} -3 &amp; -2 &amp; 0 \\ 2 &amp; 1 &amp; 1 \\ 2 &amp; 2 &amp; 0 \end{pmatrix}.$$  But $$M^{-1}AM = \begin{pmatrix} 3 &amp; 0 &amp; 0 \\ 0 &amp; 2 &amp; -2 \\ 0 &amp; 0 &amp; 2 \end{pmatrix}.$$  So there it is. Any help ?
9485
Define $T : M_{n×n}(\mathbb{R}) → M_{n×n}(\mathbb{R})$ by $T(A) := A^t$.   I know that the corresponding eigenvalues are $+1$ and $-1$, but I'm not sure how to find the eigenvectors of this transformation, in the case of a $2\times 2$ matrix it's simple, but not in the general case.