I try to understand why normal distribution is an assumption for PCA and what might happens when it is violated. I found one answer on this plattform and a lot of different answers in literature. It varies between "there is no need for normal distribution as long as the data variables are linear correlated " and "normal distribution is essential, if violated PCA can go really wrong". I would be really happy if somebody could explane the connection between normal distribution and PCA.
121301
I have a couple of quick questions about PCA:    Does the PCA assume that the dataset is Gaussian?   What happens when I apply a PCA to inherently non-linear data?   Given a dataset, the process is to first mean-normalize, set the variance to 1, take an SVD, reduce rank, and finally map the dataset into the new reduced-rank space. In the new space, each dimension corresponds to a "direction" of maximal variance.     But is the correlation of that dataset in the new space always zero, or is that only true for data that is inherently Gaussian?   Suppose I have two datasets, "A" and "B", where "A" corresponds to randomly sampled points taken from a Gaussian, while "B" corresponds to points randomly sampled from another distribution (say Poisson).    How does the PCA(A) compare to the PCA(B)?   By looking at the points in the new space, how would I determine that the PCA(A) corresponds to points sampled from a Gaussian, while PCA(B) corresponds to points sampled from a Poisson?   Is the correlation of the points in "A" 0?   Is the correlation of points in "B" also 0?   More importantly, am I asking the "right" question?   Should I look at the correlation, or is there another metric that I should consider?
65355
Why did the normal distribution become such a popular (important) distribution? I know one reason is because of CLT. Can you please give more reasons?
116677
Long ago I learnt that normal distribution was necessary to use a two sample T-test. Today a colleague told me that she learnt that for N>50 normal distribution was not necessary. Is that true?  If true is that because of the central limit theorem?
44722
Someone told me that, Gaussian distribution is conjugate to distribution because a Gaussian times a Gaussian would still be Gaussian distribution ?  Why is that ? Say the following situation: $X\sim N(\mu_x,\sigma^2_{x})$ , $Y\sim N(\mu_y,\sigma^2_{y})$  Would a new variable, $Z=XY$ be normally distributed ??
45073
Normally in principal component analysis (PCA) the first few PCs are used and the low variance PCs are dropped, as they do not explain much of the variation in the data.  However, are there examples where the low variation PCs are useful (i.e. have use in the context of the data, have an intuitive explanation, etc.) and should not be thrown away?
8209
This question just occurred to me out of the blue. PCA is a way to reduce dimensions. Another way that is often (perhaps too often) used is to take mean values of two or more variables. This is done a great deal to create "scales" or "ratings" that get published in the popular press about various political issues. Dimensions are naively "averaged" and this cumulative "average" is portrayed as a magical one-number-sums-all figure.  Is this, nevertheless, a very crude relative of PCA? If so, what assumptions does it have? I can think of the following:   Sphericity is perfect or near-perfect. All variables are perfectly orthogonal from all other variables. Variables are all perfectly scaled against each other.   What else would there be?
34567
In an introductory stats book by Nicole Radziwell "Statistics the easy way with R" , an assumption used  for nearly every statistical test (e.g.t-tets, anova, etc) is that the sample size should not be more than 10% of the population size (where the population size in known).   I havent seen it mentioned in other introductory stats books. My question is: how important is theis assumotion and what is the reason for it?  The author didn't say where this comes from (at least as far as I could see). A bit of googling suggests that it refers to making a simple random sample without replacement, if the sample size is above 10% then the observations wont be (approximately) independent, and indepedence of observations is an assumption stated in most textbooks.   In the case where you have a small population (e.g. &lt;50), should you give up on this assumption because it results in sample sizes too small to get decent estimates of variability and to check other assumptions such as normality.    Thank you for any insights.
95190
I came across this nice tutorial:  on how to do PCA in R language. I don't understand the interpretation of Figure 13.3:    So I am plotting first eigenvector vs the second eigenvector. What does that mean? Suppose eigenvalue corresponding to first eigenvector explains 60% of variation in data set and second eigenvalue-eigenvector explain 20% of variation. What does it mean to plot these against each other?
33211
I just would like to understand some information about the joint normality and the margins. I read that the normal joint distribution almost always implies that the univariate margins are all normal.   My question is:  1- Is this apply for Gaussian mixture models? For example, in the Gaussian mixture model, each mixture component follows a Gaussian distribution. Hence, does it imply that each margin will be a Gaussian distribution as well? "I understand that mixture components are used to describe the dependency structures, and they are the joint distribution. is that correct?  2- I read this from one answer to a question in this site . "So yes, the assumption of joint normality is a sufficient condition for all marginal distributions to be normal, irrespective of the dependence structure. Hence, the theory of Copulas does not affect this result in any way."  However, I really do not understand this: irrespective of the dependence structure. As I understand if the joint distribution in normal then the dependence structure is normal.   Could someone help me with my questions, please?
39131
Say I have $X \sim \mathcal N(a, b)$ and $Y\sim \mathcal N(c, d)$.  Is $XY$ also normally distributed?  Is the answer any different if we know that $X$ and $Y$ are independent?
40997
I have come across 3 questions on the title subject.   Why is it necessary to do a normality test? To check if data is imbalanced or not? Are these 4 methods of checking if the data follows normal distribution criteria both applicable to numerical and categorical variable? I am trying to check if the data follows normal distribution by following 4 methods.   Checking Distribution Drawing Box Plot Drawing QQ Plot Use skewness, kurtosis criteria  Skewness for Normal Dist is 0, Kurtosis for Normal Dist is 3. Is there a certain bound that I can use to guarantee that the data is normally distributed? (such as, 0 +/- 1 OR 3 +/- 1)
5870
As an assumption of linear regression, the normality of the distribution of the error is sometimes wrongly "extended" or interpreted as the need for normality of the y or x.  Is it possible to construct a scenario/dataset that where the X and Y are non-normal but the error term is and therefore the obtained linear regression estimates are valid?
64684
I've always taken issue with, and never been given a good answer, for how it is possible that the central limit theorem - the classical version where the distribution of sample means approaches normality - can apply to say a Poisson or Gamma distribution, where $P(x&lt;0)=0$. Or, for that matter, any other distribution for which $\exists X:X \neq -\infty ,F(X)=0$, or perhaps $\exists X:X \neq \infty, 1-F(X)=0$.  As an example, given a Gamma distribution, as the number of samples $n \rightarrow \infty$, $P( \bar{X} = \alpha) \rightarrow 1$, $\forall \alpha \geq 0$, for some $\bar{X}_i$. But if $\alpha&lt;0$, $P(\bar{X}=\alpha)=0$. There simply will never, EVER be an $\bar{X}_i&lt;0$. This suggests to me that the distribution of $\bar{X}$ cannot be, nor approach, normality because $f(\bar{X})$ must necessarily be $0$, $\forall \bar{X}&lt;0$, which does not meet the requirements of a normal distribution where $f(y)&gt;0, \forall y \in R$.  I'd feel much better about life and anything based on the CLT if someone could help me understand where my logic has gone astray.
83437
Background: I'm giving a presentation to colleagues at work on hypothesis testing, and understand most of it fine but there's one aspect that I'm tying myself up in knots trying to understand as well as explain it to others.  This is what I think I know (please correct if wrong!)   Statistics that would be normal if variance was known, follow a $t$-distribution if the variance is unknown CLT (Central Limit Theorem): The sampling distribution of the sample mean is approximately normal for sufficiently large $n$ (could be $30$, could be up to $300$ for highly skewed distributions) The $t$-distribution can be considered Normal for degrees of freedom $&gt; 30$   You use the $z$-test if:   Population normal and variance known (for any sample size) Population normal, variance unknown and $n&gt;30$ (due to CLT) Population binomial, $np&gt;10$, $nq&gt;10$   You use the $t$-test if:   Population normal, variance unknown and $n&lt;30$ No knowledge about population or variance and $n&lt;30$, but sample data looks normal / passes tests etc so population can be assumed normal   So I'm left with:    For samples $&gt;30$ and $&lt;\approx 300$(?), no knowledge about population and variance known / unknown.   So my questions are:   At what sample size can you assume (where no knowledge about population distribution or variance) that the sampling distribution of the mean is normal (i.e. CLT has kicked in) when the sampling distribution looks non-normal? I know that some distributions need $n&gt;300$, but some resources seem to say use the $z$-test whenever $n&gt;30$... For the cases I'm unsure about, I presume I look at the data for normality. Now, if the sample data does looks normal do I use the $z$-test (since assume population normal, and since $n&gt;30$)? What about where the sample data for cases I'm uncertain about don't look normal? Are there any circumstances where you'd still use a $t$-test or $z$-test or do you always look to transform / use non-parametric tests? I know that, due to CLT, at some value of $n$ the sampling distribution of the mean will approximate to normal but the sample data won't tell me what that value of $n$ is; the sample data could be non-normal whilst the sample mean follows a normal / $t$.  Are there cases where you'd be transforming / using a non-parametric test when in fact the sampling distribution of the mean was normal / $t$ but you couldn't tell?
13641
I am a grad student developing an interest for statistics. I like the material over-all, but I sometimes have a hard time thinking about applications to real life. Specifically, my question is about commonly used statistical distributions (normal - beta- gamma etc.). I guess for some cases I get the particular properties that make the distribution quite nice - memoryless property of exponential for example. But for many other cases, I don't have an intuition about both the importance and application areas of the common distributions that we see in textbooks.   There are probably a lot of good sources addressing my concerns, I would be glad if you could share those. I would be a lot more motivated into the material if I could associate it with real-life examples.
2977
In general, what is meant by saying that the fraction $x$ of the variance in an analysis like PCA is explained by the first principal component? Can someone explain this intuitively but also give a precise mathematical definition of what "variance explained" means in terms of principal component analysis (PCA)?  For simple linear regression, the r-squared of best fit line is always described as the proportion of the variance explained, but I am not sure what to make of that either. Is proportion of variance here just the extend of deviation of points from the best fit line?
13436
The Box-Cox transformation transforms our data into a normal distribution.  How is that even a proper technique? What if our data didn't come from a normal distribution? How could someone just blindly apply the Box-Cox transformation?  To re-phrase: why apply the Box-Cox transformation if our data isn't normally distributed?  Is the Box-Cox transformation used when our data SHOULD be normally distributed, but isn't?