The sample variance of $m$ samples from a gaussian is  $$ \hat{\sigma}^2_m=\frac{1}{m}\sum_{i=1}^m(x^{(i)}-\hat{\mu}_m)^2$$ How do i show that the sample variance $\hat{\sigma}_m^2$ is biased ? I.e  $$E[\hat{\sigma}_m^2] = E[\frac{1}{m}\sum_{i=1}^m(x^{(i)} - \hat{\mu}_m)^2] = \frac{m-1}{m}\sigma^2$$
122292
Let $\{ X_i | i = 1, 2, . . . ,n \}$ be a sequence of independent and identically distributed (IID) random variables from a population and define $\mu \equiv \mathbb{E}(X)$ and $\sigma^2 \equiv \mathbb{V}(X)$. Suppose we think that the mean is $\mu = \mu_0$ for some number $\mu_0$ (but we may be wrong).  Find the bias in the estimator:  $$\tilde{\sigma}^2 \equiv \frac{1}{n} \sum_{i=1}^n (X_i - \mu_0)^2,$$  as a function of $\mu$.  When is this estimator unbiased for $\sigma^2$?  I know that when we find this estimator using $\bar{X}$, it is biased and we must divide it by $n-1$ instead of $n$ to get an unbiased estimator.  But how does one do it for a number such as $\mu_0$?
102707
I would like to test $H_{0} :$ $\mu_{1} = \mu_{2}, \ \sigma_{1}^{2} = \sigma_{2}^{2}$ vs $H_{0} :$ $\mu_{1} \neq \mu_{2}, \ \sigma_{1}^{2} \neq \sigma_{2}^{2}$Is there a test statistic that would allow me to detect both differences in the mean and differences in the variance? Is there a way that I could possibly combine the $F$ - $test$ and the $t$ - $test$ to derive this test statistic? I am assuming that the population is normally distributed but what if it wasn't?I am very stuck so any help would be greatly appreciated.
9536
Given  $X \sim \mathcal{N}(\mu,\,\sigma^{2})$, what is the covariance between $X$ and $e^X$?
100201
Consider the following simple linear regression model involving the $\epsilon_i$ error term,  $$y_i = \alpha + \beta x_i + \epsilon_i$$  such that,  $$\epsilon_i \sim \mathcal N(0,\sigma^2)$$  we know that if $Z_1, Z_2, ..., Z_n$ are independent, standard normal random variables,   $$Z_i \sim \mathcal N(0,1)$$  then the sum of their squares is distributed according to the chi-square distribution with $n$ degrees of freedom,  $$\sum _{i=1}^{n} Z_{i}^{2} \sim \chi^{2}_{n}$$  Now let's prove that $\frac{(n-2)s^2}{\sigma^2}\sim \chi^{2}_{n-2}$ such that,  $$s^2 = \frac{\sum _{i=1}^{n} \hat{\epsilon}_i^2}{n-2}$$  Proof:  $\require{cancel}$  $$\epsilon_i \sim \mathcal N(0,\sigma^2)$$  $$\frac{\epsilon_i - \cancelto{0}{\overline{\epsilon}}}{\sigma} \sim \mathcal N(0, 1)$$  $$\sum _{i=1}^{n} \left(\frac{\epsilon_i}{\sigma}\right)^2 \sim \chi^{2}_{n}$$  $$\sum _{i=1}^{n} \frac{\epsilon_i^2}{\sigma^2} \sim \chi^{2}_{n}$$  How do I now move from this expression $\sum _{i=1}^{n} \frac{\epsilon_i^2}{\sigma^2}$ which uses the error term $\epsilon_i$ to this expression $\sum _{i=1}^{n} \frac{\hat{\epsilon}_i^2}{\sigma^2}$ which uses the residual term $\hat{\epsilon}_i$?  Note:  I came across this  but I got lost in the second part of the demonstration and I wanted to know if there is a simpler method that would lead to the same answer.
50755
When calculating a sample variance a factor of $N-1$ appears instead of $N$ (see this  ).  Does anybody have an intuitive way of explaining this to students who need to use this fact but maybe haven't taken a statistics course?
1439
The denominator of the (unbiased) variance estimator is $n-1$ as there are $n$ observations and only one parameter is being estimated.  $$ \mathbb{V}\left(X\right)=\frac{\sum_{i=1}^{n}\left(X_{i}-\overline{X}\right)^{2}}{n-1} $$  By the same token I wonder why shouldn't the denominator of covariance be $n-2$ when two parameters are being estimated?  $$ \mathbb{Cov}\left(X, Y\right)=\frac{\sum_{i=1}^{n}\left(X_{i}-\overline{X}\right)\left(Y_{i}-\overline{Y}\right)}{n-1} $$
72308
I've been reading over this , trying to make sense of how the mean and variance of a gaussian conditional was derived. I've come to accept that unless I allocate a dozen or so hours to refreshing my linear algebra knowledge, it's out of my reach for the time being. that being said, I'm looking for a conceptual explanation for that these equations represent: $$\mu_{1|2} = \mu_1 + \Sigma_{1,2} * \Sigma^{-1}_{2,2}(x_2 - \mu_2)$$ I read the first as &quot;Take $\mu1$ and augment it by some factor, which is the covariance scaled by the precision (measure of how closely $X_2$ is clustered about $\mu_2$, maybe?) and projected onto the distance of the specific $x_2$ from $mu_2$.&quot; $$\Sigma_{1|2} = \Sigma_{1,1} - \Sigma_{1,2} * \Sigma^{-1}_{2,2} * \Sigma_{1,2}$$ I read the second as, &quot;take the variance about $\mu_1$ and subtract some factor, which is covariance squared scaled by the precision about $x_2$.&quot; In either case, the precision $\Sigma^{-1}_{2,2}$ seems to be playing a really important role. A few questions:  Am I right to treat precision as a measure of how closely observations are clustered about the expectation? Why is the covariance squared in the latter equation? (Is there a geometric interpretation?) So far, I've been treating $\Sigma_{1,2} * \Sigma^{-1}_{2,2}$ as a ratio, (a/b), and so this ratio acts to scale the (second) $\Sigma_{1,2}$, essentially accounting for/damping the effect of the covariance; I don't know if this is valid. Anything else you'd like to add/clarify?
7840
Let random vector $x = (x_1,...,x_n)$ follow multivariate normal distribution with mean $m$ and covariance matrix $S$. If $S$ is symmetric and positive definite (which is the usual case) then one can generate random samples from $x$ by first sampling indepently $r_1,...,r_n$ from standard normal and then using formula $m + Lr$, where $L$ is the Cholesky lower factor so that $S=LL^T$ and $r = (r_1,...,r_n)^T$.  What about if one wants samples from singular Gaussian i.e. $S$ is still symmetric but not more positive definite (only positive semi-definite). We can assume also that the variances (diagonal elements of $S$) are strictly positive. Then some elements of $x$ must have linear relationship and the distribution actually lies in lower dimensional space with dimension $&lt;n$, right?  It is obvious that if e.g. $n=2, m = \begin{bmatrix} 0 \\ 0 \end{bmatrix}, S = \begin{bmatrix} 1 &amp; 1 \\ 1 &amp; 1\end{bmatrix}$ then one can generate $x_1 \sim N(0,1)$ and set $x_2=x_1$ since they are fully correlated. However, is there any good methods for generating samples for general case $n&gt;2$? I guess one needs first to be able identify the lower dimensional subspace, then move to that space where one will have valid covariance matrix, then sample from it and finally deduce the values for the linearly dependent variables from this lower-dimensional sample. But what is the best way to that in practice? Can someone point me to books or articles that deal with the topic; I could not find one.
88776
I have quite a simple question but I can't for the life of me figure it out.  For a set of iid samples $\,\,X_1, X_2, \ldots, X_n\,\,$ from distribution with mean $\,\mu$.  If you are given the sample variance as  $$ S^2 = \frac{1}{n-1}\sum\limits_{i=1}^n \left(X_i - \bar{X}\right)^2 $$  How can you write the following?  $$ S^2 = \frac{1}{n-1}\left[\sum\limits_{i=1}^n \left(X_i - \mu\right)^2 - n\left(\mu - \bar{X}\right)^2\right] $$  All texts that cover this just skip the details but I can't work it out myself. I get stuck after expanding like so  $$ S^2 = \frac{1}{n-1}\sum\limits_{i=1}^n \left(X_i^2 -2X_i\bar{X} + \bar{X}^2\right) $$  What am I missing?  Edit: A similarly equivalent expression is often given that I also can't derive but which may be more obvious is  $$  S^2 = \frac{1}{n-1}\left[\sum\limits_{i=1}^n X_i^2 - n\bar{X}^2\right] $$
80564
I am unable to find the bias of the sample variance estimator in this problem. Unfortunately I keep coming up with   Question: Suppose that the true linear model is   $y = X_1\beta_1 + X_2\beta_2 + e$,   where $e \sim MVN(0,\sigma^2 I)$, $X_1: N \times p_1, \beta_1: p_1 \times 1, X_2: N \times p_2, \beta_2: p_2 \times 1$. But we wrongly fit the model  $y = X_1\beta_1+e$ and calculate the estimators of $\beta_1$ and $\sigma^2$ as:  $\hat{\beta}_1 = (X_1'X_1)^{-1}X_1'y$,  $s^2 = \large \frac{y'[I-X_1(X_1'X_1)^{-1}X_1']y}{N-p_1}$.  (a) Show that, in general, $\hat{\beta}_1$ is a biased estimator with  Bias$(\hat{\beta}_1) = (X_1'X_1)^{-1}X_1'X_2\hat{\beta}_2$  Under what condition on $X_1$ and $X_2$ is $\hat{\beta}_1$ unbiased?  (b) Show that, in general, $s^2$ is a biased estimator with   Bias$(s^2) = \frac{1}{N-p_1}\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_2\beta_2$  Is the bias positive or negative? Under what condition on $\beta_2$ is $s^2$ unbiased?  Attempt:  For part (a):  Bias$(\hat{\beta}_1) = E((X_1'X_1)^{-1}X_1'y - \beta_1)$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'E(y) - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'(X_1\beta_1 + X_2\beta_2) - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'X_1\beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2) - \beta_1$  $\hspace{15mm}= I\beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2 - \beta_1$  $\hspace{15mm}= \beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2 - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'X_2\beta_2$  If $X_1$ and $X_2$ are orthogonal, then $\hat{\beta}_1$ is unbiased.  For part (b), I am unable to figure out how to derive:  Bias$(s^2) = \frac{1}{N-p_1}\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_2\beta_2$  I can get   $\frac{1}{N-p_1}\beta_1'X_1'+\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_1\beta_1+X_2\beta_2$  after taking the expectation of y on both sides.   Any help would be much appreciated.
80900
I'm working through an econometrics textbook and it's proving that   $$ \sigma^2 = E(\hat{\sigma}^2) = \frac{SSR}{n-2} $$  I followed the proof (an example of which is shown on ) until it reaches this step:  \begin{align} E\left[\displaystyle\sum\limits_{i=1}^n \hat{u}_{i}^{2}\right] &amp;= E\left[\displaystyle\sum\limits_{i=1}^n (u_{i}-\bar{u})^{2}\right] + (\hat{\beta}_{1}-\beta_{1})^{2} E\left[\displaystyle\sum\limits_{i=1}^n (x_{i} - \bar{x})^2\right] \\ &amp;+ 2 (\hat{\beta}_{1}-\beta _{1}) E\left[\displaystyle\sum\limits_{i=1}^n (u_{i}-\bar{u})(x_{i}-\bar{x})\right] \\ &amp;= \dots \end{align}  The book says that:   The expected value of the first term should be $(n-1)\sigma^2$ The expected value of the second term should be $\sigma^2$. The expected value of the third term should be $2 \sigma^2$ because the term itself can be written as $2(\hat{\beta}_1-\beta_1)^2 \hat{\sigma}_x^2$.   For the first term, I can work backwards and get that:  \begin{align} \hat{\sigma}^2 &amp;= \frac{1}{n-1} \displaystyle\sum\limits_{i=1}^n (u_{i}-\bar{u})^{2} \\ &amp;= \sigma^2 \end{align}  but it seems like this only works if we assume that $\hat{\sigma}^2$ is an unbiased estimator.  For the second term:  \begin{align} (\hat{\beta}_{1}-\beta_{1})^{2} E\left[\displaystyle\sum\limits_{i=1}^n (x_{i} - \bar{x})^2\right] &amp;= (\hat{\beta}_{1}-\beta_{1})^{2} E\left[(n-1) \hat{\sigma}_x^2\right] \\  &amp;= (\hat{\beta}_{1}-\beta_{1})^{2} (n - 1) E\left[\hat{\sigma}_x^2\right] \\ &amp;= (\hat{\beta}_{1}-\beta_{1})^{2} (n - 1) \hat{\sigma}_x^2 \end{align}  but this doesn't match what the text shows, so I'm a bit confused as to the final step for this term.   For the third term: \begin{align} 2 (\hat{\beta}_{1}-\beta _{1}) E\left[\displaystyle\sum\limits_{i=1}^n (u_{i}-\bar{u})(x_{i}-\bar{x})\right] &amp;= 2 (\hat{\beta}_{1}-\beta _{1}) E\left[\displaystyle\sum\limits_{i=1}^n u_{i}(x_{i}-\bar{x})\right] \\ &amp;= 2 (\hat{\beta}_{1}-\beta_{1}) \displaystyle\sum\limits_{i=1}^n E\left[u_{i}(x_{i}-\bar{x})\right] \\ &amp;= 2 (\hat{\beta}_{1}-\beta_{1}) \displaystyle\sum\limits_{i=1}^n E\left[u_{i}\right]E\left[(x_{i}-\bar{x})\right] \\ \end{align}  I would have thought that because $E[u_i] = 0$, this term would cancel out, so I'm not sure how to derive $2\sigma^2$ from it.  Can someone give me a hint in the right direction for these steps? I'm firmly trying to work through all of the proofs myself with only a little guidance from the book, but I'm not quite there yet.  I noticed , which links to , so maybe with more reading I'll see where to apply that. (If that isn't it, hints are welcome, but I'll still work through those questions to see if it helps)
97543
Suppose I have the following model: $$Y = \mu + \epsilon = X\beta + \epsilon,$$ where $Y$ is $n \times 1$, $X$ is $n \times p$, $\beta$ is $p \times 1$, and $\epsilon$ is $n \times 1$. I assume that $\epsilon$ are independent with mean 0 and variance $\sigma^2I$. In OLS, the fitted values are $\hat{Y} = HY$, where $H = X(X^TX)^{-1}X^T$ is the $N \times N$ hat matrix. I want to find the MSE of $\hat{Y}$. By the bias-variance decomposition, I know that \begin{align*} MSE(\hat{Y}) &amp;= bias^2(\hat{Y}) + var(\hat{Y})\\ &amp;= (E[HY] - \mu)^T(E[HY] - \mu) + var(HY)\\ &amp;= (H\mu - \mu)^T(H\mu - \mu) + \sigma^2H\\ &amp;= 0 + \sigma^2H \end{align*} I'm confused by the dimension in the last step. The $bias^2$ term is a scalar. However, $var(\hat{Y})$ is an $N \times N$ matrix. How can one add a scalar to an $N \times N$ matrix where $N \neq 1$?
36151
It's a standard result that given $X_1,\cdots ,X_n $ random sample from $N(\mu,\sigma^2)$, the random variable $$\frac{(n-1)S^2}{\sigma^2}$$ has a chi-square distribution with $(n-1)$ degrees of freedom, where $$S^2=\frac{1}{n-1}\sum^{n}_{i=1}(X_i-\bar{X})^2.$$ I would like help in proving the above result. Thanks.
71663
I just read this very insightful post , where the author stated that the variance of $\hat\beta$ is:  $$\text{var}(\hat\beta) = \sigma^2(\textbf{X}^\prime \textbf{X})^{-1}.$$  I couldn't figure out why it is like this. Can anyone elaborate a bit?
28039
Let $\mathbf{X}\sim \mathcal{N}(0,I)$ be an $n-$dimensional standard Gaussian vector. We know that  $$\begin{align} ||\mathbf{X}||^2&amp;=\sum_{i=1}^nX_i^2\sim\chi^2_n\\ ||\mathbf{X}||&amp;=\sqrt{\sum_{i=1}^nX_i^2}\sim\chi_n\\ \end{align}$$  Suppose the components are correlated, i.e., we consider $\mathbf{Y}\sim \mathcal{N}(0,\Sigma)$ where $\Sigma$ is positive definite. We know that $||\mathbf{Y}||^2$ has the . Is there also a corresponding generalized chi distribution for $||\mathbf{Y}||$? If so, is there some asymptotic result on the mean of such a distribution for $n\to\infty$? This is related to       But here the components of the random vector are neither independent or identically distributed.
93236
Consider $N$ independent samples $S$ obtained from a random variable $X$ that is assumed to follow a truncated distribution (e.g. a ) of known (finite) minimum and maximum values $a$ and $b$ but of unknown parameters $\mu$ and $\sigma^2$. If $X$ followed a non-truncated distribution, the maximum likelihood estimators $\widehat\mu$ and $\widehat\sigma^2$ for $\mu$ and $\sigma^2$ from $S$ would be the sample mean $\widehat\mu = \frac{1}{N} \sum_i S_i$ and the sample variance $\widehat\sigma^2 = \frac{1}{N} \sum_i (S_i - \widehat\mu)^2$. However, for a truncated distribution, the sample variance defined in this way is bounded by $(b-a)^2$ so it is not always a consistent estimator: for $\sigma^2 &gt; (b-a)^2$, it cannot converge in probability to $\sigma^2$ as $N$ goes to infinity. So it seems that $\widehat\mu$ and $\widehat\sigma^2$ are not the maximum-likelihood estimators of $\mu$ and $\sigma^2$ for a truncated distribution. Of course, this is to be expected since the $\mu$ and $\sigma^2$ parameters of a truncated normal distribution aren't its mean and variance.  So, what are the maximum likelihood estimators of the $\mu$ and $\sigma$ parameters of a truncated distribution of known minimum and maximum values?
19793
The statement  The sampling distribution of the sample variance is a chi-squared distribution with degree of freedom equals to $n-1$, where $n$ is the sample size (given that the random variable of interest is normally distributed).    My intuition  It kinda makes intuitive sense to me 1) because a chi-square test looks like a sum of square and 2) because a Chi-squared distribution is just a sum of squared normal distribution. But still, I don't have a good understanding of it.  Question  Is the statement true? Why?