According to , the following estimator of the standard deviation $$s=\sqrt{\frac{1}{n-1}\sum_{k=1}^n(x_i-\bar{x})^2}$$  for a normal variable, verifies $E[s]=C_4(n) \sigma$, where  $$ C_4(n) = \sqrt{\frac{2}{n-1}}\cdot\frac{\Gamma(\frac{n}{2})}{\Gamma(\frac{n-1}{2})}=1 - \frac{1}{4n} - \frac{7}{32n^2} - \frac{19}{128n^3} + O(n^{-4}) $$  Therefore we have an unbiased estimator for the standard deviation of a normal variable,  $$\hat{\sigma}_\text{unbiased} = \frac{1}{C_4(n)} \sqrt{\frac{1}{n-1}\sum_{k=1}^n(x_i-\bar{x})^2}$$  How could you prove this?
42859
According to the Wikipedia article on  the sample SD   $$s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}$$  is a biased estimator of the SD of the population. It states that $E(\sqrt{s^2}) \neq \sqrt{E(s^2)}$.   NB. Random variables are independent and each $x_{i} \sim N(\mu,\sigma^{2})$   My question is two-fold:   What is the proof of the biasedness? How does one compute the expectation of the sample standard deviation    My knowledge of maths/stats is only intermediate.
51291
Let $\hat{\theta}_n= -\frac{n}{\sum_{i=1}^n \log(X_i)}$, where $X_i$ are i.i.d. samples from distribution with pdf $\theta x^{\theta-1}$ for $x \in (0,1)$. How to prove that $\hat{\theta}_n$ is consistent estimator of $\theta$.
55129
Suppose $X_1, X_2,...,X_n$ be i.i.d from $N(\mu,\sigma^2)$ with unknown $\mu \in \mathcal R$ and $\sigma^2&gt;0$  Let $Z=\frac{X_1-\bar{X}}{S},$ S is the standard deviation here.  It is can be shown that  $Z$ has the Lebesgue p.d.f.  $$f(z)=\frac{\sqrt{n} \Gamma\left(\frac{n-1}{2}\right)}{\sqrt{\pi}(n-1)\Gamma\left(\frac{n-2}{2}\right)}\left[1-\frac{nz^2}{(n-1)^2}\right]^{n/2-2}I_{(0,(n-1)/\sqrt{n})}(|Z|)$$  My question is then how to get this pdf?  The question is from  in example 3.3.4  to find the UMVUE of $P(X_1 \le c)$. I can understand the logic and procedures to find the UMVUE but don't know how to get the pdf.  I think this question also relate to this    Thank you very much for help or point to any related references will be also appropriated.
46984
I am totally confused: On the one hand you can read all kinds of explanations why you have to divide by n-1 to get an unbiased estimator for the (unknown) population variance (degrees of freedom, not defined for sample size 1 etc.) - see e.g.  or .  On the other hand when it comes to variance estimation of a supposed normal distribution all of this doesn't seem to be true anymore. There it is said that the maximum likelihood estimator for variance includes only a division by n - see e.g. .  Now, can anyone please enlighten me why it is true here but not there? I mean normality is what most models boil down to (not least due to the ). So is the choice "division by n" yet the best choice for finding the best estimation for the true population variance after all?!?
37734
The formula for a one sample t-test, as given by  is  $t = \dfrac{\bar{x} - \mu_0}{s\big/\sqrt{n}}$  However, in this equation, if the sample standard deviation, $s$, is zero, $t$ will be undefined. Is there any reasonable way to get a $t$ value in this case?
28652
If $X_1,...,X_n$ are iid, what is the mean of the following variable:?  $ \sum (X_i - \bar{X})^2$  I know the answer is $\sigma^2(n-1)$, but how is this calculated in general? If I expand, I get the variables squared, and that turns quite ugly. What's the trick here?
1439
The denominator of the (unbiased) variance estimator is $n-1$ as there are $n$ observations and only one parameter is being estimated.  $$ \mathbb{V}\left(X\right)=\frac{\sum_{i=1}^{n}\left(X_{i}-\overline{X}\right)^{2}}{n-1} $$  By the same token I wonder why shouldn't the denominator of covariance be $n-2$ when two parameters are being estimated?  $$ \mathbb{Cov}\left(X, Y\right)=\frac{\sum_{i=1}^{n}\left(X_{i}-\overline{X}\right)\left(Y_{i}-\overline{Y}\right)}{n-1} $$
86916
A probably very easy computational question, but I don't really understand how it's done:  I try to compute the point estimates of a normal distribution as $N \sim (\beta, \sigma)$. Using the method of moments, we get:  $$\frac{1}{n} \sum_{i=1}^{n} X_{i} = \hat{\beta}$$  $$\frac{1}{n} \sum_{i=1}^{n} X_{i}^{2} = \hat{\sigma}^2 + \hat{\beta}^2$$  Solving the two equations for the two parameters, we get:  $$\hat{\beta} = \overline{X} $$   and   $$\hat{\sigma}^2 = \frac{1}{n} \sum_{i=1}^{n} (X_{i} - \overline{X}_n)^2 $$  My question is: How do we get to the term of $\hat{\sigma}^2$? If I try to compute it, I get:  $$\hat{\sigma}^2 = \frac{1}{n} \sum_{i=1}^{n} X_{i}^2 - (\frac{1}{n} \sum_{i=1}^{n} X_{i} )^2 $$  Can someone show me how to simplify the last term to what is stated above and what "rule" or law" is used there? Since it is used so often, I really wanna understand this.
3693
I have a set of measurements which is partitioned into M partitions. However, I only have the partition sizes $N_i$ and the means $\bar{x}_i$ from each partition. Because all measurements are assumed to be from the same distribution, I believe I can estimate the mean of the population, $\bar{y}$, and standard deviation of the mean, $\sigma_{mean}$: $$ N=\sum_{i=1}^M N_i $$ $$ \bar{y} = \frac{1}{N}\sum_{i=1}^MN_i\bar{x}_i $$  $$ \sigma_{mean}=\sqrt{\frac{1}{N}\sum_i N_i(\bar{x}_i-\bar{y})^2} $$  My questions:   Am I right in my assumptions, that the mean $\bar{y}$ can be computed as above?  How can I find the standard deviation for the population, given only the means?  I read that the standard deviation of the population and standard deviation of the mean is related with $$ \sigma_{mean}=\frac{\sigma}{\sqrt{n}} \mbox{[1]} $$ where $n$ is the number of samples used in the computation of $\bar{x}_i$. So is it actually as simple as just multiplying $\sigma_{mean}$ with $\sqrt{n}$ if $n$ for all means are the same?   If it's that simple, what do I do if each $\bar{x}_i$ is computed using a different number of samples?    [1]
80900
I'm working through an econometrics textbook and it's proving that   $$ \sigma^2 = E(\hat{\sigma}^2) = \frac{SSR}{n-2} $$  I followed the proof (an example of which is shown on ) until it reaches this step:  \begin{align} E\left[\displaystyle\sum\limits_{i=1}^n \hat{u}_{i}^{2}\right] &amp;= E\left[\displaystyle\sum\limits_{i=1}^n (u_{i}-\bar{u})^{2}\right] + (\hat{\beta}_{1}-\beta_{1})^{2} E\left[\displaystyle\sum\limits_{i=1}^n (x_{i} - \bar{x})^2\right] \\ &amp;+ 2 (\hat{\beta}_{1}-\beta _{1}) E\left[\displaystyle\sum\limits_{i=1}^n (u_{i}-\bar{u})(x_{i}-\bar{x})\right] \\ &amp;= \dots \end{align}  The book says that:   The expected value of the first term should be $(n-1)\sigma^2$ The expected value of the second term should be $\sigma^2$. The expected value of the third term should be $2 \sigma^2$ because the term itself can be written as $2(\hat{\beta}_1-\beta_1)^2 \hat{\sigma}_x^2$.   For the first term, I can work backwards and get that:  \begin{align} \hat{\sigma}^2 &amp;= \frac{1}{n-1} \displaystyle\sum\limits_{i=1}^n (u_{i}-\bar{u})^{2} \\ &amp;= \sigma^2 \end{align}  but it seems like this only works if we assume that $\hat{\sigma}^2$ is an unbiased estimator.  For the second term:  \begin{align} (\hat{\beta}_{1}-\beta_{1})^{2} E\left[\displaystyle\sum\limits_{i=1}^n (x_{i} - \bar{x})^2\right] &amp;= (\hat{\beta}_{1}-\beta_{1})^{2} E\left[(n-1) \hat{\sigma}_x^2\right] \\  &amp;= (\hat{\beta}_{1}-\beta_{1})^{2} (n - 1) E\left[\hat{\sigma}_x^2\right] \\ &amp;= (\hat{\beta}_{1}-\beta_{1})^{2} (n - 1) \hat{\sigma}_x^2 \end{align}  but this doesn't match what the text shows, so I'm a bit confused as to the final step for this term.   For the third term: \begin{align} 2 (\hat{\beta}_{1}-\beta _{1}) E\left[\displaystyle\sum\limits_{i=1}^n (u_{i}-\bar{u})(x_{i}-\bar{x})\right] &amp;= 2 (\hat{\beta}_{1}-\beta _{1}) E\left[\displaystyle\sum\limits_{i=1}^n u_{i}(x_{i}-\bar{x})\right] \\ &amp;= 2 (\hat{\beta}_{1}-\beta_{1}) \displaystyle\sum\limits_{i=1}^n E\left[u_{i}(x_{i}-\bar{x})\right] \\ &amp;= 2 (\hat{\beta}_{1}-\beta_{1}) \displaystyle\sum\limits_{i=1}^n E\left[u_{i}\right]E\left[(x_{i}-\bar{x})\right] \\ \end{align}  I would have thought that because $E[u_i] = 0$, this term would cancel out, so I'm not sure how to derive $2\sigma^2$ from it.  Can someone give me a hint in the right direction for these steps? I'm firmly trying to work through all of the proofs myself with only a little guidance from the book, but I'm not quite there yet.  I noticed , which links to , so maybe with more reading I'll see where to apply that. (If that isn't it, hints are welcome, but I'll still work through those questions to see if it helps)
98153
Let $X_1,...X_n$ be a random sample with distribution $\text{Normal}(\theta,1)$. Find the UMVUE for $\theta^2$  What I´ve done so far:   I have already shown that $T=\sum_{i=1}^nX_i$ is a complete sufficient statistic for $\theta$. Let $\widehat{\theta^2}=\bar {X}^2-\frac{1}{n}$ be an estimator for $\theta^2$   $$E[\widehat{\theta^2}]=E[\bar {X}^2-\frac{1}{n}]=E[\bar{X}^2]-E^2[\bar X]+E^2[\bar X]+\frac{1}{n}=Var(\bar X)+E^2[\bar X]+\frac{1}{n}$$  I know that $\bar X$ has distribution $\text{Normal}(\theta, \frac{1}{n})$ It follows that :$$E[\widehat{\theta^2}]=\theta^2$$ Hence $\widehat{\theta^2}=\bar {X}^2-\frac{1}{n}$ is unbiased estimator  Know I have to compute $g(T)=E[\widehat{\theta^2}|T]$ and by Lehmann-Scheffé this will be the UMVUE for $\theta^2$ but the problem is how can I compute $$E[\widehat{\theta^2}|T]=E[\bar {X}^2-\frac{1}{n}|\sum_{i=1}^nX_i]$$ I know I need to find the joint density of $\bar {X}^2-\frac{1}{n}$ and $\sum_{i=1}^nX_i$ but is there an easy way to do it?  Or is there another way to find the UMVUE? I would really appreciate if you can help me with this problem
113623
This is a problem from the "7th Kolmogorov Student Olympiad in Probability Theory":     Given one observation $X$ from a $\operatorname{Normal}(\mu,\sigma^2)$ distribution with both parameters unknown, give a confidence interval for $\sigma^2$ with a confidence level of at least 99%.   It seems to me that this should be impossible. I have the solution, but haven't read it yet. Any thoughts?  I'll post the solution in a couple days.  [Follow-up edit: Official solution posted below. Cardinal's solution is longer, but gives a better confidence interval. Thanks also to Max and Glen_b for their input.]
27128
I know that an easy to handle formula for the CDF of a normal distribution is somewhat missing, due to the complicated error function in it.  However, I wonder if there is a a nice formula for $N(c_{-} \leq x &lt; c_{+}| \mu, \sigma^2)$. Or what the "state of the art" approximation for this problem might be.
10764
Is there any intuitive way to interpret the standard deviation regardless of the probability distribution?  So for example, for the normal distribution, I know how to interpret being within 1 standard deviation - there is a a 68% of being there. For other distributions I can calculate this probability as well.  But if I don't know the distribution, how is it a meaningful statistic? Is it only meaningful in relation to other random variables that we know follow the same distribution, but have a different standard deviation?
53767
The question states:      Consider a set of random variables $X_i$, where $i=1,...n$. Each $X_i$ is   normally distributed with mean $0$ and variance $1$, i.e. $X_i$ are $\mathcal N(0,1)$.   What is the mean and the variance of the random variable $Y$, where   $Y=X_1+...+X_n$.   How do I do this?
95393
i. It is a known fact that unbiased estimator of the population variance is given by: $S^{2}=\dfrac{1}{n-1}\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2}$  This has no prior distribution assumption, so it should hold for any distribution of $X$.  ii. In linear regression, unbiased estimator of the error ($\epsilon$) variance is given by: $S^{2}=\dfrac{1}{n-p-1}\sum_{i=1}^{n}\left(y_{i}-\hat{y}_{i}\right)^{2}$  which is the sum of squares of the residuals divided by the sample size minus the number of features.  My question is, why we don't use the formula given by (i) treating the residuals as a sample from the distribution of $\epsilon$? What is the difference between the two?
33133
Consider the general linear regression model: $$y_i = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2} + \cdots + \beta_px_{ip} + \epsilon_i = \mathbf{x}_i^t \beta + \epsilon_i$$  where $\textbf{x}_i = (1,x_{i1},x_{i2},\cdots,x_{ip})^T$, $\beta=(\beta_0,\beta_1,\cdots,\beta_p)^T$ and $\epsilon_i$ are iid N(0,$\sigma^2$).  I would like to see a complete proof of the following identity from first principles: $$\sum_{i=1}^n(y_i - \bar{y})^2 = \sum_{i=1}^n(\hat{y}_i - \bar{y})^2 + \sum_{i=1}^n(y_i - \hat{y}_i)^2$$ where $\hat{y}_i= \mathbf{x}_i^t \hat{\beta} $ ($\hat{\beta}$ is the least square estimator, $\bar{y}$ ia the sample mean of $y_i$).  I know that the two terms on the right can be obtained by subtracting and adding $\hat{y}_i$ on the left side. But this introduces a "cross term": $$\sum_{i=1}^n2(\hat{y}_i - \bar{y})(y_i - \hat{y}_i)$$  Many texts claim that this is zero, but I have not seen a general proof of this statement. How can this be shown?