The sample standard deviation and standard error of the mean are biased estimators for their corresponding population population parameters. As explained  for the sample standard deviation this is due to the fact that the expectation of a square root does not equal the square root of an expectation, as can be shown by Jensens equality.   Nevertheless, we are inclined to give an interpretation to standard error estimates, in the sence that they give the mean distance of all sample estimators (means) to the population parameter (mean). However, this seems only to be legitimate if the bias is commonly small or vanishes in large samples under quite general conditions (e.g., distributions). In the proof references above, this seems to be the case for normal data. Is this generally so?  Furthermore: the sample standard error is commonly used as estimator of population standard error in a t-statistic. $$t=\frac{\hat{\theta}-\theta}{s.e.({\hat{\theta}})}$$. Is it true that we would actually need an unbiased estimator of the standard error in this formula or is the use of a biased estimator allowed. What is the role of the sample size again (for normal / non-normal data)?
100987
I was reading on the computation of the unbiased estimation of standard deviation and the source I read stated      (...) except in some important situations, the task has little relevance to   applications of statistics since its need is avoided by standard   procedures, such as the use of significance tests and confidence   intervals, or by using Bayesian analysis.   I was wondering if anyone could elucidate the reasoning behind this statement, for example doesn't the confidence interval use the standard deviation as part of the calculation? Therefore, wouldn't the confidence intervals be affected by a biased standard deviation?  EDIT:  Thanks for the answers so far, but I'm not quite sure I follow some of the reasoning for them so I'll add a very simple example. The point is that if the source is correct, then then something is wrong from my conclusion to the example and I would like someone to point how how the p-value doesn't depend on the standard deviation.  Suppose a researcher wished to test whether the mean score of fifth graders on a test in his or her city differed from the national mean of 76 with a significance level of 0.05. The researcher randomly sampled the scores of 20 students. The sample mean was 80.85 with a sample standard deviation of 8.87. This means: t = (80.85-76)/(8.87/sqrt(20)) = 2.44. A t-table is then used to calculate that the two-tailed probability value of a t of 2.44 with 19 df is 0.025. This is below our significance level of 0.05 so we reject the null hypothesis.  So in this example, wouldn't the p-value (and maybe your conclusion) change depending on how you estimated your sample standard deviation?
80564
I am unable to find the bias of the sample variance estimator in this problem. Unfortunately I keep coming up with   Question: Suppose that the true linear model is   $y = X_1\beta_1 + X_2\beta_2 + e$,   where $e \sim MVN(0,\sigma^2 I)$, $X_1: N \times p_1, \beta_1: p_1 \times 1, X_2: N \times p_2, \beta_2: p_2 \times 1$. But we wrongly fit the model  $y = X_1\beta_1+e$ and calculate the estimators of $\beta_1$ and $\sigma^2$ as:  $\hat{\beta}_1 = (X_1'X_1)^{-1}X_1'y$,  $s^2 = \large \frac{y'[I-X_1(X_1'X_1)^{-1}X_1']y}{N-p_1}$.  (a) Show that, in general, $\hat{\beta}_1$ is a biased estimator with  Bias$(\hat{\beta}_1) = (X_1'X_1)^{-1}X_1'X_2\hat{\beta}_2$  Under what condition on $X_1$ and $X_2$ is $\hat{\beta}_1$ unbiased?  (b) Show that, in general, $s^2$ is a biased estimator with   Bias$(s^2) = \frac{1}{N-p_1}\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_2\beta_2$  Is the bias positive or negative? Under what condition on $\beta_2$ is $s^2$ unbiased?  Attempt:  For part (a):  Bias$(\hat{\beta}_1) = E((X_1'X_1)^{-1}X_1'y - \beta_1)$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'E(y) - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'(X_1\beta_1 + X_2\beta_2) - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'X_1\beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2) - \beta_1$  $\hspace{15mm}= I\beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2 - \beta_1$  $\hspace{15mm}= \beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2 - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'X_2\beta_2$  If $X_1$ and $X_2$ are orthogonal, then $\hat{\beta}_1$ is unbiased.  For part (b), I am unable to figure out how to derive:  Bias$(s^2) = \frac{1}{N-p_1}\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_2\beta_2$  I can get   $\frac{1}{N-p_1}\beta_1'X_1'+\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_1\beta_1+X_2\beta_2$  after taking the expectation of y on both sides.   Any help would be much appreciated.
16109
In classical statistics, there is a definition that a statistic $T$ of a set of data $y_1, \ldots, y_n$ is defined to be complete for a parameter $\theta$ it is impossible to form an unbiased estimator of $0$ from it nontrivially. That is, the only way to have $E h(T (y )) = 0$ for all $\theta$ is to have $h$ be $0$ almost surely.   Is there a intuition behind this? Is seems like a rather mechanical way of defining this, I am aware this has been asked before, but was wondering if there was a very easy to understand intuition that would make introductory students have an easier time digesting the material.
93236
Consider $N$ independent samples $S$ obtained from a random variable $X$ that is assumed to follow a truncated distribution (e.g. a ) of known (finite) minimum and maximum values $a$ and $b$ but of unknown parameters $\mu$ and $\sigma^2$. If $X$ followed a non-truncated distribution, the maximum likelihood estimators $\widehat\mu$ and $\widehat\sigma^2$ for $\mu$ and $\sigma^2$ from $S$ would be the sample mean $\widehat\mu = \frac{1}{N} \sum_i S_i$ and the sample variance $\widehat\sigma^2 = \frac{1}{N} \sum_i (S_i - \widehat\mu)^2$. However, for a truncated distribution, the sample variance defined in this way is bounded by $(b-a)^2$ so it is not always a consistent estimator: for $\sigma^2 &gt; (b-a)^2$, it cannot converge in probability to $\sigma^2$ as $N$ goes to infinity. So it seems that $\widehat\mu$ and $\widehat\sigma^2$ are not the maximum-likelihood estimators of $\mu$ and $\sigma^2$ for a truncated distribution. Of course, this is to be expected since the $\mu$ and $\sigma^2$ parameters of a truncated normal distribution aren't its mean and variance.  So, what are the maximum likelihood estimators of the $\mu$ and $\sigma$ parameters of a truncated distribution of known minimum and maximum values?
88446
Is unbiasedness a necessary condition for an estimator to be efficient?  For example, if $\hat {\theta}= \frac{\sum_i^n X_i}{3}$, I assume $\hat {\theta}$ can't be efficient in a Cramer-Rao lower bound context because $E[\hat {\theta}]= \frac {\theta}{3}$.
52385
How do I show that 0.5Y in part (a) down below is an unbiased estimator of alpha? If n, the number of observations, is very large then Y would equal to alpha+alpha=2*alpha an then 0.5Y=alpha so E(0.5Y)=E(alpha)=alpha but the size of n is not stated so how is this question supposed to be solved?
108775
Following my question , I am wondering if there are strong views for or against the use of standard deviation to detect outliers (e.g. any datapoint that is more than 2 standard deviation is an outlier).  I know this is dependent on the context of the study, for instance a data point, 48kg, will certainly be an outlier in a study of babies' weight but not in a study of adults' weight.  Outliers are the result of a number of factors such as data entry mistakes. In my case, these processes are robust.   I guess the question I am asking is: Is using standard deviation a sound method for detecting outliers?
48305
Suppose we have a sample of size n= 9, where the sum equals 27, and the sum of the squaredvalues equals 113. Sample mean is 27/9=3 and mean of squared values is 113/9 = 12.56 Can we then say that the sample variance $S^2 = E(\overline X^2) - E(\overline X)^2 = 12.55 - 9 = 3.56$ ?
64382
An addition to the common confusion about normality testing in inferential statistics for general linear models:  I understand the assumption of normality refers to the residuals in ANOVA and regression models, but what about t-tests?  Calculation of the t-statistic uses standard deviation as a measure of dispersion, rather than sums of squares for the F-statistic in ANOVA and regression. If I understand correctly, it is because of this that there is an assumption that the samples are normally distributed (i.e. the sample standard deviation provides an accurate estimate of the population standard deviation). Is this right?  What do you do if the statistical population of the dependent variable is not distributed normally (thus samples are unlikely to be normally distributed)? Is this where transformations fit in?
17882
I did not get the why there are N and N-1 while calculating population variance. When we use N and when we use N-1?      It says that when population is very big there is no difference between N and N-1 but it does not tell why is there N-1 at the beginning.  Edit: Please don't confuse with n and n-1 which are used in estimating.  Edit2: I'm not talking about population estimation.
58425
I'm having trouble making sense out of the formula $$(\hat{p}_1-\hat{p}_2) \pm 1.96 \sqrt{\frac{\hat{p}_1 (1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2 (1-\hat{p}_2)}{n_2}}$$  Is the idea that the sampling distribution of the difference between proportions of group $1$ and proportions of group $2$ of sample size $n$ is normally distributed regardless of population distribution shape, the same way it is true of the sampling distribution of the sample means, for example?  And in this case, what does that mean exactly? Say we have a sample size of $10$; are we talking about every possible combination of differences between every possible combination of samples of size $10$ for both groups and saying that that "function" is approximately normally distributed? What does it mean to subtract one sampling distribution from another?   And then, where does the square root come from? Is it just the SE of the distribution resulting from subtracting the sampling distribution of sample proportions of group 2 from the sampling distribution of group 1?
37734
The formula for a one sample t-test, as given by  is  $t = \dfrac{\bar{x} - \mu_0}{s\big/\sqrt{n}}$  However, in this equation, if the sample standard deviation, $s$, is zero, $t$ will be undefined. Is there any reasonable way to get a $t$ value in this case?
65990
I am wondering about the different ways that Bayesian and Frequentist statistic connect with each other. I recalled that the Maximum Likelihood estimate of a parameter $\theta$ is  of that parameter. That made me wonder: Is the Mean Posterior estimate of $\theta$ an unbiased estimator? That is,  Does $\phi(x)=E(\theta\mid x)$, imply $E(\phi(x)\mid\theta)=\theta$?  Note that this is indeed a meaningful question, since $\phi(x)$, while it is a Bayesian estimator, is simply a function from the data to the real line and so can also be seen as a classical frequentist estimator. If this question cannot be answered in general, please assume the prior is uniform. If not, is there some other Bayesian estimator (i.e. a function from the posterior to $\mathbb R$) that is always an unbiased estimator in the frequentist sense?
87051
I can follow the proofs in which the asymptotic normal-distribution of a maximum likelihood estimator $\tilde{\theta}_n$ is derived.   however, does this already imply that the maximum likelihood estimates are asymptotically unbiased, i.e. do we have $$E(\tilde{\theta}_n) \to \theta \text{ as } n \to \infty?$$  Since I am aware that in general it is not true that convergence in distribution implies convergence in moments, an explanation would be nice.  The result is often stated e.g.  as "...it means that the bias of the maximum likelihood estimator is equal to zero up to the order $n^{-1/2}$")  1.) Are  some kind of regularity conditions from the mle theory used to establish this result?  2.) Or is a $\sqrt{n}$-convergence of an estimator (to a normal distribution) in general already enough to establish convergence of its moments?  Note: the Wikipedia article mentions Cox, David R.; Snell, E. Joyce (1968).  , as a source where the order of the bias is derived (formula (12) or (20)).   However in this paper I can't follow the arguments 100%, since their Taylor approximation of $L'(\widehat{\beta})$ is lacking the remainder term. What is the argument used here to discard it completely?
98069
Suppose $Y_1, \ldots, Y_n$ is a random sample from a normal distribution with mean $\mu$ and variance $\sigma^2$. Let $S^2$ be the sample variance, which is unbiased for $\sigma^2$.  GOAL: Show that $S = \sqrt{S^2}$ is a biased estimator of $\sigma$.  I realize that $(n-1)S^2/ \sigma ^2$ is chi-square with $(n-1)$ degrees of freedom.  I am given that $$E(S) = E\left[\frac{\sigma}{\sqrt{n-1}} \left[\frac{(n-1)S^2}{\sigma^2}\right]^{1/2}\right] = \frac{\sigma}{\sqrt{n-1}} \int_0^\infty v^{1/2} \frac{1}{\Gamma[(n-1)/2] 2^{(n-1)/2}} v^{(n-1)/2} e^{-v/2} \, dv$$  Can anyone help me understand why they have $v^{1/2}$ and $v^{(n-1)/2}$ (combining them $v^{n/2}$ as I think it should be $v^{1/2}$ and $v^{\frac{(n-1)}{2}-1}$ (combining both terms give $v^{(n-2)/2}$ since the pdf for chi-square distribution is $$ f(v) = \frac{v^{(x/2)-1} e^{-v/2}}{2^{x/2}\Gamma(x/2)},$$ where $x$ is degree of freedom?  Could anyone please tell me which one is the correct answer with explanation?  Thank you
83437
Background: I'm giving a presentation to colleagues at work on hypothesis testing, and understand most of it fine but there's one aspect that I'm tying myself up in knots trying to understand as well as explain it to others.  This is what I think I know (please correct if wrong!)   Statistics that would be normal if variance was known, follow a $t$-distribution if the variance is unknown CLT (Central Limit Theorem): The sampling distribution of the sample mean is approximately normal for sufficiently large $n$ (could be $30$, could be up to $300$ for highly skewed distributions) The $t$-distribution can be considered Normal for degrees of freedom $&gt; 30$   You use the $z$-test if:   Population normal and variance known (for any sample size) Population normal, variance unknown and $n&gt;30$ (due to CLT) Population binomial, $np&gt;10$, $nq&gt;10$   You use the $t$-test if:   Population normal, variance unknown and $n&lt;30$ No knowledge about population or variance and $n&lt;30$, but sample data looks normal / passes tests etc so population can be assumed normal   So I'm left with:    For samples $&gt;30$ and $&lt;\approx 300$(?), no knowledge about population and variance known / unknown.   So my questions are:   At what sample size can you assume (where no knowledge about population distribution or variance) that the sampling distribution of the mean is normal (i.e. CLT has kicked in) when the sampling distribution looks non-normal? I know that some distributions need $n&gt;300$, but some resources seem to say use the $z$-test whenever $n&gt;30$... For the cases I'm unsure about, I presume I look at the data for normality. Now, if the sample data does looks normal do I use the $z$-test (since assume population normal, and since $n&gt;30$)? What about where the sample data for cases I'm uncertain about don't look normal? Are there any circumstances where you'd still use a $t$-test or $z$-test or do you always look to transform / use non-parametric tests? I know that, due to CLT, at some value of $n$ the sampling distribution of the mean will approximate to normal but the sample data won't tell me what that value of $n$ is; the sample data could be non-normal whilst the sample mean follows a normal / $t$.  Are there cases where you'd be transforming / using a non-parametric test when in fact the sampling distribution of the mean was normal / $t$ but you couldn't tell?
33992
I have an HP 50g graphing calculator and I am using it to calculate the standard deviation of some data. In the statistics calculation there is a type which can have two values:  Sample Population  I didn't change it, but I kept getting the wrong results for the standard deviation. When I changed it to "Population" type, I started getting correct results!  Why is that? As far as I know, there is only one type of standard deviation which is to calculate the root-mean-square of the values!  Did I miss something?