I have two samples of data each estimates of a position x, y with Gaussian noise. One source has a larger variance than the other. Is this source in any way useful in providing a better estimate of the true position than the source with lower variance or should it just be discounted?
9473
I have a large set of data points of the form (mean, stdev). I wish to reduce this to a single (better) mean, and a (hopefully) smaller standard deviation.  Clearly I could simply compute $\frac{\sum data_{mean}}{N}$, however this does not take in to account the fact that some of the data points are significantly more accurate than others.  To put it simply, I wish to preform a weighted average of these data points, but do not know what the weighting function should be in terms of the standard deviation.
66566
If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to establish an scenario where the choice of a prior is difficult) ... Why would someone choose to use the Bayesian approach with 'noninformative' improper priors instead of the classical approach?
42774
I know that in the case of Gaussian mixture, the "intuition" is that you're drawing from a PDF which itself is just a sum of weighted Gaussian PDFs.  I don't understand the intuition behind how the sum of two iid Gaussian RVs leads to a Gaussian RV. I know the maths behind it, but I'm having a hard time visualizing how the sum of two Gaussian RVs leads to a Gaussian RV (and not a mixture).
84058
If the best linear approximation (using least squares) of my data points is the line $y=mx+b$, how can I calculate the approximation error? If I compute standard deviation of differences between observations and predictions $e_i=real(x_i)-(mx_i+b)$, can I later say that a real (but not observed) value $y_r=real(x_0)$ belongs to the interval $[y_p-\sigma, y_p+\sigma]$ ($y_p=mx_0+b$) with probability ~68%, assuming normal distribution?  To clarify:  I made observations regarding a function $f(x)$ by evaluating it a some points $x_i$. I fit these observations to a line $l(x)=mx+b$. For $x_0$ that I did not observe, I'd like to know how big can $f(x_0)-l(x_0)$  be. Using the method above, is it correct to say that $f(x_0) \in [l(x_0)-\sigma, l(x_0)+\sigma]$ with prob. ~68%?
7430
I recently used bootstrapping to estimate confidence intervals for a project. Someone who doesn't know much about statistics recently asked me to explain why bootstrapping works, i.e., why is it that resampling the same sample over and over gives good results. I realized that although I'd spent a lot of time understanding how to use it, I don't really understand why bootstrapping works.  Specifically: if we are resampling from our sample, how is it that we are learning something about the population rather than only about the sample? There seems to be a leap there which is somewhat counter-intuitive.  I have found a few answers to this question here which I half-understand. Particularly . I am a "consumer" of statistics, not a statistician, and I work with people who know much less about statistics than I do. So, can someone explain, with a minimum of references to theorems, etc., the basic reasoning behind the bootstrap? That is, if you had to explain it to your neighbor, what would you say?
72747
What is it mean by the statement  "X and Y explained a greater proportion of variance in Z than X alone "  More generally, what is it meant when the model "explains more variation" in the dependent variable  -- certainly this is not equivalent to "this explains the variable" more?  Thank you for the insight into this basic question.
99507
Does it make sense to compute confidence intervals and to test hypotheses when data from the whole population is available? In my opinion, the answer is no, since we can accurately compute the true values of the parameters. But then, what is the maximal proportion of data from the original population that allows us to use the aforementioned techniques?
71663
I just read this very insightful post , where the author stated that the variance of $\hat\beta$ is:  $$\text{var}(\hat\beta) = \sigma^2(\textbf{X}^\prime \textbf{X})^{-1}.$$  I couldn't figure out why it is like this. Can anyone elaborate a bit?
40734
Suppose I have a random sample $\lbrace X_i, Y_i\rbrace_{i=1}^n$. Assume this sample is such that the Gauss-Markov assumptions are satisfied such that I can construct an OLS estimator where  $$\hat{\beta}_1^{OLS} = \frac{\text{Cov}(X,Y)}{\text{Var(X)}}$$ $$\hat{\beta}_0^{OLS} = \bar{Y} - \bar{X} \hat{\beta}_1^{OLS}$$  Now suppose I take my data set and double it, meaning there is an exact copy for each of the $n$ $(X_i,Y_i)$ pairs.  My Question  How does this affect my ability to use OLS? Is it still consistent and identified?
20995
I am faced with the following question: given a probability density function $f$ over $\mathbb{R}$ with $\int_{\mathbb{R}}f(x)x^2dx=\sigma^2$ given, I am trying to find the "nearest" Gaussian to $\sqrt{f}$ in $L^2$, i.e. compute $$\inf_{a\in \mathbb{R}, b &gt; 0}\|\sqrt{f}-G_{a,b}\|_2^2$$ where $G_{a,b}(x)=a\exp(-bx^2)$ and the $\|\cdot\|_2$ is the classical $L^2$-norm over $\mathbb{R}.$  If one replaces the $L^2$-distance by the relative entropy $H(f|G_{a,b})$, it is easy to check that the infimum is a minimum given by the centered gaussian with variance $\sigma^2.$ For the $L^2$-distance, it seems more involved......
86149
I have two small data sets (sizes $n_1 = 8$ and $n_2 = 21$) which look like they have significantly different variances. I know very little about the underlying distributions, but it's definitely not safe to assume they're normal or anything nice like that, which rules out the F-test. I'm aware I could use one of the other named tests (Bartlett, Brown–Forsythe, ...) although I'm not currently quite sure what they assume about the population distribution, if anything.  Instead, I've tried my hand at using a permutation test: the null hypothesis is that the two datasets have equal variance, so relabel the data points at random and measure the difference in the two variances of the relabelled sets. Out of 1,000,000 attempts, fewer than 40,000 had a larger difference in variance (&lt;4%).  Is it correct to say that, therefore, the difference in variance of the two data sets is significant at the $p &lt; 0.05$ level? If so, is there a well-recognised name for this kind of test?
80817
The formula for computing variance has $(n-1)$ in the denominator:  $s^2 = \frac{\sum_{i=1}^N (x_i - \bar{x})^2}{n-1}$  I've always wondered why. However, reading and watching a few good videos about "why" it is, it seems, $(n-1)$ is a good unbiased estimator of the population variance. Whereas $n$ underestimates and $(n-2)$ overestimates the population variance.  What I'm curious to know, is that in the era of no computers how exactly was this choice made?  Is there an actual mathematical proof proving this or was this purely empirical and statisticians made A LOT of calculations by hand to come up with the "best explanation" at the time?   Just how did statisticians come up with this formula in the early 19th century with the aid of computers? Manual or there is more to it than meets the eye?
62861
Short version: What's the most computationally efficient method of estimating the mode of a multidimensional data set, sampled from a continuous distribution?  Long version: I've got a data set that I need to estimate the mode of. The mode does not coincide with the mean or median. A sample is shown below, this is a 2D example, but an N-D solution would be better:    Currently, my method is   Calculate kernel density estimate on a grid equal to the desired resolution of the mode  Look for the greatest calculated point   Obviously, this calculates the KDE at a lot of non-plausible points, which is especially bad if there are a lot of data points of high dimensions or I expect good resolution on the mode.  An alternate would be to use a simulated annealing, genetic algorithm, etc to find the global peak in the KDE.  The question is whether there's a smarter method of performing this calculation?
95393
i. It is a known fact that unbiased estimator of the population variance is given by: $S^{2}=\dfrac{1}{n-1}\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2}$  This has no prior distribution assumption, so it should hold for any distribution of $X$.  ii. In linear regression, unbiased estimator of the error ($\epsilon$) variance is given by: $S^{2}=\dfrac{1}{n-p-1}\sum_{i=1}^{n}\left(y_{i}-\hat{y}_{i}\right)^{2}$  which is the sum of squares of the residuals divided by the sample size minus the number of features.  My question is, why we don't use the formula given by (i) treating the residuals as a sample from the distribution of $\epsilon$? What is the difference between the two?
33140
I am looking for an intuitive explanation of the bias-variance tradeoff, both in general and specifically in the context of linear regression.
18665
It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
85242
I have a process which consists of a number of events and what is known is the timings between the events. What I'm trying to determine is a distribution that allows me to determine a likelyhood that a new sample fits the distibution.  The issue is mainly that if you have lots of samples you can approximate the result using a standard gaussian and use the mean and standard deviation. But if you only have a handful of samples, the gaussian does not accurately represent the situation.  From what I've read it is common to model waiting times using the gamma distribution. Looking at how the process evolves it looks like it matches well. The unknown is the scale parameter, since the shape parameter I think should be the number of samples. What I've worked out so far is that given the timings $X_1 ... X_N$ you can say:  $$ \sum_{n=1}^N X_n \sim \Gamma(N,\theta) $$  ($N$ is known and fixed)  However, $\theta$ is unknown, but the maximum likelihood parameter is the average of the $X_i$ (according to wikipedia anyway).   My question is, can I use this to estimate a distribution for $X_i$, that is, since the $X_i$ are independent:  $$ N X_i | \sum_n X_n \sim \Gamma(N, \tfrac{1}{N}\sum_n X_n) $$  Something else I've wondered about. Suppose I do have information about $\theta$, say a distribution. How can I incorporate this into the model?  Edit: Clarified that N is fixed.