I  know those Equations but I cannot show the standard error of fitted value, please help me out, much appreciated. $$ SE(\hat{\beta_1})=\hat{\sigma}\sqrt{\frac{1}{(n-1)s_x^2}}\\ SE(\hat{\beta_0})=\hat\sigma\sqrt{\frac{1}{n}+\frac{\bar{x}^2}{(n-1)s_x^2}}\\ \hat\sigma=\sqrt{\frac{\sum_i(Y_i-\hat{Y})}{n-2}} $$ How do I show the $SE(\hat{\beta_0}+\hat{\beta_1}x_0)=\hat{\sigma}\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{(n-1)s_x^2}}$
84058
If the best linear approximation (using least squares) of my data points is the line $y=mx+b$, how can I calculate the approximation error? If I compute standard deviation of differences between observations and predictions $e_i=real(x_i)-(mx_i+b)$, can I later say that a real (but not observed) value $y_r=real(x_0)$ belongs to the interval $[y_p-\sigma, y_p+\sigma]$ ($y_p=mx_0+b$) with probability ~68%, assuming normal distribution?  To clarify:  I made observations regarding a function $f(x)$ by evaluating it a some points $x_i$. I fit these observations to a line $l(x)=mx+b$. For $x_0$ that I did not observe, I'd like to know how big can $f(x_0)-l(x_0)$  be. Using the method above, is it correct to say that $f(x_0) \in [l(x_0)-\sigma, l(x_0)+\sigma]$ with prob. ~68%?
97543
Suppose I have the following model: $$Y = \mu + \epsilon = X\beta + \epsilon,$$ where $Y$ is $n \times 1$, $X$ is $n \times p$, $\beta$ is $p \times 1$, and $\epsilon$ is $n \times 1$. I assume that $\epsilon$ are independent with mean 0 and variance $\sigma^2I$. In OLS, the fitted values are $\hat{Y} = HY$, where $H = X(X^TX)^{-1}X^T$ is the $N \times N$ hat matrix. I want to find the MSE of $\hat{Y}$. By the bias-variance decomposition, I know that \begin{align*} MSE(\hat{Y}) &amp;= bias^2(\hat{Y}) + var(\hat{Y})\\ &amp;= (E[HY] - \mu)^T(E[HY] - \mu) + var(HY)\\ &amp;= (H\mu - \mu)^T(H\mu - \mu) + \sigma^2H\\ &amp;= 0 + \sigma^2H \end{align*} I'm confused by the dimension in the last step. The $bias^2$ term is a scalar. However, $var(\hat{Y})$ is an $N \times N$ matrix. How can one add a scalar to an $N \times N$ matrix where $N \neq 1$?
13912
I want to calculate standard error of y-intercept or constant term in the multiple regression equation $Y = b_0 + b_1X_1 + b_2X_2$  I found the formula for standard error estimation of co-efficient $b_1$ and $b_2$ as given in the link    But I am not getting any formula for estimating the standard error of b0. Could anyone help me out? Regards Koushik
97029
Ok, this is a quite basic question, but I am a little bit confused. In my thesis I write:  The standard errors can be found by calculating the inverse of the square root of the diagonal elements of the (observed) Fisher Information matrix:  \begin{align*} s_{\hat{\mu},\hat{\sigma}^2}=\frac{1}{\sqrt{\mathbf{I}(\hat{\mu},\hat{\sigma}^2)}} \end{align*} Since the optimization command in R minimizes $-\log\mathcal{L}$ the (observed) Fisher Information matrix can be found by calculating the inverse of the Hessian: \begin{align*} \mathbf{I}(\hat{\mu},\hat{\sigma}^2)=\mathbf{H}^{-1} \end{align*}  My main question: Is this correct what I am saying?  I am a little bit confused, because in this  on page 7 it says:      the Information matrix is the negative of the expected value of the   Hessian matrix   (So no inverse of the Hessian.)  Whereas in this  on page 7 (footnote 5) it says:     The observed Fisher information is equal to $(-H)^{-1}$.   (So here is the inverse.)  I am aware of the minus sign and when to use it and when not, but why is there a difference in taking the inverse or not?
15856
I was just watching a lecture on statistics and someone was calculating something called the residual standard error. It looked a lot like finding the average of the square of the residuals, the residuals being the difference between the prediction of your model and the actual values. So for a linear fit, the prediction is $\hat{y}(x_i)=mx_i+b$ and the actual value is $y_i$. So the residual is $r_i = (y_i - \hat{y}(x_i))$. The residual standard error is $\frac{1}{n-2}\sum_i r_i^2$. I don't understand why dividing by $n-2$ is necessary?   Update: I have a better idea. If there are only two data points, then the residuals would all be zero. So you could not estimate the error with only two points. But this still does not explain why dividing by $n-2$ is a good idea. It only explains why the formula is undefined for $n=2$.
70017
In An Introduction to Statistical Learning (James et al.), in section 3.7 exercise 5, it states that the formula for $\hat{\beta}_1$ assuming linear regression without an intercept is  $$\hat{\beta}_1 = \dfrac{\displaystyle\sum\limits_{i=1}^{n}x_iy_i}{\displaystyle \sum\limits_{i=1}^{n}x_i^2}\text{,}$$ where $\hat{\beta}_0 = \bar{y}-\hat{\beta}_1\bar{x}$ and $\hat{\beta}_1 = \dfrac{\displaystyle S_{xy}}{S_{xx}}$ are the usual estimates under OLS for simple linear regression ($S_{xy} = \displaystyle\sum\limits_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})$).  This is not the actual exercise; I am merely wondering how to derive the equation. Without using matrix algebra, how do I derive it?   My attempt: with $\hat{\beta}_0 = 0$, we have $\hat{\beta}_1 = \dfrac{\bar{y}}{\bar{x}} = \dfrac{S_{xy}}{S_{xx}}$.  After some algebra, it can be shown that $\displaystyle S_{xy} = \sum\limits_{i=1}^{n}x_i^2 - n\bar{x}\bar{y}$ and $\displaystyle S_{xx} = \sum\limits_{i=1}^{n}x_i^2 - n\bar{x}^2$. From here, I'm stuck.
153
In the simple linear regression model, $\hat{\beta}$ is the sum of independent normally distributed random variables.  Is it false because in linear regression there is $\beta$ and not $\hat{\beta}$?
4523
I need to show that F test is equal to T test squared, when the T test is for 2 independent groups and assuming variances are equal.  I know that $F=\frac{MSB}{MSW}=\frac{SSB/k-1}{SSW/N-K}$ and I know that $T=\frac{X-Y}{S_p \sqrt{\frac{1}{n}+\frac{1}{m}}}$,   so $T^2=\frac{(X-Y)^2}{S_p^2 ({\frac{1}{n}+\frac{1}{m}})}$  I've seen this proof in Regression but here we're not using MSE and MSR, so i'm not sure how to connect between the two.
104855
My question is essentially simple: suppose we have a sequence of heads and tails with an apparent chance of getting heads $\hat{p} = 0.5$.      h h t h t t t h t t h ...   How do we tell if this sample was drawn from a binomial distribution with this $\hat{p}$ or if it comes from a combination of different distributions which produces the same average? (e.g. $p = 0.25$ or $p = 0.75$)  Here's what I was thinking of doing:   We chop the sequence into subsections of length $n$, for example $n=5$, and calculate $\hat{p}$ in every such window: {h h t h t}, {t t h t t}, {h ...}0.6, 0.2, ... We then calculate the standard deviation in these $\hat{p}$ values: $\sigma_{ex}=\sqrt{\frac{\sum_{i=1}^M{(\hat{p}_i - \hat{\mu})^2}}{M}}$, where $\hat{\mu}$ is the average $\hat{p}$ and $M$ is the total number of windows of size $n$ we can make.  We calculate the standard deviation in $\hat{p}$ based on a binomial distribution, $\sigma_{th}=\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$.   Now comes the problem: how to determine if these two values are the same? After a lot of research, I decided to try using an F-ratio test and plugging the result into an f-distribution.  F-ratio: $F = \frac{\sigma_{ex}^2}{\sigma_{th}^2}$ (only valid for normal-distributions :S)  And then an f-distribution to get to a p-value we can check against. However, for the f-distribution I need to know the degrees of freedom in the nominator and denominator. And therein lies my problem: what are the degrees of freedom for the variance calculated from theory? (In the experimental case I think it is M)  If you have this answer or any suggestions on how to solve this problem differently, I would love to hear about it!
97436
If I have the expression below, how can I use y~N($\mu$, $\Sigma$) to show it is a $\chi^2$ distribution with ($_n-1$) degrees of freedom?  ($n$-1)($s^2$)/$\sigma^2$ how is this ~$\chi^2$ $_n$-$_1$?     I recognize this contains SSE. I know that I should show the square root produces a distribution of $N$($\mu$, $I$) . I am lost at this point. I am thinking perhaps to substitute quadratic form into the original expression, then see what cancels? Or is this totally wrong?
21116
Let ${Y_1,...,Y_n}$ be independent random variables and $Y_i$~$N(\beta x_i, 1)$ where $x_1,...,x_n$ are fixed known constants, and $\beta$ is an unknown parameter.  I'm looking to find the p-value or rejection region for  the test  $$H_0: \beta=0 \quad \text{vs} \quad H_1:\beta\ne0$$  The Likelihood Ratio Test statistic $\Lambda$ is  $$e^{-1/2\left(\frac{\sum_{i=1}^n y_i x_i}{\sum_{i=1}^n x_i^2}\right)^2\sum_{i=1}^n x_i^2}$$  I have already asked a similar question with $H_1: \beta&gt;0$ here: .   After setting $\Lambda &lt; k$ I'm left with $\sum x_iy_i$ after removing constants to the right, which is the same as my previous question. How will finding the p-value or rejection region for this test differ?
79868
Suppose, we have $n$ samples $x_i$ of a random variable: $$x \sim \mathcal N(\mu,\sigma^2) $$ Based on the samples, we want to estimate the probability that $x$ is negative: $$P(x\le0)$$ Intuitively, I would first estimate: $$\hat \mu=\frac 1 n \sum x_i$$ $$\hat \sigma^2={\frac 1 {n-1} \sum (x_i-\hat \mu)^2}$$ and then calculate: $$P(x\le0)=\frac1{\sqrt{2\pi\hat\sigma^2}} \int_{-\infty}^0 e^\frac{x-\hat\mu}{\hat \sigma} dx$$ However $\hat \mu$ and $\hat \sigma$ have variance! If I use this method, I suspect I am ignoring that variance and making an incorrect estimation. Is this reasoning right? If so, how can I estimate $P(x\le0)$ more correctly, taking $\text {VAR}[\hat \mu]$ and $\text {VAR}[\hat \sigma]$ into account?
116793
I'm a mathematician self-studying statistics and struggling especially with the language.   In the book I'm using, there is the following problem:  A random variable $X$ is given as $\text{Pareto}(\alpha,60)$-distributed with $\alpha&gt;0$. (Of course, you could take any distribution depending on one parameter for the sake of this question.) Then a sample of five values $14$, $21$, $6$, $32$, $2$ is given.   First part: "Using the method of maximum likelihood, find an estimate $\hat{\alpha}$ of $\alpha$ based on [the sample]." This was no problem. The answer is $\hat{\alpha}\approx 4.6931$.  But then: "Give an estimate for the standard error of $\hat{\alpha}$."  What is meant by this? Since $\hat{\alpha}$ is just a fixed real number, I don't see in what way it could have a standard error. Am I to determine the standard deviation of $\text{Pareto}(\hat{\alpha},60)$?  If you think the question is not clear, this information would help me as well.
226
On page 19 of the textbook  (by James, Witten, Hastie and Tibshirani--it is freely downloadable on the web, and very good), the following is stated:  Consider a given estimate $$\hat{Y} = \hat{f}(x)$$ Assume for a moment that both $$\hat{f}, X$$ are fixed. Then, it is easy to show that: $$\mathrm{E}(Y - \hat{Y})^2 = \mathrm{E}[f(X) + \epsilon - \hat{f}(X)]^2$$ $$ = [f(X) - \hat{f}(X)]^2 + \mathrm{Var}(\epsilon)$$  It is further explained that the first term represents the reducible error, and the second term represents the irreducible error. I am not fully understanding how the authors arrive at this answer. I worked through the calculations as follows: $$\mathrm{E}(Y - \hat{Y})^2 = \mathrm{E}[f(X) + \epsilon - \hat{f}(X)]^2$$ This simplifies to $[f(X) - \hat{f}(X) + \mathrm{E}[\epsilon]]^2 = [f(X) - \hat{f}(X)]^2$ assuming that $\mathrm{E}[\epsilon] = 0$. Where is the $\mathrm{Var}(x)$ indicated in the text coming from? Any suggestions would be greatly appreciated.
64162
My stats professor assigned this problem: "Show that the expected prediction error (EPE) for a squared error loss when $Y=f(X)+\varepsilon$ with estimator $\hat{f}(x)$ assuming $X=x$ is fixed and $\varepsilon~(0,\sigma^2)$ can be written as a combination of the bias and variance. In other words, show that   $$EPE(x) = E[(Y-\hat{f}(x))^2] = \sigma^2 + \text{Bias}^2 + \text{Var}(\hat{f}(x)).$$  I've come up with 3 ways to derive this relationship, but all of them depend on the assumption that $Y=f(X)+\varepsilon$ and $\hat{f}(X)$ are independent, or at least that their covariance is zero. For example, this allows me to use $\text{Var}[Y-\hat{f}(X)] = \text{Var}(Y)+\text{Var}(\hat{f}(X))$. This assumption makes intuitive sense to me because there is no necessary connection between $Y$ and the estimated value $\hat{f}(X)$ (the estimator could be a random number generator, after all). But I'm struggling to justify the independence assumption in a rigorous way. Can somebody nudge me toward understanding?
33750
Source |       SS       df       MS              Number of obs =     300 -------------+------------------------------           F(  5,   294) =16580.50        Model |  1.1499e+10     5  2.2998e+09           Prob &gt; F      =  0.0000     Residual |  40778600.2   294  138702.722           R-squared     =  0.9965 -------------+------------------------------           Adj R-squared =  0.9964        Total |  1.1540e+10   299  38593905.2           Root MSE      =  372.43   Should I be worried about this- does your F statistic size matter?  How do i find out what the p value of my model is - is it the p value attached to _cons in the bottom of my coefficients?  what do i test this F stat against and what warrants significance?
4868
I mostly see this formula when searching for a formula for the estimate of the standard error in difference between two means, and it is also used in . $$\Delta=\sqrt{s_1^2/N_1+s_2^2/N_2}$$ But I've also seen this one (and this is the one my book uses): $$\Delta'=\sqrt{\dfrac{\left(N_1-1\right)s_1^2+\left(N_2-1\right)s_2^2}{N_1+N_2-2}\left(\dfrac{1}{N_1}+\dfrac{1}{N_2}\right)}$$ As these are two very different formulas, how come they are used seemingly interchangeably?