Exercise :     Prove that the variance of $\hat{y}_{x_0} = \hat{b_0} + \hat{b_1}x_0$ is :   $$\text{Var}(\hat{y}_{x_0}) = \frac{\sigma^2\sum x_i^2}{n\sum(x_i-\bar{x})^2}+\frac{\sigma^2x_0^2}{\sum(x_i-\bar{x})^2}-\frac{2x_0\sigma^2\bar{x}}{\sum (x_i-\bar{x})^2}=\sigma^2\bigg[\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\bigg]$$   Question : How would one go on to proving all that stuff ? It seems too much to handle at first sight and I am a total beginner at this course.
121063
I have asked in another question how $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$.  Note that $\hat{y}_h$ = $b_0 + b_1X_h$ which is a regression line estimate at some given $X_h$.  This question concerns why the term $Cov(b_0,b_1)$ alone yields the RHS.  Substituting $b_0 = Y - b_1X$ we get that $Cov(Y,b_1) - XCov(b_1,b_1)$ = $Cov(\frac{\sum{Y_i}}{n},\sum k_iY_i) - XVar{(b_1)}$. Here X and Y without subscript are arithmetic means.  We can then rearrange to obtain $\sum \frac{k_i Var(Y_i)}{n} - \frac{X\sigma^2}{S_{xx}}$ which quickly yields the desired result.  My question is, why does this work?  This single term does not seem like it should alone yield the RHS.  Have I made an error in algebra?
59284
The question is-     Let $X_1,X_2,..,X_n$ be iid random variables from a continuous distribution whose density is symmetric about $0$. Suppose $\mathbb{E}(|X_1|)=2$ and define $Y=\sum_{i=1}^{n}X_i$ and $Z=\sum_{i=1}^{n}I(X_i&gt;0)$. Then calculate covariance between $Y$ and $Z$.   My attempt:  $E(X_i)=0$ for all $i=1(1)n$ because $X$ is symmetric about $0$ and $E(|X|) $ exists.   Now,  $Cov (Y,Z)=E(YZ)-E(Y)E(Z)$ $=E(YZ)-0$ $=E[(\sum_{i=1}^{n}X_i)(\sum_{i=1}^{n}I(X_i&gt;0)]$  $=(\sum_{i=1}^{n}E[(X_i.I(X_i&gt;0))]$ $+\sum\sum_{i \neq j}E(X_i)E(I(X_j&gt;0)$ as $X_i,X_j$ are independent.   $=\sum_{i=1}^{n}E[(X_i.I(X_i&gt;0)] +0 $ as $E(X_i)=0$  $ =\sum_{i=1}^{n}\{E[X_i.I(X_i&gt;0)|I(X_i&gt;0)=1]×1/2] + E[X_i.I(X_i&gt;0)|I(X_i&gt;0)=0]×1/2]\}$  $=\sum_{i=1}^{n}E[X_i.I(X_i&gt;0)|I(X_i&gt;0)=1]×1/2] +0$  $=\sum_{i=1}^{n}E[X_i|X_i&gt;0]×1/2]$  $=2n×(1/2)$ $=n$  Is my reasoning correct ? Thanks in advance!
34219
In the vein of , I'm now at a roadblock on question 3 of this sheet:    (note: it's not my intention to ask every question I get stuck on here, merely the ones which have interesting general results; it just so happens that the two I've struggled with so far fit this criteria)  It goes as follows:  Let $N$ be a non-negative integer-valued random variable with mean $\mu_1$ and variance $\sigma_1^2$, and let $X_1, X_2, ...$ be random variables, each with mean $\mu_2$ and variance $\sigma_2^2$; furthermore, assume that $N, X_1, X_2,  . . .$ are independent. Without using generating functions, calculate the mean and variance of the random variable $S_N = X_1 + ... + X_N$ (when $N=0$ interpret $S_N$ as $0$).  I have the answer to calculating the mean, which I'll write up as an answer for future reference. Here's where I've got to with the variance bit:  $\mathbb{E}(S_N^2) = 0.P(N=0) + \sum_{r=1}^\infty\mathbb{E}((\sum_{i=1}^rX_i)^2).P(N=r)$  For $r=1$, we have the inner expectation as: $\mathbb{E}(X_1^2)$ which is equal to $\sigma_2^2+\mu_2^2$.  For $r &gt; 1$, we have the inner expectation equal to: $r(\sigma_2^2 + 2\mu_2^2)$  So $\mathbb{E}(S_N^2) = (\sigma_2^2+\mu_2^2)P(N=1) + (\sigma_2^2 + 2\mu_2^2)\sum_{r=2}^\infty rP(N=r)$  which is almost in the form where I can use the same trick as in calculating the mean, but not quite.  Any help is much appreciated.
1348
I am trying to prove the statement:  If $X\sim\mathcal{N}(0,\sigma_1^2)$ and $Y\sim\mathcal{N}(0,\sigma_2^2)$ are independent random variables, then $\frac{XY}{\sqrt{X^2+Y^2}}$ is also a Normal random variable.  For the special case $\sigma_1=\sigma_2=\sigma$ (say), we have the well-known result that $\frac{XY}{\sqrt{X^2+Y^2}}\sim\mathcal{N}\left(0,\frac{\sigma^2}{4}\right)$ whenever $X$ and $Y$ are independent $\mathcal{N}(0,\sigma^2)$ variables. In fact, it is more generally known that $\frac{XY}{\sqrt{X^2+Y^2}},\frac{X^2-Y^2}{2\sqrt{X^2+Y^2}}$ are independent $\mathcal{N}\left(0,\frac{\sigma^2}{4}\right)$ variables. A proof of the last result follows by using the transformation $(X,Y)\to(R,\Theta)\to(U,V)$ where $x=r\cos\theta,y=r\sin\theta$ and $u=\frac{r}{2}\sin(2\theta),v=\frac{r}{2}\cos(2\theta)$. Indeed, here $U=\frac{XY}{\sqrt{X^2+Y^2}}$ and $V=\frac{X^2-Y^2}{2\sqrt{X^2+Y^2}}$. I tried to imitate this proof for the problem at hand but it appears to get messy. If I haven't made any error, then for $(u,v)\in\mathbb{R}^2$ I end up with the joint density of $(U,V)$ as $$f_{U,V}(u,v)=\frac{2}{\sigma_1\sigma_2\pi}\exp\left[-\sqrt{u^2+v^2}\left(\frac{\sqrt{u^2+v^2}+v}{\sigma_1^2}+\frac{\sqrt{u^2+v^2}-v}{\sigma_2^2}\right)\right]$$ I have the multiplier $2$ above as the transformation is not one-to-one. So density of $U$ would be given by $\displaystyle \int_{\mathbb{R}}f_{U,V}(u,v)\,\mathrm{d}v$, which isn't readily evaluated. Now I am interested to know if there is a proof where I can only work with $U$ and don't have to consider some $V$ to show that $U$ is Normal. Finding the CDF of $U$ doesn't look so promising to me at the moment. I would also like to do the same for the case $\sigma_1=\sigma_2=\sigma$. That is, if $X$ and $Y$ are independent $\mathcal{N}(0,\sigma^2)$ variables then I wish to show that $Z=\frac{2XY}{\sqrt{X^2+Y^2}}\sim\mathcal{N}(0,\sigma^2)$ without using a change of variables. If somehow I can argue that $Z\stackrel{d}{=}X$, then I'm done. So two questions here, the general case and then the particular case. Related posts on Math.S.E: . . Edit. This problem is in fact due to L. Shepp as I found out in the exercises of An Introduction to Probability Theory and Its Applications (Vol. II) by Feller, alongwith a possible hint:  Surely, $U=\frac{XY}{\sqrt{X^2+Y^2}}=\frac{1}{\sqrt{\frac{1}{X^2}+\frac{1}{Y^2}}}$ and I have the density of $\frac{1}{X^2}$ at hand. Let's see what I could do now. Apart from this, a little help with the integral above is also welcome.
41923
I am a probability theory beginner. The expression for the variance of a random variable $x$ (of a random process is   $$\sigma^2 = E(x^2) - (\mu_{x})^2$$  If $E(x^2) = (\mu_{x})^2$, then $\sigma^2 = 0$. Can this happen ? Can a random variable have a density function whose variance (the second central moment alone) is $0$ (other than the dirac delta function).
56322
Given a random variable $X\sim\mathcal N(0,\sigma^2)$, how can we prove that $E[X^4]=3\sigma^4$? I am having trouble even starting with the proof.
88319
I stumbled over a rather simple result of OLS regression which is  $$ Var[\hat\beta] = \sigma^2(X^TX)^{-1} $$ where $\sigma$ is the variance of the error term $u$ and $X$ is the regressor matrix.  I first just accepted the proof in my textbook but now I am thinking that it either uses sloppy notation or I am missing something. $\hat\beta$ is the estimated and $\beta$ is the true parameter (assuming unbiasedness).  It states that   \begin{align} Var[\hat\beta] &amp;= E[(\hat\beta - \beta)(\hat\beta-\beta)^T] \\                &amp;= E[(X^TX)^{-1}X^Tuu^TX(X^TX)^{-1}]         \\                &amp;=  (X^TX)^{-1}X^T E[uu^T] X(X^TX)^{-1} \end{align}  but $X$ was only assumed to be exogenous and not non-stochastic. Under this assumption I think $X$ cannot be dragged outside the expectation operator.   Momentarily, I think that it should be $Var[\hat\beta|X]$ to make sense. Is that the case? My web research couldn't clarify this. I only found similar derivations to the above without further explainations.
113606
I am not a statistician, but am working a proof for the upper bound of an expression which contains the variance of a variable which obtains its values from a closed interval, [0,1]. I have seen in several places (mostly forums and web pages) the result that $\text{Var}(X)\le \frac{(b-a)^2}{4}$ for $a\le X\le b$. I would like to use, and cite, this result but am having an unusually difficult time finding a solid reference. Could someone here recommend one for me?
22712
... and why ?  Assuming $X_1$,$X_2$ are independent random-variables with mean $\mu_1,\mu_2$ and variance $\sigma^2_1,\sigma^2_2$ respectively. My basic statistics book tells me that the distribution of the $X_1-X_2$ has the following properties:    $E(X_1-X_2)=\mu_1-\mu_2$ $Var(X_1-X_2)=\sigma^2_1 +\sigma^2_2$   Now let's say $X_1$, $X_2$  are t-distributions with $n_1-1$, $n_2-2$ degrees of freedom. What is the distribution of $X_1-X_2$ ?   This question has been edited: The original question was "What are the degrees of freedom of the difference of two t-distributions ?". mpiktas has already pointed out that this makes no sense since $X_1-X_2$ is not t-distributed, no matter how approximately normal $X_1,X_2$ (i.e. high df) may be.
41027
I'm new to statistics and reading stats books I found different definitions for Variance.  Definition1: $ s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2 $  Definition2: $ s^2 = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 $  I would like to know why is used different definition for same thing?
73824
I need to prove that $$\rho(X,Y) = \pm 1 \implies Y = aX+b,$$ for some constants $a, b$.  With the help of  and  here's what I've got so far:  Let $X, Y$ be random variables and $a$ some constant. $aX + Y$ is also a random variable whose variance by definition is non-negative $$V(aX+Y) \ge 0.$$ From variance properties: $$a^2 V(X)+2a\text{Cov}{X,Y}+V(Y)\ge 0$$ This is a quadratic as a function of $a$, which has a maximum of 1 root. It has a root only for $a$ that satisfies $$a^2 V(X)+2a\text{Cov}{X,Y}+V(Y)=0$$ let it be $a_0$. That means that $$V(a_0X+Y)=0$$  How can I continue from here?  Thank you.
88460
How do I prove the following? I don't know where to start.     If $X$ is a random variable with $E^{\mathbb P}[X] = \mathbb P(X&gt;0)=1$   and $ \mathbb Q$ is the  probability measure defined by $ \mathbb Q(A)=E^{\mathbb Q}[X1_A] $ then $E^{\mathbb Q}[Y]=E^{\mathbb P}[XY]$
63367
When reading a proof I came across the following step:  $$\operatorname{Var}(x^Ty) = x^T\operatorname{Var}(y)x$$ $x$ and $y$ are column vectors. How can you derive this?
80741
I'm trying to understand my teacher's solution to this problem:     Let $X_1,X_2,...$ sequence of iid random variables with expected value   $\mu$. Define $Y_n = \frac{2}{n(n+1)} \sum_{j=1}^{n} jX_j$. Prove that   $Y_n \to_\mathbb{P} \mu$.   His solution:  Let $\sigma^2 = \rm{Var}(X_i)$.  $E(Y_n) = \frac{2}{n(n+1)} \sum_{j=1}^{n} jE(X_n) = \mu$.  $\displaystyle\rm{Var}(Y_n) = \frac{4}{n^2(n+1)^2} \sum_{j=1}^{n} j^2 \rm{Var}(X_n) = \frac{4\sigma^2}{n^2(n+1)^2}\bigg(\frac{n(n+1)(2n+1)}{6}\bigg) = \frac{2\sigma^2}{3}\bigg(\frac{2n+1}{n(n+1)}\bigg) \leq \frac{2\sigma^2}{3}\bigg(\frac{2}{3}\bigg) = \sigma^2 ,\forall n$  Then it follows by Tchebychev's Weak Law of Large Numbers.  My doubt is in the last step. Isn't  Tchebychev's WLLN only true for uncorrelated random variables? In this case, $\mathbb{E}(Y_n Y_m) \neq \mu^2$, since they aren't independent, so $\rm{Cov}(Y_n,Y_m) \neq 0$, right?  Thanks.  Edit: As u/clarinetist noted, there's a problem in this exercise, since the WLLN states the convergence of the arithmetic mean of random variables, whose expected value isn't $\mu$.
76285
I am referring to this  page 198-9:    I have troubles understanding the case with unequal variance.  In general it :   or on :    Therefore if we assume unequal variance, I would think like this:  $s_{\bar{X}_T-\bar{X}_C}=\sqrt{s^2_T/n_1+s^2_C/n_2}$  Howerver, they give this for the standard error:    I have troubles with deriving these two different formulas.
226
On page 19 of the textbook  (by James, Witten, Hastie and Tibshirani--it is freely downloadable on the web, and very good), the following is stated:  Consider a given estimate $$\hat{Y} = \hat{f}(x)$$ Assume for a moment that both $$\hat{f}, X$$ are fixed. Then, it is easy to show that: $$\mathrm{E}(Y - \hat{Y})^2 = \mathrm{E}[f(X) + \epsilon - \hat{f}(X)]^2$$ $$ = [f(X) - \hat{f}(X)]^2 + \mathrm{Var}(\epsilon)$$  It is further explained that the first term represents the reducible error, and the second term represents the irreducible error. I am not fully understanding how the authors arrive at this answer. I worked through the calculations as follows: $$\mathrm{E}(Y - \hat{Y})^2 = \mathrm{E}[f(X) + \epsilon - \hat{f}(X)]^2$$ This simplifies to $[f(X) - \hat{f}(X) + \mathrm{E}[\epsilon]]^2 = [f(X) - \hat{f}(X)]^2$ assuming that $\mathrm{E}[\epsilon] = 0$. Where is the $\mathrm{Var}(x)$ indicated in the text coming from? Any suggestions would be greatly appreciated.
96308
I have a distribution $X$. By playing around with random samples from $X$, I've determined that $Var(X^i) &gt; Var(X)$ where $i &gt; 1$. However, I can't seem to find a formula for the expected variance of $X^i$, or why it should be greater than $Var(X)$.  Moving away from normal distributions, should it generalize that the scale parameter of any $X^i$ will be greater than $X$?