I am going through the book  and I'm finding it extremely terse. I have a background in probability but not statistics so perhaps that is why. Anyway, in Chapter 3, section 3.2 (p.47), they discuss the unbiased estimator of the variance of the $y_i$. They write,      Typically one estimates the variance $\sigma^2$ by   $$ \hat{\sigma}^2 = \frac{1}{N − p − 1} \sum_{i = 1}^N ( y_i − \hat{y}_i)^2.$$   The $N − p − 1$ rather than $N$ in the denominator makes $\hat{\sigma}^2$ an unbiased estimate of $\sigma^2$.   Here the $y_i$ are the observations which we assume are distributed like  $ x_i^T \beta + \zeta_i$, where $x_i$ are the inputs, $\zeta_i$ are independent of everything, mean zero, variance $\sigma^2$, and $\beta$ is fixed. $\hat{y}_i$ are the estimated output from the linear regression we have constructed. Ie., $\hat{y}_i = x_i^T \hat{\beta}$ with $\hat{\beta}$ the linear multiplier, $\hat{\beta} = (X^T X)^{-1}X^T y$.  They make this comment without any explanation. Can someone explain why this is true. And even better, why this should be obvious.   Edit: Thanks to @gung for the link but I found that calculation still unclear and missing details. I'm putting down my calculation here just for completeness.  Some notation: Let $X$ be the $N \times (p+1)$ matrix of samples by characteristics (the extra $1$ is for the column of all ones!). Let $y = (y_1,...,y_N)$ and $\hat{y} = (\hat{y}_1,...,\hat{y}_N)$.   Under these conditions, $\hat{y} = X \hat{\beta}$ and $y = X \beta + \zeta$ where $\zeta$ is an $n\times 1$ vector of independent Gaussians with common variance $\sigma^2$. Using our formula for $\hat{\beta}$ above this gives $$ \begin{align} \hat{\beta} = (X^TX)^{-1}X^T y = \beta + (X^TX)^{-1}X^T \zeta \\ \end{align} $$ And so,  $$ \begin{align} y - \hat{y} &amp; = X\beta + \zeta - (X\beta + X(X^TX)^{-1}X^T \zeta) \\ &amp; = (I_N - X(X^TX)^{-1}X^T) \zeta \\ &amp; := (I_N - B) \zeta, \end{align} $$ with $B:= X(X^TX)^{-1}X^T$. Therefore, $$ \begin{align} \sum_{i=1}^N \mathbb{E} (y_i - \hat{y}_i)^2 &amp; = \mathbb{E} \left[(y-\hat{y})^T(y-\hat{y}) \right] \\ &amp; =\mathbb{E} \left[ \zeta^T (I_N-B)^T (I_N-B) \zeta \right] \\ &amp; = \sigma^2 Tr((I_N-B)^T(I_N- B)) . \end{align} $$ The last equality follows because the $\zeta$ are independent with common variance $\sigma^2$.  Finally, $$ \begin{align} (I_N-B)^T(I_N-B) &amp;= I_N - X(X^TX)^{-1}X^T. \\ \end{align} $$ Therefore, $$ \begin{align} Tr((I_N-B)^T(I_N-B)) &amp; = Tr(I_N) - Tr(X(X^TX)^{-1}X^T) \\ &amp; = Tr(I_N) - Tr(I_{p+1}) \\ &amp; =N-p-1 \end{align} $$
I am unable to find the bias of the sample variance estimator in this problem. Unfortunately I keep coming up with   Question: Suppose that the true linear model is   $y = X_1\beta_1 + X_2\beta_2 + e$,   where $e \sim MVN(0,\sigma^2 I)$, $X_1: N \times p_1, \beta_1: p_1 \times 1, X_2: N \times p_2, \beta_2: p_2 \times 1$. But we wrongly fit the model  $y = X_1\beta_1+e$ and calculate the estimators of $\beta_1$ and $\sigma^2$ as:  $\hat{\beta}_1 = (X_1'X_1)^{-1}X_1'y$,  $s^2 = \large \frac{y'[I-X_1(X_1'X_1)^{-1}X_1']y}{N-p_1}$.  (a) Show that, in general, $\hat{\beta}_1$ is a biased estimator with  Bias$(\hat{\beta}_1) = (X_1'X_1)^{-1}X_1'X_2\hat{\beta}_2$  Under what condition on $X_1$ and $X_2$ is $\hat{\beta}_1$ unbiased?  (b) Show that, in general, $s^2$ is a biased estimator with   Bias$(s^2) = \frac{1}{N-p_1}\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_2\beta_2$  Is the bias positive or negative? Under what condition on $\beta_2$ is $s^2$ unbiased?  Attempt:  For part (a):  Bias$(\hat{\beta}_1) = E((X_1'X_1)^{-1}X_1'y - \beta_1)$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'E(y) - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'(X_1\beta_1 + X_2\beta_2) - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'X_1\beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2) - \beta_1$  $\hspace{15mm}= I\beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2 - \beta_1$  $\hspace{15mm}= \beta_1 + (X_1'X_1)^{-1}X_1'X_2\beta_2 - \beta_1$  $\hspace{15mm}= (X_1'X_1)^{-1}X_1'X_2\beta_2$  If $X_1$ and $X_2$ are orthogonal, then $\hat{\beta}_1$ is unbiased.  For part (b), I am unable to figure out how to derive:  Bias$(s^2) = \frac{1}{N-p_1}\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_2\beta_2$  I can get   $\frac{1}{N-p_1}\beta_1'X_1'+\beta_2'X_2'[I-X_1(X_1'X_1)^{-1}X_1']X_1\beta_1+X_2\beta_2$  after taking the expectation of y on both sides.   Any help would be much appreciated.
What is the correlation if the standard deviation of one variable is 0? I am a seafarer (marine engineer) and I have to often visit different countries. This time I received my sign-off from Iran and they stamped my passport. Would this cause a problem for getting a US C1D1 visa since my next company wants me to have this visa? I am an indian national. How can I modify the advanced search form when my hook runs too early? I'm using  to modify the content of the form on the search page &mdash; I want to remove the "phrase" and "not" fields, and restrict the "only of the types" checkboxes to a hand selected list of types.  I have the following code, which is being called and which I think should work:  function MYMODULE_form_search_form_alter(&amp;$form, &amp;$form_state, $form_id) {     unset($form['basic']);     unset($form['advanced']['keywords']['phrase']);     unset($form['advanced']['keywords']['negative']);     unset($form['advanced']['type']);     $form['advanced']['type'] = array(         '#type' =&gt; 'checkboxes',         '#title' =&gt; t('Only of the type(s)'),         '#options' =&gt; array(             'blog' =&gt; 'Blog entry',             'event' =&gt; 'Event',             'news' =&gt; 'News article'         )     );     // print_r($form);     // die; }   Except  gets to the search form after my form alter hook runs, so I don't get a chance to modify any of the form elements that it's adding.  I'm running Drupal 7.22.  How can I modify the advanced search form when my module hook runs before the hook that defines the advanced search form? Eigenvalues of Householder matrix What would be the eigenvalues for a Householder matrix defined as: $H = I - 2 u u^T$? Can someone explain it  to me intuitively or with a simple proof? Why should a CTE start with a semi-colon? I was just looking at a post on  where Aaron Bertrand proposes using a CTE instead of a numbers table, which is an elegant way of performing the task at hand.  My question is, why does the first line of the CTE begin with a semi-colon?    ;WITH n AS (SELECT TOP (10000) n FROM    (SELECT n = ROW_NUMBER() OVER     (ORDER BY s1.[object_id])     FROM sys.all_objects AS s1     CROSS JOIN sys.all_objects AS s2   ) AS x ORDER BY n ) SELECT n FROM n ORDER BY n; -- look ma, no gaps!   Is this to ensure the WITH statement does not get parsed into a previous SELECT or something?  I see nothing in SQL Server 2005 BOL about using a semi-colon prior to the WITH. options for appearance of links in hyperref Linking files in g++ How do I execute a string containing Python code in Python? How do I execute a string containing Python code in Python? How to model an entity type that can have different sets of attributes? I accidentally removed my /etc/apt/sources.list file. Is it hosted by Canonical somewhere so I can re-add it to the system? It's a bit of an emergency, so I hope someone can provide a quick answer. What is a good reference for rigorous Electromagnetism and Electrodynamics? Is there any good book on Electromagnetism from a more mathematical point of view? By this I mean a book which makes use of differential forms and maybe De Rham cohomology. I was also searching for something that would include E&amp;M in the big picture of Gauge theories. Graphics horribly corrupted on Ubuntu 16.04 in VirtualBox, showing random strips of pixelated colors Inside a VirtualBox VM, I have installed myself an Ubuntu 16.04.1 LTS Server installation. However, when I installed the ubuntu-desktop package and rebooted, this happened:    Exact command I used: sudo apt-get update; sudo apt-get install ubuntu-desktop ..and then a reboot command.  This just seems like a corrupted top-left portion of the screen, as my cursor can appear in view (but corrupted):    This is a purely graphical issue because I can use the keyboard to successfully progress to a desktop:    Note: On startup it shows correct graphics for a split-second then reverts to glitch.    Any help on this? When a holomorphy ring is a PID? I will use the notation and language of Stichtenoth, Algebraic Function Fields and Codes.  Let $F$ be a function field over a finite field $\mathbb F_q$, $S$ a non empty set of places (possibly infinite) and $O_S$ the holomorphy ring of $S$, i.e., $O_S:=\bigcap_{P\in S} O_P$ (being $O_P$ the valuation ring of the place $P$).  Give necessary and sufficient conditions for $O_S$ to be a Principal Ideal Domain.  Many thanks in advance!  G. How do I show that $\sum_{cyc} \frac {a^6}{b^2 + c^2} \ge \frac {abc(a + b + c)}2?$ Let $a, b, c$ be positive real numbers, show that  $$\frac {a^6}{b^2 + c^2} + \frac {b^6}{c^2 + a^2} + \frac {c^6}{a^2 + b^2} \ge \frac {abc(a + b + c)}2.$$  I think this is likely to turn out to be proved by Hölder, but I can't see how. Any hints will be appreciated. Use of "Proper" as a postpositive adjective I have an inkling that the following word usage is grammatically-correct; however, it is immensely difficult to search for confirmation, as all I receive for queries like &quot;Proper as an adjective&quot; are reams of results on &quot;Proper adjectives&quot;. The word itself I am using as a postpositive adjective, similar in form to the phrase &quot;time immemorial&quot;.  The shelter remains in name only; the shelter proper is in another town. Work in theory remained, although operations proper had mostly been outsourced.  Can anyone advise? Can't access global variable inside function Statistics interview questions Why doesn't my Bash script recognize aliases? Why is my Wi-Fi process using 3GB of memory? How do you get \mathbb{1} to work (characteristic function of a set)? I am trying to get the digit one printed the \mathbb{} font in math mode, however the command  \mathbb{1}   outputs some rubbish. I certainly saw the character I am after in some pdf's generated from latex, hence my question: how do you get it to work?