I needed some help trying to understand why the sum of the observed values $Y_i$ equals the sum of the estimated values $\hat{Y}_i$.
55834
I'm taking a course on regression models and one of the properties provided for linear regression is that the residuals always sum to zero when an intercept is included.   Can someone provide a good explanation for why this is the case?
63286
Background:   I'm trying to follow .  I understand the basics of MLE estimation: likelihood, score, observed and expected Fisher information and the Fisher scoring technique. And I know how to justify .    The question:   I can't understand even the first line of this method :(   What's the intuition behind the $z_i$ working variables defined as:  $$ z_i = \hat\eta_i + (y_i -\hat\mu_i)\frac{d\eta_i}{d\mu_i}$$  Why are they used instead of $y_i$ to estimate $\beta$?   And what's their relation with the response/link function which is the connection between $\eta$ and $\mu$  If anyone has a simple explanation or can direct me to a more basic-level text about this I would be grateful.
43327
I'm trying to see why this is the case. I have taken  the LHS and added $X\hat{\beta}$ and subtracted $X\hat{\beta}$.  From this I can get both terms on the right but I end up with a cross product term  $2(y-X\hat{\beta})^T(X\hat{\beta} - X\beta)$  that I can't prove equals to zero. I think there is some trick I'm missing, anybody remember it?
62559
I'm just getting started with some machine learning, and until now I have been dealing with linear regression over one variable.  I have learnt that there is a hypothesis, which is:  $h_\theta(x)=\theta_0+\theta_1x$  To find out good values for the parameters $\theta_0$ and $\theta_1$ we want to minimize the difference between the calculated result and the actual result of our test data. So we subtract  $h_\theta(x^{(i)})-y^{(i)}$  for all $i$ from $1$ to $m$. Hence we calculate the sum over this difference and then calculate the average by multiplying the sum by $\frac{1}{m}$. So far, so good. This would result in:  $\frac{1}{m}\sum_{i=1}^mh_\theta(x^{(i)})-y^{(i)}$  But this is not what has been suggested. Instead the course suggests to take the square value of the difference, and to multiply by $\frac{1}{2m}$. So the formula is:  $\frac{1}{2m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})^2$  Why is that? Why do we use the square function here, and why do we multiply by $\frac{1}{2m}$ instead of $\frac{1}{m}$?
109242
How would I show that $$d_E = \sqrt{\sum_{i=1}^n(x_i-y_i)^2}$$ and $$d_\infty= \sum_{i=1}^n|x_i-y_i|$$ and $$\sup\{|x_1-y_1|,|x_2-y_2|,...,|x_n-y_n|\}$$ are topologically equivalent on $\mathbb{R}^n$? I can do it for $\mathbb{R}^2$, but I'm not sure how a generalization looks like.
53948
Doing a little reading over the break (The Probabilistic Method by Alon and Spencer); can't come up with the solution for this seemingly simple (and perhaps even a little surprising?) result:     (A-S 1.6.3) Prove that for every two independent identically distributed real random variables $X$ and $Y$,   $$Pr[|X-Y| \leq 2] \leq 3 Pr[|X-Y| \leq 1].$$
43100
I'm having a little trouble with the proof that the expected value of $x_i$ is $ \bar{X} $.  What I have is   $E[x_i]=\sum_{j=1}^{N}X_j Pr(x_i=X_j) $  Then,  $Pr(x_i=X_j) = 1/N $  This is the bit I can't understand, how does that probability evaluate to that value.  I know the denominator is how many ways you can choose n out N. I think that the numerator should be how many ways you can choose (n-1) out of (N-1). But I seem to have an extra n.  Any advice ?
67801
Let $Y_1, Y_2, ..., Y_n$ be iid random variables and $B_1, B_2, ..., B_n$ be Borel sets. It follows that  $P(\bigcap_{i=1}^{n} (Y_i \in B_i)) = \Pi_{i=1}^{n} P(Y_i \in B_i)$...I think?  If so, does the converse hold true? My Stochastic Calculus professor says it does (or maybe misinterpreted him somehow?),  I was  the  that independence of the n random variables was equivalent to saying for any indices $i_1, i_2, ..., i_k$ $P(\bigcap_{j=i_1}^{i_k} (Y_j \in B_j)) = \Pi_{j=i_1}^{i_k} P(Y_j \in B_j)$.  So, if the RVs are independent, then we can choose $i_j=j$ and k=n to get $P(\bigcap_{i=1}^{n} (Y_i \in B_i)) = \Pi_{i=1}^{n} P(Y_i \in B_i)$, but given $P(\bigcap_{i=1}^{n} (Y_i \in B_i)) = \Pi_{i=1}^{n} P(Y_i \in B_i)$, I don't know how to conclude that for any indices $i_1, i_2, ..., i_n$ $P(\bigcap_{j=i_1}^{i_k} (Y_j \in B_j)) = \Pi_{j=i_1}^{i_k} P(Y_j \in B_j)$, if that's even the right definition.  p.17  seems to suggest otherwise. idk  Help please?  Also this:    or    So, this answer is to use the Omega part to establish pairwise independence and ultimately conclude independence. Without that assumption, we cannot conclude independence. Is that right? Why does that not contradict the definition of independence: $P(\bigcap_{i=1}^{n} (Y_i \in B_i)) = \Pi_{i=1}^{n} P(Y_i \in B_i)$ ?
25326
I am trying to show that $$\sum_{i=1}^ne_i = 0$$  using matrices (or vectors).  I have two hints, so to speak:  $$ HX = X$$ where $H$ is the hat matrix, and that $$\sum_{i=1}^ne_i = e'1$$  My previous solution, , is wrong since I expanded $Y =X\beta$, leaving out the error term. If I include it I end up with a tautology.
73007
I know I mess up different things. However I want to get better understanding what is motivation behind logistic regression.  $p(y_i=1|x_i)=(1+e^{-x_iw})^{-1}$  and according to OLS  $S(w)=\sum_{i}^{}(y_i-\hat{y_i})^2$,where $y \in \{1,0\}$.  What if $\hat{y_i}=p(y_i=1|x_i)$ and estimate $w$ by finding minimum to $S(w)$.  Why this is incorrect approach?
67777
Here's an  from one of the questions in the newsletter.  I'm not trying to suggest that it's incorrect, I just don't understand why it's an acceptable practice.  I know that simplifying assumptions are made all the time, but why is this useful?  \begin{equation} \begin{split} p(\theta|y) &amp;= \frac{p(y|\theta)p(\theta)}{p(y)} \\ &amp;\propto\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}\theta^{\alpha-1}(1-\theta)^{\beta-1}*\binom{n}{y}\theta^y(1-\theta)^{n-y} \\ &amp;\propto\theta^{\alpha-1}(1-\theta)^{\beta-1}*\theta^y(1-\theta)^{n-y} \\ &amp;\propto\theta^{\alpha+y-1}(1-\theta)^{\beta+n-y-1} \\ &amp;=\frac{\Gamma(\alpha+y-1)\Gamma(\beta+n-y-1)}{\Gamma(\alpha+\beta+n-1)}\theta^{\alpha+y-1}(1-\theta)^{\beta+n-y-1} \end{split} \end{equation}  My issue is that we go from "proportional-to" to "equals", which is fine, but what exactly is this saying?  As in, I understand how we go from each line to the line following it, but I don't understand what this tells us.  Sure, we eventually get back to a Beta distribution, but only after removing a ton of values.  Are these values not important?  Does it mean anything profound that a Beta prior with a binomial likelihood has the property of conjugacy?  It can make calculations simpler, sure, but is this useful in practice, if so why?      EDIT 1: So are these constants actually thrown out, or accounted for later? I understand that constants don't matter with respect to proportionality and integration, but throwing them out completely just seems counter-intuitive to me. Sure, if you're multiplying/dividing a bunch of numbers by an expression with a constant, and all you care about is the relationship of those numbers to each other, then you can throw out constants and the relationship remains the same. Is that what's going on here?  EDIT 2: From what I can tell, we get rid of constants, simplify, and then take the integral and normalize that integral. By virtue of the normalization, we're inherently accounting for the constants that we removed before. Is this correct? I think I understand now, for whatever reason, I kept thinking of the original equation as a scalar value rather than a probability distribution.
910
I stumbled across this relationship while I was messing around. What's the proof, and how do I understand it intuitively? It doesn't really make sense to me that the sum of odd numbers up to $2x + 1$ should equal $x^2$.
11523
$E[X|Y]$ is equal to the almost surely unique and deterministic function of $Y$, say $\varphi (Y)$, such that $$E[X f(Y)]= E[\varphi (Y)f(Y)]$$ for all bounded, measurable and non-negative $f$'s. How can I prove this fact? In particular, how the a.s. uniqueness and the equality of the conditional expectation to $\varphi(Y)$ follows from the definition of the conditional expectation defined as usual by $$E[X|Y]=\sum_x x\cdot P(X=x|Y)?$$
82294
I have a simple question, but it is hard to google it. I have this equation here:  $$y(t, x) = \sum_{i=1}^{d}(|x_i| \wedge t)^{2} $$  Here $x$ is a size $d$ signal and $t$ is just a scalar. I am not sure how to read that equation in english... I understand everything except for how they use the $\wedge$ here...  If context helps, this is part of a cost function, based on a threshold $t$ that is selected, for your vector $x$.
76920
Suppose for two random variables $X_1$ and $X_2$ $\sigma(X_1) = \sigma(X_2)$. Why $E[Y | X_1] = E[Y | X_2]$ a.e. ?  the set where $X_1= X_1(\omega)$ can be different from the set $X_2= X_2(\omega) $. How can then conditional expectation conditioned on the event can be equal ?
37847
Actually I have no idea how to begin. I tried to represent it as a product of two series or as a sum of two series but have no result with this. Any idea ?
49792
Let $X_{i} \in \{0,1\}$ be Bernouli random variable with probability of success $p_{i}$, i.e., $P(X_{i}=1) = p_{i}$ and $P(X_{i}=0) = 1-p_{i}$ and let $Y=\sum_{i=1}^{n}X_{i}$ for $n&gt;0$. Is it relatively simple to estimate the pmf $P(Y=y)$ when $p_{i} \neq p_{j}$ for $i \neq j$?