In linear regression we can use A:B to show the first order interaction of variable A and B. But it is hard to know what effect was caused by A and what was caused by B. So I want to understand how exactly A:B worked
17017
I have a model: $$ \ln({\rm earnings}) = a+b_1{\rm female}+b_2{\rm white}+b_3{\rm female}\times{\rm white} $$ ${\rm female}$ and ${\rm white}$ are dummy variables.  I have interpreted $b_1$ and $b_2$:    $b_1$ = change in female earnings comparing to male given you are non white $b_2$ = change in white earnings comparing to non white given you are male   But I am unable to interpret the coefficient of the interaction term ($b_3$). Please help me with this.  Let me make it more clear what I need out of this regression $$ \ln({\rm earnings}) = 2.618656-.0899657{\rm female}+.382019{\rm white}-.2754126 {\rm female}\times{\rm white} $$ Now i know there is gender pay difference with b1, i also know there is race pay difference with b2. Now with b3 i need to know is their a gender pay gap for whites only. How can i figure that out with regression above and without test.
108108
Consider model $logit(p)=a+bx$. I would like to get an analytic formula of $a$ and $b$ like in linear regression. In linear regression, we can get a formula of estimates of $a$ and $b$.  I tried using MLE. But it is too complicated for me.  I am doing research about the statistical genetics. Here $p$ is penetrance and $x$ is genotype. In linear regression, we can use the coefficient $b$ to represent the covariance matrix of $x$ and trait $y$ (both of them should be standardized). I am thinking whether we can use the $b$ in logistic model to recover the information about the covariance matrix of $x$ and $y$ (binary trait) in meta-analysis (they do not provide individual level data). But some cohort will prove the odds ratio and coefficient t-statistic. I do not know how to use it to recover the covariance matrix of $x$ and $y$.   But some summary statistics will be provided like estimated odds ratio, Coefficient t-statistic.
33133
Consider the general linear regression model: $$y_i = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2} + \cdots + \beta_px_{ip} + \epsilon_i = \mathbf{x}_i^t \beta + \epsilon_i$$  where $\textbf{x}_i = (1,x_{i1},x_{i2},\cdots,x_{ip})^T$, $\beta=(\beta_0,\beta_1,\cdots,\beta_p)^T$ and $\epsilon_i$ are iid N(0,$\sigma^2$).  I would like to see a complete proof of the following identity from first principles: $$\sum_{i=1}^n(y_i - \bar{y})^2 = \sum_{i=1}^n(\hat{y}_i - \bar{y})^2 + \sum_{i=1}^n(y_i - \hat{y}_i)^2$$ where $\hat{y}_i= \mathbf{x}_i^t \hat{\beta} $ ($\hat{\beta}$ is the least square estimator, $\bar{y}$ ia the sample mean of $y_i$).  I know that the two terms on the right can be obtained by subtracting and adding $\hat{y}_i$ on the left side. But this introduces a "cross term": $$\sum_{i=1}^n2(\hat{y}_i - \bar{y})(y_i - \hat{y}_i)$$  Many texts claim that this is zero, but I have not seen a general proof of this statement. How can this be shown?
47949
Like, I don't get it. In regression, we say  $$Y_i = a + bX_i + \epsilon$$ where $X_i$ is just a real number for each observation $i$,  and in analysis of variance with two groups, we say $$Y_i = a + bX_i + \epsilon$$ where $X_i$ is again just a real number, but this time either 0 (group 1) or 1 (group 2), so a is mean of group 1, and a+b is mean of group 2.  But despite the interpretation, the model is exactly the same. So why do people split it up into regression and anova? the model is the same?
118865
I want to mean center my interaction terms in a regression model (i.e., make the mean zero for each variable). I understand that I am supposed to mean center my variables first and then multiply them together to create my interaction term. But is it a problem that when I multiply two negative scores, I will have a positive score? I haven't been able to find a good answer to this. Thank you!
95518
There's regression model where $Y = a + bX$ with $a = 1.6$ and $b=0.4$, which has a correlation coefficient of $r = 0.60302$.  If $X$ and $Y$ are then switched around and the equation becomes $X = c + dY$ where $c=0.4545$ and $d=0.9091$, it also has an $r$ value of $0.60302$.  I'm hoping someone can explain why $(d\times b)^{0.5}$ is also $0.60302$.
46340
In OLS regression, what does a one unit change in X represent in terms of a beta-related change in Y when the outcome variable has been transformed to $1/\sqrt(Y)$?   (Transformation was done to ensure linearity in the model. My predictor variables are 0-1 dummies.)  I've consulted the following useful posts, but I still don't know what is the answer:
4119
The coefficient of an explanatory variable in a multiple regression tells us the relationship of that explanatory variable with the dependent variable. All this, while 'controlling' for the other explanatory variables.  How I have viewed it so far:  While each coefficient is being calculated, the other variables are not taken into account, so I consider them to be ignored.   So am I right when I think that the terms 'controlled' and 'ignored' can be used interchangeably?
824
I have multiple regression with, say 3 independent variables: $Y=B_0+B_1x_1+B_2x_2+B_3x_3$ I would like to test if $B_2+3B_3$ is significantly different from zero, i.e.  $$H_0: B_2+3B_3=0$$ $$H_1: B_2+3B_3\neq 0$$ Can you please help to find appropriate way to test for significance of linear functions of two coefficients as in above example.  Many thanks in advance.
113453
I'm taking an intro to statistics class and currently we are covering linear regression. The course is good but unfortunately it doesn't always gives all the information that is needed. For example it gives formulas for the slope and y-intercept as  $b=(r)\frac{s_Y}{s_X}=\frac{n \sum{xy} - \sum{x} \sum{y}}{n \sum{x^2} - (\sum{x})^2}$ , where  $r$ is the correlation coefficient  $s_Y$ is the standard deviation of the $Y$ scores  $s_X$ is the standard deviation of the $X$ scores   and  $a = \bar{y} - b \bar{x} = \frac{\sum{y} - b \sum{x}}{n}$  However it doesn't explain why the formulas are what they are. In other words it doesn't explain how these formulas where derived. Can someone show the derivation, especially the one for the slope, $b$, i.e. why $b=(r) \frac{s_Y}{s_X}$?  Any help is appreciated.
4753
I am reading Elements of Statistical Learning, and they derive the equation for linear regression using differentiation on a function which takes a matrix as input. This is making my head spin a bit and I am looking for some resources to practice this type of differentiation, and become more comfortable with it.  This is the problem. Let $X \in \mathbb{R}^{N \times p}$, $b \in \mathbb{R}^{p \times k}$, and $y \in \mathbb{R}^{N \times k}$.  Define $RSS(X, y) = \operatorname{tr}[(Xb - y)^t(Xb -  y)]$.  I want to differentiate $RSS$ with respect to $b$ and set the derivative to zero since I want to minimize the function, $RSS$.  The book claims this is $X^t(Xb - y) = 0$, which gets us, $b = (X^tX)^{-1}Xy$ if $(X^tX)$ is singular. It does not show how it applied the derivatives (which is expected since thats not what this book is about).  I can sort of reason that we have some sort of product rule happening. It seems likely that $d/db(\operatorname{tr}(A)) = \operatorname{tr}(d/db(A))$. But right now I am just writing down symbols that somehow represent some ideas in my head.  In my Principles of Mathematical Analysis book (by Rudin) he covers when the input is a vector, and the output can be a vector. However, this seems to be input is a matrix, and the output in some of the subproblems is a matrix.  Any recommendations for books I can look at?
83784
I have two series of measurements values, first series is X and second is Y. I need to model Y as a function of X, where I know the method that was used to measure X is two times better then the method used for measure Y in terms of error variance.  I red about regression with error term in the independent variable, but I'm not sure what to do when there is also error term in the dependent variable and they are different.  Which technique/tools/constraints should I use for this sort of case?
17284
I'm seeking your help in understanding the difference between two different contrasts for dichotomous variables.  On this page:  under "Dichotomous Predictor Variables", there are two ways to code dichotomous predictors: using the contrast 0,1 or the contrast 1,-1. I kind of understand the distinction here (0,1 is dummy coding and 1,-1 adds to one group and subtracts from the other) but don't understand which to use in my regression.  For example, if I have two dichotomous predictors, gender (m/f) and athlete (y/n), I could use contrasts 0,1 on both or 1,-1 on both. What would be the interpretation of a main effect or an interaction effect when using the two different contrasts? Does it depend on whether my cells are of different sizes?
13478
The first sentence of this  page claims that  "In econometrics, an endogeneity problem occurs when an explanatory variable is correlated with the error term. "  My question is that how can this ever happen? Isn't regression beta chosen such that the error term is orthogonal to the column space of the design matrix?
25121
I am teaching myself regression using Regression Modeling Strategies by Harell and the author goes at quite the length to showcase the importance of modeling interactions and transformations of the initial variables. I can't help but wonder how to approach this in a more structured/automated way when dealing with a lot of potential variables.  Can we use recursive partitioning, for example, to somehow to do the work for us and then use the output as variables, shrink the estimates with LASSO to deal with colinearity and do a final step where we use some sort of filtering for feature importance.  In my mind this will leave us with a well specified model which can be manually inspected and improved if need be, but is this reasonable? Are there other ways to approach this? Are there some resources that deal with problems like this?
78199
Other than literally testing each possible combination of variable(s) in a model (x1:x2 or x1*x2 ... xn-1 * xn). How do you identify if an interaction SHOULD or COULD exist between your independent (hopefully) variables?   What are best practices in attempting to identify interactions? Is there a graphical technique that you could or do use?
66760
I have a linear regression model where the dependent variable is logged and an independent variable is linear. The slope coefficient for a key independent variable is negative: $-.0564$. Not sure how to interpret.   Do I use the absolute value then turn it into a negative like this: $(\exp(0.0564)-1) \cdot 100 = 5.80$  or  Do I plug in the negative coefficient like this: $(\exp(-0.0564)-1) \cdot 100 = -5.48$  In other words, do I use the absolute figure and then turn that into a negative or do I plug in the negative coefficient? How would I phrase my findings in terms of a one-unit increase in X is associated with a __ percent decrease in Y? As you can see, these two formulas produce 2 different answers.