I have the same results for two regression  1) y= number of board directors in t+1 x= a dummy variable with 1 if there is at least an institutional investor among the shareholders in t.   2) y= a dummy variable with 1 if there is at least an institutional investor among the shareholders in t+1 X= number of board directors in t  The coefficient of the two regression are positive and significant.   I want to know which conclusion can I write regarding the causality of the relation.
123696
I know correlation does not imply causation but instead the strength and direction of the relationship. Does simple linear regression imply causation? Or is an inferential (t-test, etc.) statistical test required for that?
92227
In a multiple regression, the regression coefficient of the two subsamples is opposite to that of the full sample. Both coefficients are significant. The sum of the number of obs in the two subsamples is equal to that of the full sample. Is this normal? What are the possible causes? Thanks.
17017
I have a model: $$ \ln({\rm earnings}) = a+b_1{\rm female}+b_2{\rm white}+b_3{\rm female}\times{\rm white} $$ ${\rm female}$ and ${\rm white}$ are dummy variables.  I have interpreted $b_1$ and $b_2$:    $b_1$ = change in female earnings comparing to male given you are non white $b_2$ = change in white earnings comparing to non white given you are male   But I am unable to interpret the coefficient of the interaction term ($b_3$). Please help me with this.  Let me make it more clear what I need out of this regression $$ \ln({\rm earnings}) = 2.618656-.0899657{\rm female}+.382019{\rm white}-.2754126 {\rm female}\times{\rm white} $$ Now i know there is gender pay difference with b1, i also know there is race pay difference with b2. Now with b3 i need to know is their a gender pay gap for whites only. How can i figure that out with regression above and without test.
74140
In the univariate regression settings, we try to model   $$y = X\beta +noise$$  where $y \in \mathbb{R}^n$ a vector of $n$ observations and $X \in \mathbb{R}^{n \times m}$ the design matrix with $m$ predictors. The solution is $\beta_0 = (X^TX)^{-1}Xy$.  In the multivariate regression settings, we try to model   $$Y = X\beta +noise$$  where $y \in \mathbb{R}^{n \times p}$ is a matrix of $n$ observations and $p$ different latent variables. The solution is $\beta_0 = (X^TX)^{-1}XY$.  My question is how is that different than performing $p$ different univariate linear regression? I read  that in the latter case we take into consideration correlation between the dependent variables, but I don't see it from the math.
59171
In multiple linear regression, I can understand the correlations between residual and predictors are zero, but what is the expected correlation between residual and the criterion variable? Should it expected to be zero or highly correlated? What's the meaning of that?
104108
Consider multiple linear regression. This question might be deceptively simple, but I'm trying to intuitively understand why, say if I have predictors X1 and X2, then interactions between these predictors can be adequately captured by X1 * X2.  I know interaction terms are modelled as products, just because that's what I was taught in school, and that's what everyone says to do.  I'm guessing there is maybe some geometric argument.  But why is a product (of say two numeric features, and not the extra complexity of multiplying by one being a dummy variable while other is numeric etc) going to adequately capture interactions?  Why are not "interactions" best captured by another f(X1, X2) by default instead of specifically X1 * X2?  I can see the idea that X1 * X2 may capture situations where the signs of X1 and X2 are the same or not, but then why would not, say, by default interactions be modelled by say f(X1, X2) = sign(X1) * sign(X2) instead of f(X1, X2) = X1X2?  I realise I can add any other f(X1, X2) to a regression or any predictive model, but finding the exact shape of interactions by hand coding is time consuming. How do I know X1X2 is a good first guess?
41718
I did the multiple linear regression on a dataset of 412 observations, with one response variable (Y) and 25 explanatory variables(X1-X25). Y and most of Xs are not normally distributed. Besides, there are some correlation between several Xs. The plot show that the residuals strongly correlated with Y positively and weakly correlated with fitted Y negatively.(Sorry.As I'm newer in this website, I am n't allowed to post images.)  To address these problems, I have tried the principle component regression, the weighted least squared regression and the ridge regression. They all didn't work. I want to know what's wrong with the regression. Why did the residuals correlate with observed Y so obvious?
111569
A is positively related to B.  C is the outcome of A and B, but the effect of A on C is negative and the effect of B on C is positive.  Can this happen?
120459
Does the following qualify as a univariate regression?  $$y=b_0+b_1x+b_2x^2+\epsilon$$  I fully comprehend the implications of adding regressors and need no background information - a "yes" or "no" will do :-)
19290
I am doing a simple marketing project that has the following types of variables:   X1 - continuous (e.g. income) X2 - categorical (e.g. gender) Y - continuous (e.g. number of a product type purchased such as tubs of ice-cream)    I am interested in the relationship between income (X1) and product purchase (Y) but also the effect of gender (X2) on this relationship. (i.e. interaction or moderation effect).  I have centered X1 and have used the general linear model in SPSS. The result on Y is as follows:   X1 - significant X2 - not significant X1*X2 - not significant   How do I interpret this result in terms of main effect and interaction?
73646
I learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant. For example, assume the size of a house and the number of bedrooms are correlated. When predicting the cost of a house using these two predictors, one of them can be dropped because they are both providing a lot of the same information. Intuitively, this makes sense, but I have a some more technical questions:   How does this effect manifest itself in p-values of the regression coefficients when including only one or including both predictors in the model? How does the variance of the regression coefficients get affected by including both predictors in the model or just having one? How do I know which predictor the model will choose to be less significant? How does including only one or including both predictors change the value/variance of my forecasted cost?
3418
The standard scheme of instrumental variable in terms of causality (-&gt;) is:  Z -&gt; X -&gt; Y  Where Z is an instrument, X an endogenous variable, and Y a response.  Is it possible, that following relations:  Z &lt;- X -&gt;Y  Z &lt;-&gt; X -&gt;Y  are also valid?  While correlation between instrument and variable is satisfied, how may I think of exclusion restriction in such cases?    NOTE: The notation &lt;-&gt; is not explicit and might lead to different understandings of the problem. Still, the answers highlight this issue and use it to show important aspects of the problem. When reading, please proceed with caution about this part of the question.
14012
Is it possible to have a (multiple) regression equation with two or more dependent variables?  Sure, you could run two separate regression equations, one for each DV, but that doesn't seem like it would capture any relationship between the two DVs?
35899
I have a set of predictors in a linear regression, as well as three control variables. The issue here is that one of my variables of interest is only statistically significant if the control variables are included in the final model. However, the control variables themselves are not statistically significant.  Here is how the multicollinearity of all my variables look like (including control variables):   &gt; vif(lm(return ~ EQ + EFF + SIZE + MOM + MSCR + UMP, data = as.data.frame(port.df)))        EQ      EFF     SIZE      MOM     MSCR      UMP   3.687171 3.481672 2.781901 1.064312 1.438596 1.003408   &gt; vif(lm(return ~ EQ + MOM + MSCR, data = as.data.frame(port.df)))        EQ      MOM     MSCR   1.359992 1.048142 1.412658    My variables of interest are EQ, MOM and MSCR, and the control variables are EFF, SIZE and UMP. EQ is only significant if the three control var are included, and becomes insignificant when they are not:   Here are the coefficients (1rst row) and t-stats (2nd row) when control variables are included (notice that EQ is statistically significant)         intercept           EQ          EFF        SIZE         MOM       MSCR          UMP [1,] 0.005206246 -0.006310531 0.0001229055 0.004125551 0.007738259 0.00473377 5.838596e-06 [2,] 1.866628909 -1.746583234 0.0388823612 1.178460997 2.145062820 2.08131100 1.994863e-01  Now, here is the result of the regression when the control variables are excluded (notice that EQ is NOT statistically significant anymore)         intercept           EQ         MOM       MSCR [1,] 0.007313402 -0.002111833 0.007128606 0.00668364 [2,] 2.652662996 -0.595391117 2.036985378 2.80177366    The problem is that when I include my control variables, all my variables of interest are significant, but my control variables are not.  Which variables should I include in my final model? How should I structure my final model then, given the fact that the model will be used for forecasting?  Thank you,
73628
Let's say that there exists some "true" relationship between $y$ and $x$ such that $y = ax + b + \epsilon$, where $a$ and $b$ are constants and $\epsilon$ is i.i.d normal noise. When I randomly generate data from that R code: x &lt;- 1:100; y &lt;- ax + b + rnorm(length(x)) and then fit a model like y ~ x, I obviously get reasonably good estimates for $a$ and $b$.  If I switch the role of the variables as in (x ~ y), however, and then rewrite the result for $y$ to be a function of $x$, the resulting slope is always steeper (either more negative or more positive) than that estimated by the y ~ x regression. I'm trying to understand exactly why that is and would appreciate it if anyone could give me an intuition as to what's going on there.
39424
When I analyze my variables in two separate (univariate) logistic regression models, I get the following:  Predictor 1:    B= 1.049,    SE=.352,    Exp(B)=2.85,    95% CI=(1.43, 5.69),    p=.003    Constant:    B=-0.434,    SE=.217,    Exp(B)=0.65,                            p=.046  Predictor 2:    B= 1.379,    SE=.386,    Exp(B)=3.97,    95% CI=(1.86, 8.47),    p&lt;.001    Constant:    B=-0.447,    SE=.205,    Exp(B)=0.64,                            p=.029   but when I enter them into a single multiple logistic regression model, I get:  Predictor 1:    B= 0.556,    SE=.406,    Exp(B)=1.74,    95% CI=(0.79, 3.86),    p=.171 Predictor 2:    B= 1.094,    SE=.436,    Exp(B)=2.99,    95% CI=(1.27, 7.02),    p=.012    Constant:    B=-0.574,    SE=.227,    Exp(B)=0.56,                            p=.012   Both predictors are dichotomous (categorical). I have checked for multicollinearity.  I am not sure if I have given enough info, but I cannot understand why predictor 1 has gone from being significant to non-significant and why the odds ratios are so different in the multiple regression model. Can anyone provide a basic explanation of what is going on?
29900
Specifically, I want to know if there is a difference between lm(y ~ x1 + x2) and glm(y ~ x1 + x2, family=gaussian). I think that this particular case of glm is equal to lm. Am I wrong?