In a regression, when you get negative coefficient which you know should be positive, why it is necessary to include possible omitted variable that is likely to have positive coefficient and negatively correlated with the variable already in the model? How will this change the sign of the variable?  For example, if the dependent variable is car price, and the independent variable is fuel economy (with the wrong negative sign). Why it is advised to include perhaps curb weight which is negatively correlated with fuel economy and positively with car price?
16129
In my logistic regression the sign of coefficients of a variable (location distance of an amenity) changes based on other variables (with time -ve, with travel distance +ve) in the model. When the location distance is the only variable in the model, it has +ve sign.    Should the variable need to maintain the +ve sign no matter what other variables are added in the model? Does changing sign signify a multicollinearity issue? Some IVs are gaining significance while in a bivariate model, they didn't show significance and vice versa. Is it okay to add variables that don't have much significance (ex: travel distance has a significance of 0.33 individually, but 0.05 when added with other variables) but becomes significant in the model? thanks.
59968
I don't even know if this question makes sense, but what is the difference between multiple regression and partial correlation (apart from the obvious differences between correlation and regression, which is not what I am aiming at)?  I want to figure out the following: I have two independent variables ($x_1$, $x_2$) and one dependent variable ($y$). Now individually the independent variables are not correlated with the dependent variable. But for a given $x_1$ $y$ decreases when $x_2$ decreases. So do I analyze that by means of multiple regression or partial correlation?  edit to hopefully improve my question: I am trying to understand the difference between multiple regression and partial correlation. So, when $y$ decreases for a given $x_1$ when $x_2$ decreases, is that due to the combined effect of $x_1$ and $x_2$ on $y$ (multiple regression) or is it due to removing the effect of $x_1$ (partial correlation)?
60614
Lets assume you are a social science researcher/econometrician trying to find relevant predictors of demand for a service. You have 2 outcome/dependent variables describing the demand (using the service yes/no, and the number of occasions). You have 10 predictor/independent variables that could theoretically explain the demand (e.g., age, sex, income, price, race, etc). Running two separate  multiple regressions will yield 20 coefficients estimations and their p-values. With enough independent variables in your regressions you would sooner or later find at least one variable with a statistically significant correlation between the dependent and independent variables.   My question: is it a good idea to correct the p-values for multiple tests if I want to include all independent variables in the regression? Any references to prior work are much appreciated.
13360
I create a linear regression model with interaction term in the model say: $$y=a_0+a_1x_1+a_2x_2+a_3x_1*x_2+e$$ where $x_1$ is continuous and $x_2$ is binary. Now, I have couple of questions:   If $x_1$ and $x_2$ both are significant and have positive coefficients, that means that if $x_1$ and $x_2$ increase, then $y$ increases. What can I say if $x_1*x_2$ is statistically insignificant? Does it mean that increase in. Does it mean that for $x_2=1$, the effect of $x_1$ on $y$ is not changed, i.e. increase in $x_1$ still leads to increase in $y$? What if the interaction is statistically significant? What can I say if the interaction term is statistically insignificant and negative? Does it indicate that if $x_2=1$, increase in $x_1$ actually leads to decrease in $y$?   Thanks.
53893
Say we fit a simple linear regression without the intercept term. I know this is inadvisable for the most part, unless there are some situations where it is reasonable to assume that when $x=0$, $y$ is also 0. I have been reading that when fitting without the intercept, it may cause the residual term to have non-zero mean. It makes sense but I can't seem to prove to myself (out of personal curiosity) why this is true. Can anyone help me understand it through a proof or some equations?
51049
I'm using linear regression to predict a price which is obviously positive. I have only one feature which is gross_area. I standardized it (z-score) I got this kind of value:  array([[ 1.        , -0.48311432],        [ 1.        ,  0.68052306],        [ 1.        ,  2.1426852 ],        [ 1.        , -1.17398593],        [ 1.        , -0.16265712]])   Where the 1 is the constant for the intercept term. I predict the parameters(predictors) and I got this:  array([[ 31780004.85045217],        [ 27347542.4693376 ]])   Where the first cell is the intercept term and the second cell correspond to the parameter found for my feature gross_area.  My problem is the following, when I take for example the fourth line and I compute the matrix multiplication XB to get my prediction, I got this:  In [797]: np.dot(training[4], theta) Out[797]: array([-325625.35640697])   Which is totally wrong since I cannot have negative value for my dependent variable. It seems like because of my normalization where I got negative value for my feature, I ended up with a negative predicted value for some tuple. How can is it possible and how can I fix this ? Thank you.  This is what I have predict graphically:   with y=price , x =gross area
57576
If I have a linear regression of the form:  $$y \sim 1+\beta_1x_1 + \beta_2x_1x_2,$$  where $x_2$ is a Boolean variable depending on another variable $x_{2cont}$, a positive variable:  $$x_2 = \begin{cases} 1 &amp; \mbox{if } x_{2cont} &gt; 5 \\ 0 &amp; \mbox{if } 0 &lt; x_{2cont} \le 5. \end{cases}$$  And I got regression results. How shall I interpret the regression results?  esp. how shall I interpret the two coefficients I get from this regression,  $\beta_1 \text{ and } \beta_2$?
3668
I have a question regarding interpretation of regression results based on data where I have some negative values. Since the residuals were positively skewed I needed to log-transform my data, and because of negative values my log transformation looked like $\ln(Y-\min(Y)+1)$. Normally, if we have $\ln$ in both sides of the equation, then our $b$ coefficient means a percentage change of DV for 1 % change of IV. But I have big doubts whether the same is valid for such log-transformed data, where I added a constant to my DV in order to make the minimum value equal to 1. I can't also find any literature on this topic. Does anyone know for sure what would be the correct interpretation in such a situation?
11648
I have a numerical dependent variable, and many independent variables. Most of my independent variables are dummy variables, but I have some categorical and numerical variables, too. I tried forward and backward model selection in R, but R returns my model empty! When I run separate simple regressions, there seems to be a significant relationship between my independent and dependent variables!  My question is: Am I going to have biased results if I run separate simple regressions with each variable, and then run a multiple regression with all the significant variables?
74353
I understand from this question -    that the R squared value of a linear regression model can be negative if the intercept is constrained. And this makes sense if you define R squared as -   $$R^2 = 1-\frac{SSE}{SST}$$  One says $SSE&gt;SST$. But then,  $$SST = SSA + SSE $$ Total Sum of squares  = Sum of squared errors + Sum of squared residual. And with this we get -  $$R^2 = \frac{SSA}{SST}$$ And now it is hard to imagine how $R^2$ can be negative. Aren't SSA and SST >0 always?
25992
Give some examples where a simple correlation coefficient has a sign opposite to that of the corresponding partial correlation coefficient and comment on it.   It is a question from an examination paper.Please help.
85929
If I want to investigate how two continuous variables are linked, what is the difference between calculating the correlation coefficient (Pearson's $r$) versus calculating the (simple linear) regression coefficient?  I see people who, if the regression coefficient is significantly different from zero, talk about the two variables as if they are correlated, which is confusing as it suggests that the two coefficients (correlation, regression) are the same thing.  Having said that, isn't $r$ a measure of the (regression line) slope anyway? I'm confused!!
118865
I want to mean center my interaction terms in a regression model (i.e., make the mean zero for each variable). I understand that I am supposed to mean center my variables first and then multiply them together to create my interaction term. But is it a problem that when I multiply two negative scores, I will have a positive score? I haven't been able to find a good answer to this. Thank you!
46340
In OLS regression, what does a one unit change in X represent in terms of a beta-related change in Y when the outcome variable has been transformed to $1/\sqrt(Y)$?   (Transformation was done to ensure linearity in the model. My predictor variables are 0-1 dummies.)  I've consulted the following useful posts, but I still don't know what is the answer:
88114
I'm not used to using variables in the date format in R. I'm just wondering if it is possible to add a date variable as an explanatory variable in a linear regression model. If it's possible, how can we interpret the coefficient? Is it the effect of one day on the outcome variable?   See my  with an example what I'm trying to do.
19575
X and Y are not correlated (-.01); however, when I place X in a multiple regression predicting Y, alongside three (A, B, C) other (related) variables, X and two other variables (A, B) are significant predictors of Y. Note that the two other (A, B) variables are significantly correlated with Y outside of the regression.   How should I interpret these findings? X predicts unique variance in Y, but since these are not correlated (Pearson), it is somehow difficult to interpret.  I know of opposite cases (i.e., two variables are correlated but regression is not significant) and those are relatively simpler to understand from a theoretical and statistical perspective. Note that some of the predictors are quite correlated (e.g., .70) but not to the extent that I would expect substantial multicollinearity. Maybe I am mistaken, though.    NOTE: I asked this question previously and it was closed. The rational was that this question is redundant with the question "". Perhaps I do not understand the other question, but I believe these are entirely separate questions, both mathematically and theoretically. My question is entirely independent from if "a regression is significant". Furthermore, several predictors are significant, while the other question entails variables not being significant, so I don't see the overlap. If these questions are redundant for reasons I do not understand, please insert a comment prior to closing this question. Also, I was hoping to message the moderator who closed the other question to avoid identical questions, but I couldn't find an option to do so.
40268
I am currently working to build a model using a multiple linear regression. After fiddling around with my model, I am unsure how to best determine which variables to keep and which to remove.  My model started with 10 predictors for the DV. When using all 10 predictors, four were considered significant. If I remove only some of the obviously-incorrect predictors, some of my predictors that were not initially significant become significant. Which leads me to my question: How does one go about determining which predictors to include in their model? It seemed to me you should run the model once with all predictors, remove those that are not significant, and then rerun. But if removing only some of those predictors makes others significant, I am left wondering if I am taking the wrong approach to all this.  I believe that  thread is similar to my question, but I am unsure I am interpreting the discussion correctly. Perhaps this is more of an experimental design topic, but maybe someone has some experience they can share.