Lets consider normalized variables X and Y. Slope of a lm(Y~X) is Cor(Y,X)*sd(Y)/sd(X) and for lm(X~Y) its Cor(X,Y)*sd(X)/sd(Y). Since sd(X) = sd(Y) = 1. The slopes of lm(Y~X) and lm(X~Y) are always bound to be equal.   That would mean the regression lines for both of the models will be, $Y = mX$ and $X = mY$.   Lets take an example.   x = rnorm(20) y = rnorm(20) x = (x-mean(x))/sd(x) y = (y-mean(y))/sd(y) lm(x~y)  Call: lm(formula = x ~ y)  Coefficients: (Intercept)            y     4.333e-17   -1.272e-01    lm(y~x)  Call: lm(formula = y ~ x)  Coefficients: (Intercept)            x    -4.333e-17     -1.272e-01      Suppose we have two normalized variables X and Y as shown above. They have Cor(X,Y) = Cor(Y,X) = -0.127.   If I use X as a predictor variable of Y, I get Y = -0.127X; then, I expect X = -1/0.127 Y.  If I use X as a predictor variable of Y, I get Y = -0.127X;    It looks like both statements are inconsistent. How to interpret this result?
1643
The Pearson correlation coefficient of x and y is the same, whether you compute pearson(x, y) or pearson(y, x). This suggests that doing a linear regression of y given x or x given y should be the same, but I don't think that's the case.   Can someone shed light on when the relationship is not symmetric, and how that relates to the Pearson correlation coefficient (which I always think of as summarizing the best fit line)?
119859
I have a model similar to below with a continuous outcome, continuous predictor and a categorical variable. the goal of this analysis is to make inferences on the group variable. The issue is that y is bounded by x in the form of y >= -x. When the model makes predictions you can see in the plot below that the predicted value is outside the possible range. See the last plot.   How can this be remedied? A custom loss function to penalize going beyond the bound and use gradient descent regression? Or is there some other way? Thank you.   x =   rep(seq(-10,10,1),each=5)  y =  rep(0,length(x) ) weights = sample( seq(1,1000,1) ,length(x)) groups = rep( letters[1:5], times =length(x)/5  )   ## the outcome y has the relationship y &gt;= -x for(i in 1:length(x)){   y[i] = sample(  seq(-x[i],-x[i]+ ifelse(x[i]&lt;0 ,                                            sample(seq(1,10,.1),1),                                           sample(seq(5,40,.1),1) ),1)                        ,1) }   dat =data.frame(x = x, y = y, weights = weights, group = groups) model = lm(dat = dat, y ~ x + group + group * x, weights = weights) dat$FIT= predict(model, dat, allow.new.levels = TRUE)  library(ggplot2) ggplot(data = dat, aes(x = x, y = y, color = group))+geom_point( aes(size = weights))+ geom_line(data = dat, aes(x = x, y = FIT, color = group))    #data for new predictions newdata = data.frame( x = seq(-100,100,1) ,                        group = rep("c", length( seq(-100,100,1) )),                       bound = -seq(-100,100,1)                       )  ## when pridicting new data the outcome y has the relationship y &gt;= -x ## but the predicted vales are below the bound. See how red line is below blue to the left of zero new_preds  = predict(model, newdata, allow.new.levels = TRUE) ggplot(data = newdata, aes(x = x, y = new_preds, color = group))+geom_line()  +   geom_line(data = newdata, aes(x = x, y = bound, color = "green"))
85929
If I want to investigate how two continuous variables are linked, what is the difference between calculating the correlation coefficient (Pearson's $r$) versus calculating the (simple linear) regression coefficient?  I see people who, if the regression coefficient is significantly different from zero, talk about the two variables as if they are correlated, which is confusing as it suggests that the two coefficients (correlation, regression) are the same thing.  Having said that, isn't $r$ a measure of the (regression line) slope anyway? I'm confused!!
70317
I have the following model: $$ |X| = B_0 +B_1 \cdot y + B_2 \cdot z , $$ where $z$ and $y$ are normally distributed random variables, and $B_1$ and $B_2$ denote the coefficients.   My dependent variable contains either the positive or negative values of $X$. The positive and negative values of $X$ together are normally distributed, with a mean and median of around zero. For statistical reasons I would like to separately test the positive and negative values. This splits the left and right side of the otherwise normally distributed variable $X$. This clearly violates the normality assumption used for linear regression models. If I want to compare groups I would use a nonparametric type of test, but in this case I want to run the equivalent of a linear regression.  Is this possible?
78937
Being aware of , I am curious about the question how big standardized coefficients can get. I had a discussion with my professor about that issue and she was arguing standardized coefficients (beta) in multiple linear regressions can not become greater than |1|. I have also heard that predictors with standardized coefficients greater than 1 should not be be included/appear in multiple linear regression. When I recently estimated a multiple linear regression in R using lm(), I estimated the standardized coefficients with lm.beta() function from the package 'lm.beta'. In the results I could observe a standardized coefficient greater than one. Right now I am just not sure about what is the truth.  Can standardized coefficients become greater than |1|? If yes, what does that mean and should they be excluded from the model? If yes, why?  I would be very thankful, if somebody could make this issue clear for me.
79370
I have carried out a linear regression. The plot below shows the distribution of the response variable:    I believe the response variable is beta distributed, therefore virtually the exact opposite of normally distributed. However, when including all my predictor variables in the linear regression, the residuals turn out to be quite normally distributed, as shown in this plot:    Has my model satisfied the assumptions of linear regression? Might there be a better model to use?
100272
I am a little bit confused on what the assumptions of linear regression are. So far I checked whether:  all of the explanatory variables correlated linearly with the response variable. (This was the case) there was any collinearity among the explanatory variables. (there was little collinearity). the Cook's distances of the datapoints of my model are below 1 (this is the case, all distances are below 0.4, so no influence points). the residuals are normally distributed. (this may not be the case)  But I then read the following:  violations of normality often arise either because (a) the distributions of the dependent and/or independent variables are themselves significantly non-normal, and/or (b) the linearity assumption is violated.  Question 1 This makes it sound as if the independent and depend variables need to be normally distributed, but as far as I know this is not the case. My dependent variable as well as one of my independent variables are not normally distributed. Should they be? Question 2 My QQnormal plot of the residuals look like this:   That slightly differs from a normal distribution and the shapiro.test also rejects the null hypothesis that the residuals are from a normal distribution: &gt; shapiro.test(residuals(lmresult)) W = 0.9171, p-value = 3.618e-06  The residuals vs fitted values look like:   What can I do if my residuals are not normally distributed? Does it mean the linear model is entirely useless?
116552
I am trying to run a linear regression with both categorical and continuous predictors. I have coded the categorical predictor (with three levels) into three dummy variables, and entered the two dummy variables into the regression along with the continuous predictors. My question is: How do I interpret the beta coefficients of the categorical predictors? I can interpret the other beta coefficients from the continuous predictors as "_% of the variance in x can be explained by y" but I am not sure how to interpret the beta from the categorical predictors. The t statistic for both of my dummy variables were significant, but I'm having difficulty interpreting it in the context of the full regression. Any help would be appreciated!  Thank you!
76957
This question is inspired from the long discussion in comments here:    In the usual linear regression model, for simplicity here written with only one predictor: $$      Y_i = \beta_0 + \beta_1 x_i + \epsilon_i $$ where the $x_i$ are known constants and $\epsilon_i$ are zero-mean independent error terms. If we in addition assume normal distributions for the errors, then the usual least squares estimators and the maximum likelihood estimators of $\beta_0, \beta_1$ are identical.  So my easy question: do there exist any other distribution for the error terms such that the mle are identical with the ordinary least squaeres estimator?  The one implication is easy to show, the other one not so.
3668
I have a question regarding interpretation of regression results based on data where I have some negative values. Since the residuals were positively skewed I needed to log-transform my data, and because of negative values my log transformation looked like $\ln(Y-\min(Y)+1)$. Normally, if we have $\ln$ in both sides of the equation, then our $b$ coefficient means a percentage change of DV for 1 % change of IV. But I have big doubts whether the same is valid for such log-transformed data, where I added a constant to my DV in order to make the minimum value equal to 1. I can't also find any literature on this topic. Does anyone know for sure what would be the correct interpretation in such a situation?
88765
I have a question about the interpretation of the coefficients of an interaction between continuous and categorical variable. here is my model:  model_glm3=glm(cog~lg_hag+race+pdg+sex+as.factor(educa)+(lg_hag:as.factor(educa)),                 data=base_708)  Coefficients:                          Estimate Std. Error t value Pr(&gt;|t|)     (Intercept)               21.4836     2.0698  10.380  &lt; 2e-16 *** lg_hag                     8.5691     3.7688   2.274  0.02334 *   raceblack                 -8.4715     1.7482  -4.846 1.61e-06 *** racemexican               -3.0483     1.7073  -1.785  0.07469 .   racemulti/other           -4.6002     2.3098  -1.992  0.04687 *   pdg                        2.8038     0.4268   6.570 1.10e-10 *** sexfemale                  4.5691     1.1203   4.078 5.15e-05 *** as.factor(educa)2         13.8266     2.6362   5.245 2.17e-07 *** as.factor(educa)3         21.7913     2.4424   8.922  &lt; 2e-16 *** as.factor(educa)4         19.0179     2.5219   7.541 1.74e-13 *** as.factor(educa)5         23.7470     2.7406   8.665  &lt; 2e-16 *** lg_hag:as.factor(educa)2 -21.2224     6.5904  -3.220  0.00135 **  lg_hag:as.factor(educa)3 -19.8083     6.1255  -3.234  0.00129 **  lg_hag:as.factor(educa)4  -8.5502     6.6018  -1.295  0.19577     lg_hag:as.factor(educa)5 -17.2230     6.3711  -2.703  0.00706 ***   lets say the equation of the model is:  E[cog] = a + b1(lg_hag) + b2(educa2*lg_hag) + b3(educa3*lg_hag) + b4(educa4*lg_hag)  + b5(pdg, centered) + other covars, where  b1 = difference in cog  with higher lg_hag among lowest education (coded as 1) b1 + b2 = difference in cog with higher lg_hag among middle education (coded as 2) b1 + b3 = difference in cog with higher lg_hag among high education (coded as 3) b1 + b3 = difference in cog with higher lg_hag among very high education (coded as 4) b5 = difference in cog with each unit increase in pdg   My question is: if my interpretation is right, how to construct confidence intervals for each effect estimate of interactions (e.g: b1+b2) from the confidence intervals of b1 and b2.
50407
Consider the  function     $$ r(x) = \mathbb{E}(Y \mid X = x) $$   This has been called the regression function in a textbook I'm using. I'm trying to figure out the relationship between this function and the classical linear regression model.  So, I know that it is a theorem* that we may write  $$ Y = r(X) + \epsilon $$  for some random variable $\epsilon$ s.t. $\mathbb{E}(\epsilon) = 0$.  Now suppose that we have     $$ Y = \beta_0 + \beta_1 X + \epsilon $$   This is the classical 1-dimensional regression function (assuming the $\beta_0$ and $\beta_1$ minimize the residual sum of squares).  Question: Is it then a mathematical theorem that if $Y$ is defined as above, that   $$ r(X) = \mathbb{E}(Y \mid X) = (\beta_0 + \beta_1 X)? $$  And is this why the function $\mathbb{E}(Y \mid X)$ is called the "regression function"?  EDIT: The theorem that I am making use of is as follows (from All of Statistics pg. 89):     Regression models are sometimes written as      $$ Y = r(X) + \epsilon $$      where $\mathbb{E}(\epsilon) = 0$. We can always rewrite a regression model this way. To see this, define $\epsilon = Y - r(X)$ and hence $Y = Y + r(X) - r(X) = r(X) + \epsilon$. Moreover, $\mathbb{E}(\epsilon) = \mathbb{E}\mathbb{E}(\epsilon \mid X) = \mathbb{E}(\mathbb{E}(Y - r(X)) \mid X) = \mathbb{E}(\mathbb{E} ( Y \mid X) - r(X)) = \mathbb{E}(r(X) - r(X)) = 0$.
28401
I am trying to understand the origin of the curved shaped of confidence bands associated with an OLS linear regression and how it relates to the confidence intervals of the regression parameters (slope and intercept), for example (using R):  require(visreg) fit &lt;- lm(Ozone ~ Solar.R,data=airquality) visreg(fit)     It appears that the band is related to the limits of the lines calculated with the 2.5% intercept, and the 97.5% slope, as well as with the 97.5% intercept, and the 2.5% slope (although not quite):  xnew &lt;- seq(0,400) int &lt;- confint(fit) lines(xnew, (int[1,2]+int[2,1]*xnew)) lines(xnew, (int[1,1]+int[2,2]*xnew))     What I don't understand are two things:   What about the combination of 2.5% slope &amp; 2.5% intercept as well as 97.5% slope and 97.5% intercept? These give lines that are clearly outside the band plotted above. Maybe I don't understand the meaning of a confidence interval, but if in 95%  of the cases my estimates are within the confidence interval, these seem like a possible outcome? What determines the minimum distance between the upper and lower limit (i.e. close to the point where the two lines added above intercept)?    I guess both questions arise because I don't know/understand how these bands are actually calculated.   How can I calculate the upper and lower limits using the confidence intervals of the regression parameters (without relying on predict() or a similar function, i.e. by hand)? I tried to decipher the predict.lm function in R, but the coding is beyond me. I'd appreciate any pointers towards relevant literature or explanations suitable for stats beginners.  Thanks.
14909
I want to perform a very simple linear regression in R. The formula is as simple as $y = ax + b$. However I would like the slope ($a$) to be inside an interval, let's say, between 1.4 and 1.6.  How can this be done?
77219
Brief caveat- I haven't dusted off my stats knowledge since some university courses a few years ago, and I'm struggling with cobwebs.  I have a model where a linear 1 to 1 relationship has been assumed in the past. The model assumes that revenue from Product A will translate to Product B- it's not perfect, but it's what we've been using for now. I was given the task of doing some exploratory work on this relationship.   I modelled the relationship between revenue B and revenue A, and was able to work out a rough coeffecient. This coeffecient seems to be significant (t test within bounds), but I know the t test is testing whether the slope of this regression line is significantly different than zero. I would like to know if this slope is significantly different than 1, the previous relationship we assumed. How can I adjust a hypothesis test for this? I know I'm going to see the answer and hate myself for asking, thanks for helping me shake this out of my brain.
95518
There's regression model where $Y = a + bX$ with $a = 1.6$ and $b=0.4$, which has a correlation coefficient of $r = 0.60302$.  If $X$ and $Y$ are then switched around and the equation becomes $X = c + dY$ where $c=0.4545$ and $d=0.9091$, it also has an $r$ value of $0.60302$.  I'm hoping someone can explain why $(d\times b)^{0.5}$ is also $0.60302$.
53325
I have read many threads here on how to interpret coefficients in a regression where the predictor and the dependent variable are log-transformed. Most give an answer for a one or ten percent change. However, I am not sure whether I am right about interpreting other changes.  I have a beta coefficient of $-0.5057$. Given the standard interpretation (as elasticity), a one percent increase in the predictor variable ($X$) leads to an decrease of $0.5057$ percent in the dependent variable ($Y$).  Alternatively a ten percent increase in $X$ leads to a $5.057$ percent decrease in $Y$.  I found the formulas $1.1^{-0.5057}$ and $\exp(-0.5057(\log(1.1)))$ to calculate the affect of any change in $X$ on $Y$. Are these formulas right because I get for a ten percent increase slightly different results than the $5.057$ decrease.  Furthermore, is it right that the percentage values for the changes in $Y$ do not increase proportionally with increases in $X$? If a one percent increase in $X$ leads to an decrease of $0.5057$ percent in $Y$, it is wrong to assume that a $50$ percent increase in $X$ leads to a $(50 \times  0.5057)$ decrease in $Y$?   $\exp(-0.5057(\log(1.5)))= 0.8146$. Given the difference between $1$ and this value, it suggests that $Y$ decreases by $18.5$ percent. Am I right?  The formulas mentioned above suggest so. Is that because of the nonlinearity in the model?