I have some questions. In a linear model, I want to force intercept to zero. The program (I used JMP) does not provide R-squared when intercept becomes zero.  So, I calculated R-squared by myself by given SSR/SST in the program. The R-squared was higher than before forcing to zero.  Is this possible? Because linear fitting is based on minimizing error, so if I change it artificially, I think R-squared should be decreased. Also, the linear line (Green line) for intercept 0 on the graph, seems not to have 99% R-squared.  If this R-squared is not correct, how can I calculate R-squared when forcing intercept to zero?   Many thanks,
31641
In a simple linear model with a single explanatory variable,  $\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$  I find that removing the intercept term improves the fit greatly (value of $R^2$ goes from 0.3 to 0.9). However, the intercept term appears to be statistically significant.  With intercept:   Call: lm(formula = alpha ~ delta, data = cf)  Residuals:      Min       1Q   Median       3Q      Max  -0.72138 -0.15619 -0.03744  0.14189  0.70305   Coefficients:             Estimate Std. Error t value Pr(&gt;|t|)     (Intercept)  0.48408    0.05397    8.97   &lt;2e-16 *** delta        0.46112    0.04595   10.04   &lt;2e-16 *** --- Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1   Residual standard error: 0.2435 on 218 degrees of freedom Multiple R-squared: 0.316,    Adjusted R-squared: 0.3129  F-statistic: 100.7 on 1 and 218 DF,  p-value: &lt; 2.2e-16    Without intercept:   Call: lm(formula = alpha ~ 0 + delta, data = cf)  Residuals:      Min       1Q   Median       3Q      Max  -0.92474 -0.15021  0.05114  0.21078  0.85480   Coefficients:       Estimate Std. Error t value Pr(&gt;|t|)     delta  0.85374    0.01632   52.33   &lt;2e-16 *** --- Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1   Residual standard error: 0.2842 on 219 degrees of freedom Multiple R-squared: 0.9259,   Adjusted R-squared: 0.9256  F-statistic:  2738 on 1 and 219 DF,  p-value: &lt; 2.2e-16    How would you interpret these results? Should an intercept term be included in the model or not?  Edit  Here's the residual sums of squares:  RSS(with intercept) = 12.92305 RSS(without intercept) = 17.69277
69494
Let's say I have some data, and then I fit the data with a model (a non-linear regression). Then I calculate the R-squared ($R^2$).  When R-squared is negative, what does that mean? Does that mean my model is bad? I know the range of $R^2$ can be [-1,1]. When $R^2$ is 0, what does that mean as well?
75992
I'm running some regression analyses and got pretty confused about R's output when it comes to robust regression models. When I run a OLS -- using the command lm() -- I get this as output:  Coefficients:                 Estimate Std. Error t value Pr(&gt;|t|)   But when I run a robust linear model using the rlm() command, the output looks like so:  Coefficients:                 Value   Std. Error t value   How do I get  the p-values and the significance-levels in an rlm then? Without that, the whole analysis is somewhat pointless. Unfortunately, I couldn't find an answer to that anywhere, so I hope someone around here can help me out. Thanks a lot!!
28195
I am trying to calibrate two variables $(X,Y)$ of different measuring techniques from two instruments, the result of the linear regression analysis appears as shown in the image. The result shows the regression constant is not statistically significant but the model is significant. I have tried to remove the regression constant (it is a very small value close to zero) and $R$ of the new model is raised to 90%. Is it correct to remove the regression constant?
21823
I've got a very small coefficient (-0.04) and R-squared (0.028) but a significant P value (&lt;0.0001). My question is:   Is my result still meaningful? How to interpret it?   The result is from a linear regression model in a big database in R.  The independent variable (B) has more than 200 values, whereas the dependent variable (A) has 13 values.   The potential correlation is below:   So, I ran a linear regression model between A and B and the result is:  &gt;mod1&lt;-lm(A~B)          Call:         lm(formula = A ~ B)           Residuals:              Min      1Q  Median      3Q     Max           -63.174 -11.816  -1.651  10.184 118.001            Coefficients:                        Estimate Std. Error t value Pr(&gt;|t|)              (Intercept) 132.274547   0.303723 435.511  &lt; 2e-16 ***          B            -0.036675   0.009052  -4.052 5.13e-05 ***          ---         Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1          Residual standard error: 18.52 on 8093 degrees of freedom            (123 observations deleted due to missingness)         Multiple R-squared:  0.002024,  Adjusted R-squared:  0.001901          F-statistic: 16.42 on 1 and 8093 DF,  p-value: 5.134e-05   As you can see, the coefficient of B is only -0.03 and adjusted R squared is only 0.1% but with a p value &lt;0.0001. Is my result reasonable and countable? Surely, my database is larger (8000 records) and even a very small effect size will show a significant P value. But how would I interpret this?
78617
I Googled this, but some answers still leave me confused. One of the answers says:       "I just think of it in terms of the fitted line having 2 parameters, the intercept and the slope. But the intercept doesn't count after we adjust for the mean (which we do in obtaining SStotal), so that leaves just one df for regression."   How can you drop the intercept after adjust for the mean?
45060
Ridge regression coefficient estimate $\hat{\beta}^R$ are the values that minimize the   $$ \text{RSS} + \lambda \sum_{j=1}^p\beta_j^2. $$  My questions are:   If $\lambda = 0$, then we see that the expression above reduces to the usual RSS. What if $\lambda \to \infty$?  I do not understand the textbook explanation of the behaviour of the coefficients. To aid in understanding the concept behind a particular term, why is the term called RIDGE Regression? (Why ridge?) And what could have been wrong with the usual/common regression that there is a need to introduce a new concept called ridge regression?    Your insights would be great.
97963
From the training output below, it looks like R squared is not calculated by the traditional formula 1 - SSE/SST, since lower error rate has lower R squared. So how is it calculated?    colsample_bytree  min_child_weight  RMSE      Rsquared    0.4               3                 16963.39  0.8799191   0.4               5                 16813.24  0.8788395
22322
What is the meaning of t value and Pr(&gt;|t|) when using summary() function on linear regression model in R?  Coefficients:                               Estimate Std. Error t value Pr(&gt;|t|)     (Intercept)                    10.1595     1.3603   7.469 1.11e-13 *** log(var)                        0.3422     0.1597   2.143   0.0322 *
50438
I'm giving a presentation about fitting lines.  I have a simple linear function, $y=1x+b$.  I'm trying to get scattered data points that I can put in a scatter plot that will keep my line of best fit the same equation.   I'd love to learn this technique in either R or Excel--whichever is easier.
97434
Many statistics textbooks state that adding more terms into a linear model always reduces the sum of squares and in turn increases the r-squared value. This has led to the use of the adjusted r-squared. But is it possible that adding a term into a linear model reduces the sum of squares by zero and in turns keep the r squared value exactly the same?
59453
If this is a duplicate question, please point to the right way, but the similar questions I've found here haven't been sufficiently similar. Suppose I estimate the model $$Y=\alpha + \beta X + u$$  and find that $\beta&gt;0$. However, it turns out that $X=X_1+X_2$, and I suspect $\partial Y/\partial X_1 \ne \partial Y / \partial X_2$, and in particular, that $\partial Y/\partial X_1 &gt; \partial Y / \partial X_2$. So I estimate the model $$Y=\alpha + \beta_1 X_1 + \beta_2 X_2 +u$$and find significant evidence for $\beta_1,\beta_2&gt;0$. How can I then test whether $\beta_1&gt; \beta_2$? I considered running another regression $$Y=\alpha +\gamma(X_1 - X_2) + u$$ And testing whether $\gamma&gt;0$. Is this the best way?  Also, I need to generalize the answer to many variables, i.e. suppose we have $$Y=\alpha + \beta^1 X^1 + \beta ^2 X^2 + \dots + \beta^nX^n + u$$ where for each $j=1,\dots,n$, $X^j=X_1^j+X_2^j$, and I would like to test for each $j$ whether $\partial Y/\partial X_1^j \ne \partial Y / \partial X_2^j$.  By the way, I am primarily working in R.
25999
Goal: create a plot to compare the empirical convergence rate of a numerical method with the one expected from the theory.  Accordingly, given a loglog-plot, I would like to add the line which has a prescribed slope (in this case, the convergence of the method) and the best intercept to fit the data (e.g., in a least-square sense).  Question: how to do a linear regression with fixed slope in pgfplots?  What I have done so far is the following: (1) compute the regression line (of the logarithm of the quantities), (2) save the intercept, (3) plot the line with the slope expected from the theory and this intercept.  This is a good approximation of what I am looking for, provided the data are in agreement with the theory.  Thanks  MdFB
41443
As the title to my question says, I am confused as to when the $R^2$ of a model fit does not equal the slope of the regression between observed and predicted values.   I am trying to present model prediction statistics in a similar way to those presented in the summary figures of the Globcolor validation report () - (e.g. figure from page 53 of the .pdf):    Here we see that they present the plot of observed versus predicted Chlorophyll concentrations, as well as statistics relating to its regression (e.g. the dashed line: $R^2$, $RMS$, $\alpha$ - intercept, and $\beta$ - slope).   My issue is that in my comparisons, I always get exactly the same value for the overall model fit $R^2$ and $\beta$-slope of the observed versus predicted regression.  Basic question: When (if ever) can these be different?  I have included a basic example of my problem in the following R script:  set.seed(1) n &lt;- 100 x &lt;- runif(n) e &lt;- rnorm(n) a &lt;- 3 b &lt;- 5 y &lt;- a + x*b + e  #fit model fit &lt;- lm( y ~ x )  #plot regression plot(x,y) abline(fit)  #plot predicted versus observed png("plot.png", units="in", width=5, height=5, res=400) par(mar=c(5,5,1,1)) pred &lt;- predict(fit) plot(y, pred, xlim=range(c(y,pred)), ylim=range(c(y,pred)), xlab="observed", ylab="predicted") abline(0,1, lwd=2, col=8)  #add regression fit2 &lt;- lm(pred ~ y) lgd &lt;- c(     paste("R^2 =", round(summary(fit2)$r.squared,3)),     paste("Offset =", round(coef(fit2)[1],3)),     paste("Slope =", round(coef(fit2)[2],3)) ) legend("topleft", legend=lgd) abline(fit2, lwd=2) legend("bottomright", legend=c("predicted ~ observed", "1:1"), col=c(1,8), lty=1, lwd=2)  dev.off()  cor(pred, y)^2 # also the same
103152
Assume a simple regression model, $y = \beta_0 + x\beta_1 + u$. I decide to change the units of measurement for the explanatory variable and the response variable. Do the $\beta_0$ and $\beta_1$ parameters change as well?  I assume that the $\beta$ parameters will stay constant.  Does anyone know for sure?  How does it change the value of $R^2$?
28105
Is it possible for more than one "linear regression" line to fit a given set of points? (i.e.... "least squares" is minimised equally in both cases)  I'm assuming a simple one-variable regression in which we have a single "dependent" variable for a given x-value. It may be that regression is forced "through the origin" for the purpose of the Q (if that matters), so, I know it wouldn't be a great fit.  Or is there something intrinsic about a regression line that means it is unique? I am interested in a 'proof' (could be just an intuitive proof) if it exists, that a regression line must be unique.
33462
I was skimming through  by Cosma Shalizi (in particular, section 2.1.1 of the ), and was reminded that you can get very low $R^2$ even when you have a completely linear model.  To paraphrase Shalizi's example: suppose you have a model $Y = aX + \epsilon$, where $a$ is known. Then $\newcommand{\Var}{\mathrm{Var}}\Var[Y] = a^2 \Var[x] + \Var[\epsilon]$ and the amount of explained variance is $a^2 \Var[X]$, so $R^2 = \frac{a^2 \Var[x]}{a^2 \Var[X] + \Var[\epsilon]}$. This goes to 0 as $\Var[X] \rightarrow 0$ and to 1 as $\Var[X] \rightarrow \infty$.  Conversely, you can get high $R^2$ even when your model is noticeably non-linear. (Anyone have a good example offhand?)  So when is $R^2$ a useful statistic, and when should it be ignored?