Sometimes in OLS model we have constant for example -2345 significant and doesn't have a mean. Why we must keep it in the model? Why when we drop it the results change? What does it mean?  And sometimes it is insignificant. Why?
11199
I am running linear regression models and wondering what the conditions are for removing the intercept term.   In comparing results from two different regressions where one has the intercept and the other does not, I notice that the $R^2$ of the function without the intercept is much higher. Are there certain conditions or assumptions I should be following to make sure the removal of the intercept term is valid?
41509
I have 4 IVs in my model that directly effect the DV. The  of the correlation &amp; regression analyses showed that:  IV1&amp;DV:   Pearson Correlation Coefficient: insignificant  Regression Beta&amp;t values: significant  IV2&amp;DV:  Pearson Correlation Coefficient: significant  Regression Beta&amp;t values: insignificant  IV3&amp;DV:   Pearson Correlation Coefficient: significant  Regression Beta&amp;t values: significant  IV4&amp;DV:   Pearson Correlation Coefficient: significant  Regression Beta&amp;t values: significant  No multicollinearity problem was detected and also all the regression assumptions have been perfectly met!  Why the results of the regression analysis turned out to be completely opposite of correlation analysis for IV1 and IV2?!! Why they're being contradictory?! Does it make any sense? Is it acceptable? TQ.
104155
Why is that? Why $TSS = ESS + RSS$ ,iff only we have an intercept(constant term) in our regression model?Why it doesn't work, when model doesn't include intercept?
6412
Recently, randomly browsing questions triggered a memory of on off-hand comment from one of my professors a few years back warning about the usage of ratios in regression models. So I started reading up on this, leading eventually to Kronmal 1993.  I want to make sure that I'm correctly interpreting his suggestions on how to model these.   For a model with a ratio with the same denominator on both the dependent and independent side: $ Z^{-1}Y = Z^{-1}1_n\beta_0 + Z^{-1}X\beta_X + \beta_Z + Z^{-1}\epsilon $   Regress dependent ratio on the (inverse) denominator variable in addition to the other ratios Weight by the (inverse) denominator variable  For a model with dependent variable as a ratio: $ Y = \beta_0 + \beta_XX + Z1_n\alpha_0 + ZX\alpha_X + Z^{-1}\epsilon $   Regress numerator by original variables, denominator, and denominator times original variables [what about categorical variables?] Weight by (inverse) denominator  For model with only independent variable ratios: $ Y = \beta_0 + X\beta_X + Z^{-1}1_n\beta_{Z^{-1}} + W\beta_W  + Z^{-1}W\beta_{Z^{-1}W} + \epsilon $   Include numerator and (inverse) denominator as main effects, ratio as interaction term.    Are my interpretations here correct?
97434
Many statistics textbooks state that adding more terms into a linear model always reduces the sum of squares and in turn increases the r-squared value. This has led to the use of the adjusted r-squared. But is it possible that adding a term into a linear model reduces the sum of squares by zero and in turns keep the r squared value exactly the same?
97425
I have a question:      Does  $$N(0, x)$$ mean a normal distribution with mean $0$ and variance $x$? Or standard deviation $x$?   The notation seems ambiguous sometimes.
16129
In my logistic regression the sign of coefficients of a variable (location distance of an amenity) changes based on other variables (with time -ve, with travel distance +ve) in the model. When the location distance is the only variable in the model, it has +ve sign.    Should the variable need to maintain the +ve sign no matter what other variables are added in the model? Does changing sign signify a multicollinearity issue? Some IVs are gaining significance while in a bivariate model, they didn't show significance and vice versa. Is it okay to add variables that don't have much significance (ex: travel distance has a significance of 0.33 individually, but 0.05 when added with other variables) but becomes significant in the model? thanks.
117656
If I have the below functional form for an OLS regression, how do I interpret the $x$ and $x^2$? I cannot interpret them separately, correct? Do I interpret them as a summation of the two coefficients, so the effect of a one unit change of $x$ on $y$ is $0.5 + 0.3 = 0.8$?  $$ y = a + 0.5x + 0.3x^2 + e  $$    Update: Now with 2 squared terms, and one that is an interaction term with the other and a binary variable, is there no rule for how to interpret the coefficients for all values of $x$? Or does it depend on the value of $x$? \begin{align} y = 0.04\; &amp;+\; 0.05\times {\rm drop}\; +\; 0.008\times {\rm season}\; -\; 0.0004\times {\rm season}^2    \\          &amp;+\; 0.002\times {\rm dropseason}\; -\; 0.0003\times {\rm dropseason}^2\; -\; 0.029\times p   \\          &amp;-\; 0.01\times r_1\; +\; 0.003\times r_2\; +\; 0.01\times r_3\; +\; 0.003\times v_5\; +\; e \end{align}
92525
Someone claims that the  adjusted $R^2$ will increase with the addition of an extra variable.  I wonder why, as it is called adjusted (in contrast to the normal $R^2$).   The only condition it has to satisfy (to increase the adjusted $R^2$) is that the F-value (by the way, how is it simple to calculate it?) of the null hypothesis that the new variable is greater than 1.  Can someone give me a hint where the links between the adjusted $R^2$ and the F-Stat of that test are?   And however, who wants to include a new variable in a multiple OLS regression model anyway, if the beta was tested to be 0? Therefore adjusted $R^2$ always changes.
108590
Is there a way for standard deviation to ignore negative numbers? In other words, how can you make it so that 2 or 3 deviations do not extend into negative territory?   When looking at a set of only positive numbers, within one or two standard deviations, I end up with negative numbers. It's essentially impossible for a data point to be negative.  Am I doing something wrong?   Also, when using deviations from the mean, do you add or subtract the full deviation from the mean or divide by two and plus and minus that to get one deviation from mean   I'm sure this embarrassingly simple for this group but this is not my strong point!   Please let me know if I can clarify.  My recent experience with statistics is relatively limited.  Thanks!
42979
How would you explain the concept of mean, median, and mode of a list of numbers and why they are important to somebody with only basic arithmetic skills? Let's not mention skewness, CLT, central tendency, their statistical properties, etc.  I have explained to somebody that mean is just a quick and dirty way to "summarize" a list of numbers. But looking back, this is hardly illuminating.  Any thoughts or real world examples?
69726
I've estimated an ARMA-GARCH model. All the parameters except the intercept ($\omega$) in the GARCH part are significant. So I tried to estimate the model again imposing that $\omega = 0$. I compared the information criteria of the two models and found out that keeping omega in the model gives the model a better fit.  Questions:   Does it make sense?  Can I keep $\omega$ in the model, although it appears non significant at 5% level, just because of better information criteria? Is it rigorous from a statistical point of view?
33462
I was skimming through  by Cosma Shalizi (in particular, section 2.1.1 of the ), and was reminded that you can get very low $R^2$ even when you have a completely linear model.  To paraphrase Shalizi's example: suppose you have a model $Y = aX + \epsilon$, where $a$ is known. Then $\newcommand{\Var}{\mathrm{Var}}\Var[Y] = a^2 \Var[x] + \Var[\epsilon]$ and the amount of explained variance is $a^2 \Var[X]$, so $R^2 = \frac{a^2 \Var[x]}{a^2 \Var[X] + \Var[\epsilon]}$. This goes to 0 as $\Var[X] \rightarrow 0$ and to 1 as $\Var[X] \rightarrow \infty$.  Conversely, you can get high $R^2$ even when your model is noticeably non-linear. (Anyone have a good example offhand?)  So when is $R^2$ a useful statistic, and when should it be ignored?
40268
I am currently working to build a model using a multiple linear regression. After fiddling around with my model, I am unsure how to best determine which variables to keep and which to remove.  My model started with 10 predictors for the DV. When using all 10 predictors, four were considered significant. If I remove only some of the obviously-incorrect predictors, some of my predictors that were not initially significant become significant. Which leads me to my question: How does one go about determining which predictors to include in their model? It seemed to me you should run the model once with all predictors, remove those that are not significant, and then rerun. But if removing only some of those predictors makes others significant, I am left wondering if I am taking the wrong approach to all this.  I believe that  thread is similar to my question, but I am unsure I am interpreting the discussion correctly. Perhaps this is more of an experimental design topic, but maybe someone has some experience they can share.
69363
Provided a sample size "N" that I plan on using to forecast data. What are some of the ways to subdivide the data so that I use some of it to establish a model, and the remainder data to validate the model?  I know there is no black and white answer to this, but it would be interesting to know  some "rules of thumb" or usually used ratios. I know back at university, one of our professors used to say model on 60% and validate on 40%.
26246
I'm having difficulty wrapping my head around scale parameters.  How exactly do they work?  Why are they sometimes ignored?  (in other words, when is it important to preserve them in a calculation?)   Thank you for any simple, lay explanations.     UPDATE: What motivated the question was the Gamma distribution; in reading of it with relation to Poisson priors, I saw it described as simply $\propto  x^{\alpha-1}  e ^{-x /\beta }$    However, the question is more of a general nature.  While I get the concept of proportionality, I do not yet fully grasp when it is important for items to be equal versus simply proportional; or rather why so often in the literature I see items referred to as proportional with the scaling factor dropped.
14223
I have analysed my data as they are. Now I want to look at my analyses after taking the log of all variables. Many variables contain many zeros. Therefore I add a small quantity to avoid taking the log of zero.   So far I've added 10^-10, without any rationale really, just because I felt like adding a very small quantity would be advisable to minimize the effect of my arbitrarily chosen quantity. But some variables contain mostly zeros, and therefore when logged mostly -23.02. The range of the ranges of my variables is 1.33-8819.21, and the frequency of zeroes also varies dramatically. Therefore my personal choice of "small quantity" affects the variables very differently. It is clear now that 10^-10 is a completely unacceptable choice, as most of the variance in all the variables then comes from this arbitrary "small quantity".  I wonder what would be a more correct way of doing this.  Maybe it's better to derive the quantity from each variables individual distribution? Are there any guidelines about how big this "small quantity" should be?  My analyses are mostly simple cox models with each variable and age/sex as IVs. The variables are the concentrations of various blood lipids, with often considerable coefficients of variation.  Edit: Adding the smallest non-zero value of the variable seems practical for my data. But maybe there is a general solution?  Edit 2: As the zeros merely indicate concentrations below the detection limit, maybe setting them to (detection limit)/2 would be appropriate?