I recently ran two tests in R - one using glm() and one using lm() with the goal being to test the relationship between a binary response and binary predictor.  I ran glm() first and got an estimate of -0.68 for the predictor coefficient which I thought was pretty good.  P&lt;.05 and AIC of 653.    When I ran lm() however I got an estimate of -.14, a multiple r-squared of .008, P&lt;.05.    My understanding is that linear regression is usually a poor choice for a categorical response compared with logistic regression, but when is this not the case? I noticed in this post  that the author states there's middle ground where it does make sense to use linear regression.  Are there any common rules (or rules of thumb you personally use) that determine when to try out linear regression on a categorical response?  Do any of these differ from the author's cases?
25174
What is the difference between linear regression and logistic regression?  When would you use each?
62050
I have data from a survey experiment in which respondents were randomly assigned to one of four groups:  &gt; summary(df$Group)        Control     Treatment1     Treatment2     Treatment3              59             63             62             66    While the three treatment groups do vary slightly in the stimulus applied, the main distinction that I care about is between the control and treatment groups. So I defined a dummy variable Control:  &gt; summary(df$Control)      TRUE FALSE         59   191    In the survey, respondents were asked (among other things) to choose which of two things they preferred:   &gt; summary(df$Prefer)       A   B  NA's      152  93   5    Then, after receiving some stimulus as determined by their treatment group (and none if they were in the control group), respondents were asked to choose between the same two things:  &gt; summary(df$Choice)   A    B  149  101    I want to know if the being in one of the three treatment groups had an effect on the choice that respondents made in this last question. My hypothesis is that respondents who received a treatment are more likely to choose A than B.       Given that I am working with categorical data, I have decided to use a logit regression (feel free to chime in if you think that's incorrect). Since respondents were randomly assigned, I am under the impression that I shouldn't necessarily need to control for other variables (e.g. demographics), so I have left those out for this question. My first model was simply the following:  &gt; x0 &lt;- glm(Product ~ Control + Prefer, data=df, family=binomial(link="logit")) &gt; summary(x0)  Call: glm(formula = Choice ~ Control + Prefer, family = binomial(link = "logit"),      data = df)  Deviance Residuals:      Min       1Q   Median       3Q      Max   -1.8366  -0.5850  -0.5850   0.7663   1.9235    Coefficients:                     Estimate Std. Error z value Pr(&gt;|z|)     (Intercept)           1.4819     0.3829   3.871 0.000109 *** ControlFALSE         -0.4068     0.3760  -1.082 0.279224     PreferA              -2.7538     0.3269  -8.424  &lt; 2e-16 *** --- Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1  (Dispersion parameter for binomial family taken to be 1)      Null deviance: 328.95  on 244  degrees of freedom Residual deviance: 239.69  on 242  degrees of freedom   (5 observations deleted due to missingness) AIC: 245.69  Number of Fisher Scoring iterations: 4   I am under the impression that the intercept being statistically significant is not something that holds interpretable meaning. I thought perhaps that I should include an interaction term as follows:  &gt; x1 &lt;- glm(Choice ~ Control + Prefer + Control:Prefer, data=df, family=binomial(link="logit")) &gt; summary(x1)  Call: glm(formula = Product ~ Control + Prefer + Control:Prefer, family = binomial(link = "logit"),      data = df)  Deviance Residuals:      Min       1Q   Median       3Q      Max   -2.5211  -0.6424  -0.5003   0.8519   2.0688    Coefficients:                                  Estimate Std. Error z value Pr(&gt;|z|)     (Intercept)                         3.135      1.021   3.070  0.00214 **  ControlFALSE                       -2.309      1.054  -2.190  0.02853 *   PreferA                            -5.150      1.152  -4.472 7.75e-06 *** ControlFALSE:PreferA                2.850      1.204   2.367  0.01795 *   --- Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1  (Dispersion parameter for binomial family taken to be 1)      Null deviance: 328.95  on 244  degrees of freedom Residual deviance: 231.27  on 241  degrees of freedom   (5 observations deleted due to missingness) AIC: 239.27  Number of Fisher Scoring iterations: 5   Now the respondents status as in a treatment group has the expected effect. Was this a valid set of steps? How can I interpret the interaction term ControlFALSE:PreferA? Are the other coefficients still the log odds?
29900
Specifically, I want to know if there is a difference between lm(y ~ x1 + x2) and glm(y ~ x1 + x2, family=gaussian). I think that this particular case of glm is equal to lm. Am I wrong?
78937
Being aware of , I am curious about the question how big standardized coefficients can get. I had a discussion with my professor about that issue and she was arguing standardized coefficients (beta) in multiple linear regressions can not become greater than |1|. I have also heard that predictors with standardized coefficients greater than 1 should not be be included/appear in multiple linear regression. When I recently estimated a multiple linear regression in R using lm(), I estimated the standardized coefficients with lm.beta() function from the package 'lm.beta'. In the results I could observe a standardized coefficient greater than one. Right now I am just not sure about what is the truth.  Can standardized coefficients become greater than |1|? If yes, what does that mean and should they be excluded from the model? If yes, why?  I would be very thankful, if somebody could make this issue clear for me.
17052
I have a data that where the outcome is the proportion of a species observed in an area by a machine on 2 separate days. Since the outcome is a proportion and does not include 0 or 1 I used a beta regression to fit the model. Temperature is used as  an independent variable. Here is some toy R code:  set.seed(1234) library(betareg) d &lt;- data.frame(   DAY = c(1,1,1,1,2,2,2,2),   Proportion = c(.4,.1,.25, .25, .5,.3,.1,.1),   MACHINE = c("A","B","C","D","H","G","K","L"),   TEMPERATURE = c(rnorm(8)*100) ) b &lt;- betareg(Proportion ~ TEMPERATURE,   data= d, link = "logit", link.phi = NULL, type = "ML") summary(b) ## Call: ## betareg(formula = Proportion ~ TEMPERATURE, data = d, link = "logit", link.phi = NULL, type = "ML") ##  ## Standardized weighted residuals 2: ##     Min      1Q  Median      3Q     Max  ## -1.2803 -1.2012  0.3034  0.6819  1.6494  ##  ## Coefficients (mean model with logit link): ##               Estimate Std. Error z value Pr(&gt;|z|)     ## (Intercept) -1.0881982  0.2620518  -4.153 3.29e-05 *** ## TEMPERATURE  0.0003469  0.0023677   0.147    0.884     ##  ## Phi coefficients (precision model with identity link): ##       Estimate Std. Error z value Pr(&gt;|z|)   ## (phi)    9.305      4.505   2.066   0.0389 * ## --- ## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1    Above you can see the TEMPERATURE coefficient is .0003469. Exponentiating, exp(.0003469) = 1.000347  Update incorporating replies and comments:  You can see here how increasing temperature by 1 unit from -10 to 10 increase the proportion  nd &lt;- data.frame(TEMPERATURE = seq(-10, 10, by = 1)) nd$Proportion &lt;- predict(b, newdata = nd) nd$proportion_ratio &lt;- nd$Proportion/(1 - nd$Proportion) plot(Proportion ~ TEMPERATURE, data = nd, type = "b")   The interpretation is: A 1-unit change in TEMPERATURE leads to a relative change of 1.000347 ≈0.04%  in the Proportion: $$\frac{\mathrm{E}(\mathtt{Proportion})}{1-\mathrm{E}(\mathtt{Proportion})}$$   The key word there is relative change so when you compare exp(coef(b))[2] to nd$proportion_ratio[2] / nd$proportion_ratio[1]  you'll see they are the same  ## ratio of proportion nd$proportion_ratio[2] / nd$proportion_ratio[1]  exp(coef(b))[2] nd$proportion_ratio[-1] / nd$proportion_ratio[-20]
113200
I am running a mixed model in R. However I am having some difficulty understanding the type of model I should be running for the data that I have. Let's call the dependant variable the number of early button presses in a computerised experiment. An experiment is made up of multiple trials. In each trial a participant has to press a button to react to a target appearing on a screen. However they may press the button too early and this is what is being measured as the outcome variable. So for example, participant A may have in total 3 early button presses in an experiment across trials whereas participant B may have 15. In a straightforward linear regression model using the lm command in R, I would think this outcome is a continuous numerical variable. As well... its a number that participants score on in the experiment. However I am not trying to run a linear regression, I am trying to run a mixed model with random effects. My understanding of a mixed model in R is that the data format that the model takes from should be structured to show every participant by every trial. When the data is structured like this at trial level suddenly I have a lot of 1s and 0s in my outcome column. As of course at a trial level participants may accidently press the button too early scoring a 1, or not and score a 0. Does this sound like something that needs to be considered as categorical. If so would it then be looked at through the glmer function with family set to binomial? Thanks
31320
In R I have a categorical variable that I performed logistic regression on and got the following result:  glm(formula = mortality ~ SMOKE, family = binomial, data = c.data)  Deviance Residuals:      Min       1Q   Median       3Q      Max   -0.2155  -0.2155  -0.2155  -0.1860   2.8515    Coefficients:             Estimate Std. Error z value Pr(&gt;|z|)     (Intercept)  -4.0483     0.3189 -12.694   &lt;2e-16 *** SMOKEN        0.2968     0.3559   0.834    0.404     --- Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1  (Dispersion parameter for binomial family taken to be 1)      Null deviance: 492.45  on 2369  degrees of freedom Residual deviance: 491.72  on 2368  degrees of freedom AIC: 495.72  Number of Fisher Scoring iterations: 6   Is the value for the intercept the same as SMOKEY (has a history of smoking)?
112189
I' m asking this question out of sheer curiosity, my teacher was not able to explain it.   If I'm using logistic regression with categorical variables they are coded like {1,2,3}. I guess it wouldn't change my results if I used {4,5,6}. But what if linearity of coding wasn't kept? (say {4,10,99})? What I deal with is just the way of coding factor variables but is it possible that the statistical inference could be "distorted" this way? Or, worst case scenario, would I be able to draw nonsense conclusions from categorical data just because they are coded in some way?
89163
Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by predict.glm using the argument type="response" into binary responses, i.e. $Y=1$ or $Y=0$. What's the quickest &amp; most canonical way to do this in R?   While, again, I'm aware of predict.glm, I don't know where exactly the cutoff value $P(Y_i=1|\hat X_{i})$ lives -- and I guess this is my main stumbling block here.
43873
I am using a logit model. My dependent variable is binary. However I have an independent variable which is categorical and contains the responses: 1.very good, 2.good, 3.average, 4.poor and 5.very poor. So, it is ordinal ("quantitative categorical"). I am not sure how to handle this in the model. I am using gretl.  [Note from @ttnphns: Although the question says the model is logit (because the dependent is categorical), the crucial issue - ordinal independent variables - is basically alike, be the dependent categorical or quantitative. Therefore the question is equally relevant to, say, linear regression too - as it is to logistic regression or other logit model.]
9017
I'm struggling with the interpretation of a regression model where a categorial variable (5 levels) is dummy coded. Here is the result of my calculation in R:  Call: lm(formula = DV ~ Age + Gender + factor(Categorial) +  Continuous 1 + Continuous 2 + Continuous 3,  data = dat)  Residuals:  Min       1Q   Median       3Q      Max  -1.30058 -0.25326  0.00349  0.28123  1.49877   Coefficients:                       Estimate Std. Error t value Pr(&gt;|t|)    (Intercept)           -0.42367    0.30694  -1.380  0.16842    Age                   -0.05949    0.02026  -2.936  0.00356 ** Gender                -0.01800    0.04828  -0.373  0.70952    factor(Categorial)2   -0.30625    0.12645  -2.422  0.01596 *  factor(Categorial)3   -0.03441    0.07752  -0.444  0.65736    factor(Categorial)4   -0.12603    0.09914  -1.271  0.20453    factor(Categorial)5   -0.08417    0.13269  -0.634  0.52630     Continuous 1           0.12080    0.04346   2.779  0.00575 ** Continuous 2          -0.06592    0.04383  -1.504  0.13354    Continuous 3          -0.06230    0.03475  -1.793  0.07392 .  --- Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1  Residual standard error: 0.4259 on 336 degrees of freedom   (6 observations deleted due to missingness) Multiple R-squared:  0.1315,    Adjusted R-squared:  0.1057  F-statistic: 5.089 on 10 and 336 DF,  p-value: 6.353e-07   Ok. Age, Factor 2 of the categorial variable and the first continuous variable are significant predictors of the dependent variable. so far so good.   What I'm not understanding is:   The reference category of the dummy coded categorial variable is the intercept and the first category of the categorial variable. right? How do I interpret this?  When doing an anova with the categorial variable as a independent variable, this factor is a significant predictor. With the results of the linear model, one could conclude that this is only due to category 2, right? Can I test contrasts with this linear regression model (e.g. Category1 vs. Category2)? Should I include interactions?   I'd be glad for any help :-)
90072
I have a set of data with an ordinal response ranging from 1-5 (worst to best) and a categorical predictor with five unordered levels. The experiment is a language experiment whereby subjects are asked to rate different sentence types. In the literature it seems that people fit lmer() most of the time by scaling the ordinal response per subject (i.e. taking the mean response and standard deviation of that subject and dividing the every single response by it.). On the other hand, it seems a reasonable approach to use ordered linear mixed effects probit models or ordered linear mixed effects logit models as implemented in e.g. MCMCglmm() or clmm().  (1) I find myself asking what the best method would be. When is one method to be prefered over the other in an experiment as I have sketched above?  (2) Why do people prefer fitting an lmer() model with a scaled response? What is the advantage?  (3) If it is advisable to use an ordered probit or logit model, how do I decide between an ordered probit or ordered logit model?  Thanks for any help!
78191
I will give my examples with R calls. First a simple example of a linear regression with a dependent variable 'lifespan', and two continuous explanatory variables.  data.frame(height=runif(4000,160,200))-&gt;human.life human.life$weight=runif(4000,50,120) human.life$lifespan=sample(45:90,4000,replace=TRUE) summary(lm(lifespan~1+height+weight,data=human.life))  Call: lm(formula = lifespan ~ 1 + height + weight, data = human.life)  Residuals: Min       1Q   Median       3Q      Max  -23.0257 -11.9124  -0.0565  11.3755  23.8591   Coefficients:              Estimate Std. Error t value Pr(&gt;|t|)     (Intercept) 63.635709   3.486426  18.252   &lt;2e-16 *** height       0.007485   0.018665   0.401   0.6884     weight       0.024544   0.010428   2.354   0.0186 *   --- Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1   Residual standard error: 13.41 on 3997 degrees of freedom Multiple R-squared: 0.001425,   Adjusted R-squared: 0.0009257  F-statistic: 2.853 on 2 and 3997 DF,  p-value: 0.05781   In order to find the estimate of 'lifespan' when the value of 'weight' is 1, I add (Intercept)+height=63.64319  Now what if I have a similar data frame, but one where one of the explanatory variables is categorical?  data.frame(animal=rep(c("dog","fox","pig","wolf"),1000))-&gt;animal.life animal.life$weight=runif(4000,8,50) animal.life$lifespan=sample(1:10,replace=TRUE) summary(lm(lifespan~1+animal+weight,data=animal.life))  Call: lm(formula = lifespan ~ 1 + animal + weight, data = animal.life)  Residuals: Min      1Q  Median      3Q     Max  -4.7677 -2.7796 -0.1025  3.1972  4.3691   Coefficients:             Estimate Std. Error t value Pr(&gt;|t|)     (Intercept) 5.565556   0.145851  38.159  &lt; 2e-16 *** animalfox   0.806634   0.131198   6.148  8.6e-10 *** animalpig   0.010635   0.131259   0.081   0.9354     animalwolf  0.806650   0.131198   6.148  8.6e-10 *** weight      0.007946   0.003815   2.083   0.0373 *   --- Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1   Residual standard error: 2.933 on 3995 degrees of freedom Multiple R-squared: 0.01933,    Adjusted R-squared: 0.01835  F-statistic: 19.69 on 4 and 3995 DF,  p-value: 4.625e-16   In this case, to find the estimate of 'lifespan' when the value of 'weight' is 1, should I add each of the coefficients for 'animal' to the intercept: (Intercept)+animalfox+animalpig+animalwolf? Or what is the proper way to do this?  Thanks Sverre
9311
I did a toy experiment with linear regression, but getting different results for $R^2$, could any one help me?  library(ggplot2) fit=lm(price~carat+depth+table+x+y+z-1,data=diamonds) summary(fit) sse=crossprod(diamonds$price-fit$fitted.values) sst=crossprod(diamonds$price-mean(diamonds$price)) 1-sse/sst
100591
I read in a machine learning book that parameters of linear regression can be estimated (among other methods) by gradient descent, while parameters of logistic regression are usually estimated by maximum likelihood estimation.  Is it possible to explain to a novice (me) why we need different methods for linear/logistic regression. a.k.a why not MLE for linear regression and why not gradient descent for logistic regression?
47903
I would like to get a p-value and an effect size of an independent categorical variable (with several levels) -- that is "overall" and not for each level separately, as is the normal output from lme4 in R. It is just like the thing people report when running an ANOVA.  How can I get this?
90412
I need to analyse data and am unsure whether or not I should use a multinomial logistic regression analysis or a binary logistic regression analysis. I have just conducted a latent class analysis and am interested in seeing if my variables can predict class membership. There are only two classes and so my response variable only has two categories; however the variables range from having 2 to 3 to 4 and are mostly nominal categorical data.