I have this categorical variable that represents the country for a person. How can I use this in regression model. If there had been less levels for a category I could have used dummy binary variables to encode them. But in the case of countries, there are more than 200 countries, so it doesn't make sense to use binary variables. What should I do?  Obviously, one of the ways would be to make the variable coarse, take continent instead of country to reduce the levels. But this is not what I want to do?  Lets suppose I have a categorical variable which is the user id. What should I do in this case?
122937
I need to recode the variable school setting (urban, sub-urban and rural settings) into a dummy variable. I know that when creating a dummy variable, there is one category less (so 2 rather than three conditions) and that the urban is the largest group and should be used as baseline (not sure if baseline is the right word, but hopefully you know what I mean). However, I don't know what to do from here.  Is the variable changed into: 1. Two dummy variables: setting 1 with urban/sub-urban and setting 2 with urban/rural? 2. One variable where urban is given the value 1, and urban and sub-urban are given the value 0? 3. Both? 4. Have I completely missed something?  Hope you can help me!!
22442
I have a dataset that consists of only categorical variables and a target variable. I want to predict the (binary) target variable with the categorical variables.  I am trying to do this in Python and sklearn.  The categorical variables have many different values. I was thinking to create dummy variables for each value in all the categorical variables. However, in the end this results in a sparse data set with thousands of variables.  How would you go about to reduce the number of dummy variables? Would you use Chi2 to select useful features?  Or would you not make dummy variables at all?
20454
I'm not sure of the best way to code my categorical predictor variable for use in a hierarchical regression in order to test my specific hypothesis. This categorical variable has 3 levels representing 3 groups. I want to compare group 1 to group 2, group 1 to group 3 and group 2 to group 3. I know that for dummy coding I create k-1 variables, so 2 dummy variables in my case and code these variables with 0s and 1s while choosing one level of the categorical variable to be a reference category.   However, I'm not sure this is the best way of making the comparisons I wish to make as it appears I could only compare each group to the reference category, am I correct? So if group 3 was the reference category I could compare group 1 to group 3 and group 2 to group 3 but I could not compare group 1 to group 2. What alternative method of coding should I use to make these comparisons? My regression model will also contain continuous variables. I'm an undergrad psychology student and statistics are not my strong point simple answers would be best for me. I use SPSS. Thank you!
27529
I want to make a logistic model from my survey data. It is a small survey of four residential colonies in which only 154 respondents were interviewed. My dependent variable is "satisfactory transition to work". I found that, of the 154 respondents, 73 said that they have satisfactorily transitioned to work, while the rest did not. So the dependent variable is binary in nature and I decided to use logistic regression. I have seven independent variables (three continuous and four nominal). One guideline suggest that there should be 10 cases for each predictor / independent variable (Agresti, 2007). Based on this guideline I feel that it is OK to run logistic regression.    Am I right? If not please let me know how to decide the number of independent variables?
17051
I am trying to run a Multiple Regression model but I am not sure what should I do to prepare the data input. In my model the dependent variable is the artificial surfaces in hectares, the independent are the GDP (monetary value) and Population (inhabitants). The histogram shows that the data is not normal distributed.    I would like to know if I need to normalize the distribution of the data in order to get the "bell" curve and/or  if I need to normalize the data (with the max-min or standardize).
108619
It looks like you can use coding for one categorical variable, but I have two categorical and one continuous predictor variable. Can i use multiple regression for this in SPSS and if so how? thanks!
47771
In logistic regression with an intercept term and with at least one dependent variable which is categorical, is there a closed form for the variance of the sum of the intercept and the coefficient of the categorical variable, or do you have to sample from a multivariate distribution with the means and variances of the intercept and the coefficient to get a reliable measure of the variance of this sum?  $P(y) = \frac{exp(y)}{1 + exp(y)}$  $y = \beta_{0} + \beta_{1}x + \epsilon$  where x is categorical.  Would the formula for the variance of a sum (of two random variables) be applicable here?  $Var(\beta_{0} + \beta_{1}) = Var(\beta_{0}) + Var(\beta_{1}) + 2Cov(\beta_{0}, \beta_{1})$  The reason I ask, is that in  I got the impression that no closed form existed for the variance in question, and the advice was to sample from a multivariate distribution with the means and variances of $\beta_{0}$ and $\beta_{1}$  set.seed(1) dependent.var &lt;- sample(c(TRUE, FALSE), 100, replace = TRUE, prob = c(0.3, 0.7)) independent.var &lt;- ifelse(dependent.var, sample(c("Red", "Blue"), replace = TRUE,   size = 10, prob = c(0.8, 0.2)), sample(c("Red", "Blue"), size = 10,   replace = TRUE, prob = c(0.4, 0.6))) table(dependent.var, independent.var) ##               independent.var ## dependent.var Blue Red ##         FALSE   42  26 ##         TRUE     7  25 my.fit &lt;- glm(dependent.var ~ 1 + independent.var, family = binomial(logit)) coef(summary(my.fit))                     Estimate Std. Error   z value     Pr(&gt;|z|) (Intercept)        -1.791759  0.4082482 -4.388897 0.0000113927 independent.varRed  1.752539  0.4951042  3.539737 0.0004005255 &gt; vcov(my.fit)                    (Intercept) independent.varRed (Intercept)          0.1666666         -0.1666666 independent.varRed  -0.1666666          0.2451282   The logit of TRUE for a "Red" case is $ \beta_{0}+\beta_{1} \approx -0.039$. Is the variance for this estimate exactly  vcov(my.fit)[1,1] + vcov(my.fit)[2,2] + 2 * vcov(my.fit)[1,2] [1] 0.07846154 ?   Or is this only an approximation, and a more accurate measure is to be found by sampling, e.g.  library(MASS) var(rowSums(mvrnorm(n = 1E7, mu = coef(my.fit), Sigma = vcov(my.fit)))) [1] 0.07842985 ?   In this simple example, the sampling method does not seem to provide more accurate estimates of the variance (using 1E7 samples).   it is stated that "There is a correspondance between the covariance matrix of the fit parameters and Δχ2 confidence regions only for the case of Gaussian uncertainties on the input measurements.". Is that a reason against relying on the closed form above, or is there perhaps another reason for the advice to sample instead of deriving the variance analytically in cases like this?  EDIT: (In response to the answer given by StasK). The advice I originally got was to simulate from the full model, not from the vcov(), so here is the code to simulate from the full model:  library(arm) sim.i &lt;- sim(my.fit, 100000) logit.for.TRUE.red &lt;- sim.i@coef[,1] + sim.i@coef[,2] var(logit.for.TRUE.red) [1] 0.07781206
85139
I have some data in which there are 5 categorical explanatory variables (concern, breath, weath, sleep, act) and 1 continuous response variable (tto). In addition, every categorical explanatory variable is divided into 5 levels which show how strong a person feels about it. level 1 and level 5 show the perfect and worst states respectively.   I was advised to create a box plot to see the relationship between the explanatory variables and the response variable. The plot is given below. However, I do not know how to read a box plot. Can any one please help me interpret it.
99785
I'm fairly new to Machine Learning/Modelling and I'd like some background to this problem. I have a dataset where the number of observations is $n&lt;200$ however the number of variables is $p\sim 8000$. Firstly does it even make sense to consider building a model on a dataset like this or should one consider a variable selection technique to start with such as ridge regression or Lasso? I've read that this situation can lead to over-fitting. Is that the case for all ML techniques or do some techniques handle this better than others? Without too much maths a simple explanation on why the maths start to breakdown for $p&gt;n$ would be appreciated.
35328
I understand that when label encoding is used ,the numeric number can be interpreted to have an order and a model could assume a linear relationship. However shouldn't this be a problem when there are in-fact many levels in a categorical variable e.g. country.  How about the case of binary variables which for e.g. instead of gender.male ( 1,0) what if I just used Gender ( 1,0) where gender = 0 is female and 1 is male for instance. This shouldn't impact the model as much as a label encoding to a feature with multiple levels? How would this be for a case where I have a feature three levels ( -1,0,1) where -1 means not applicable, 0 means 'No' and 1 means 'Yes', so instead of having 2 columns feature.not_applicable(1,0) and feature.No(1,0) - mathematically how would models be impacted - models here would be GLMS,  boosting models, Random forests etc. Is label encoding recommended when feature has say &lt;=3 to 4 levels and above that is recommended to do one hot encoding or $n-1$ dummy variables?
12268
In the context of feature selection it is common to recode categorical variables with more than 2 categories into dummies. Selection methods such as elastic nets or lasso regression select the best predictors, whereby it is possible that only some dummies of each categorical variable are selected. I am wondering, if there can appear some problems due to this procedure. I found some comments about the topic on  and a , stating that the procedure should be used carefully, but that there are no general problems. However, I was not able to find any detailed literature or any educated guidelines, which could be followed.  Question: Can there appear any problems, if not all dummies of a categorical variable are selected for a model?  For example, I could imagine that the automatic selection relies on the order of the categories and the resulting reference category. Let's say there is a variable with categories A, B, and C. A dummy recoding into dummyB and dummyC would probably result in different variable selections compared to a dummy recoding into dummyA and dummyB.  Any advice or literature is highly appreciated!  UPDATE:  Based on Ben's comment I found some literature about a comparison of the lasso and the group lasso, which addresses my question:      However, based on this literature 2 further questions appeared:  1) It seems like the normal lasso is still used regularly, whereby the group lasso doesn't appear that often in current literature. Is there a specific reason for that?  2) When I have categorical variables with many categories, isn't it a problem, if I select the whole categorical variable? Or in other words, is it sometimes advantageous to use the lasso instead of the group lasso?
40357
I'm doing a multivariate logistic regression where all my independent variables are categorical and binary. I have transformed all my categorical variables into dummies in order to have  reference groups and interpret my odds-ratios. However, I would like to check if there are eventually  multicollinearity issues.   I plan to calculate Spearman correlation between my independent variables and calculate VIF too. Nevertheless, I have several questions: Is Vif appropriate in this case? Do I have to calculate VIF and Spearman correlation on my categorical variables or their associated dummies?   I have tested other ways to detect multicollinearity (I check if the coefficients vary a lot if I increase my sample size or drop or add variables) and I'm pretty sure there is not a collinearity issue but I would like to have a "quantitative" proof.
9969
I am building a regression model and I need to calculate the below to check for correlations   Correlation between 2 Multi level categorical variables Correlation between a Multi level categorical variable and continuous variable  VIF(variance inflation factor) for a Multi level categorical variables   I believe its wrong to use Pearson correlation coefficient for the above scenarios because Pearson only works for 2 continuous variables.   Please answer the below questions   Which correlation coefficient works best for the above cases ?  VIF calculation only works for continuous data so what is the alternative?  What are the assumptions I need to check before I use the correlation coefficient you suggest?  How to implement them in SAS &amp; R?
1109
I have a data having 2 continuous and 4 categorical variables. Each categorical variable has 3 levels. I want to know how to include the variables  in the model. I am using SPSS  Variables:   Sales - Dependent Variable Retail Price Location : TX, CA, PY Display Location in Store : DL1, DL2, DL3 Price Point : 2.5, 4.5, 1.0   I have created 9 dummy variables. 3 for Location, 3 for Display Location in Store and 3 for Price Point.  I am confused on how to use these independent variables. How to add the independent variables into the models. Should I add all the nine dummy variables or two dummy variables of each categorical variables along with Retail Price.  I searched on net but , did not get any solution. I would appreciate any kind of help.  Thanks in advance.
42862
I am running an OLS regression using dummy variables built from categorical variables. Say, race became race1, race2 and race3. I omit race1 in order to escape the dummy variable trap and run OLS and gain some coefficients for race2 and race3.  What would be the coefficient for race1 in the regression equation? Or I just should not include the omitted dummy variable into the final equation to estimate the predicted value?
111728
I have six dependent variables (count data) and several independent variables, I see that in a MMR the script goes like this:  my.model &lt;- lm(cbind(DV1,DV2,DV3,DV4,DV5,DV6) ~ IV1 + IV2 + ... + IVn)   But, since my data are counts, I want to use a generalized linear model and I tried this:  my.model &lt;- glm(cbind(DV1,DV2,DV3,DV4,DV5,DV6) ~ IV1 + IV2 + ... + IVn, family="poisson")   And appears this error message:  Error in glm.fit(x = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  :    (subscript) logical subscript too long`   Can anyone explain me this error message or a way to solve my problem?
20272
I have trouble interpreting logistic regressions. And reading through several materials just confused me even more. Perhaps an example along with my course of thinking might clarify - what exactly I'm getting wrong.  Say, we have a data set for any (generic) product sales, lets make it cars for sake of example (data is completely made up, so results might as well lack any logic). We have a sample of people which are car owners where some own particular car and the rest have any sort of other car. Goal is to estimate probabilities of selling this particular car to some potential client with given set of parameters or put in other words - try to make artificial segmentation of market by ranking these segments by sales probability. E.g. We'll define 2 parameters - age of client and age of car. And lets make it all categorical - we'll define 3 client age groups and 3 car age groups.   car: 1 0 client_age: [under 30] [31 - 40] [over 41] car_age: [under 2] [2 - 5] [over 5]   So, regression would look like car ~ client_age + car_age  And the output:  Intercept: -0.8 client_age[31 - 40]: 0.5 client_age[over 41]: -0.6 car_age[2 - 5]: 0.2 car_age[over 5]: -0.9   With all coefficients being significant at 95%.  So.. now the course of thinking. The general probability of buying a particular car would simply be the ratio of car == 1 to the size of sample. Lets make it 3%. And as I understand it - logistic regression for categorical variables shows improvement in odds (exponent of coefficient) over the baseline. e.g. client_age[31 - 40] would have exp(0.5) or 1.648 or 65% more likely to own this particular car over client_age[21 - 30]. With everything else held constant. Similarly client_age[over 41] would have exp(-0.6) or 0.54881 or 45% less likely to own this car over client_age[21 - 30]. Same applies to car age.  So, my list of questions:    What is the role/interpretation of intercept?  Would it be possible to get all combinations of categorical values and rank them by probability relative to average 3% probability of buying the car? Is there a point in using logistic regression in this particular example? By that I mean - estimating logistic regression only with  categorical variables when I can simply calculate probability for every particular subset of general data set? How would these results compare?