In a follow up to one of the business problem i have discussed here, for one of the logistic regression model, i have 19 predictor variables of     which around 8 are categorical with multiple categories.     Instead of including all in the model where one could correlate to another, i am thinking of using a chi square test of independence at a given p value. With this i would come to realization if there is any correlation among the     categorical predictors.    If so i would include the correlated variables one by one in the model and would used the significant one. I have researched on this on SE where they have suggested to use chi- square but unsure if my approach to drop these correlated var is justified.
6924
I have a dataframe with many observations and many variables. Some of them are categorical (unordered) and the others are numerical.  I'm looking for associations between these variables. I've been able to compute correlation for numerical variables (Spearman's correlation) but :   I don't know how to measure correlation between unordered categorical variables. I don't know how to measure correlation between unordered categorical variables and numerical variables.   Does anyone know how this could be done? If so, are there R functions implementing these methods?
19575
X and Y are not correlated (-.01); however, when I place X in a multiple regression predicting Y, alongside three (A, B, C) other (related) variables, X and two other variables (A, B) are significant predictors of Y. Note that the two other (A, B) variables are significantly correlated with Y outside of the regression.   How should I interpret these findings? X predicts unique variance in Y, but since these are not correlated (Pearson), it is somehow difficult to interpret.  I know of opposite cases (i.e., two variables are correlated but regression is not significant) and those are relatively simpler to understand from a theoretical and statistical perspective. Note that some of the predictors are quite correlated (e.g., .70) but not to the extent that I would expect substantial multicollinearity. Maybe I am mistaken, though.    NOTE: I asked this question previously and it was closed. The rational was that this question is redundant with the question "". Perhaps I do not understand the other question, but I believe these are entirely separate questions, both mathematically and theoretically. My question is entirely independent from if "a regression is significant". Furthermore, several predictors are significant, while the other question entails variables not being significant, so I don't see the overlap. If these questions are redundant for reasons I do not understand, please insert a comment prior to closing this question. Also, I was hoping to message the moderator who closed the other question to avoid identical questions, but I couldn't find an option to do so.
38554
I am wanting to find the correlation between two variables: a categorical vegetation variable (with categories 1-4)  and a continuous variable of erosion (with values anywhere between 0.2 and 9). I think I would be required to use a non parametric test to find the correlation, but bit of a newby to stats so could anyone suggest the best test to use? I also wanted to know how sample size can affect these tests, ie with a sample size of 100 or 1000?
20272
I have trouble interpreting logistic regressions. And reading through several materials just confused me even more. Perhaps an example along with my course of thinking might clarify - what exactly I'm getting wrong.  Say, we have a data set for any (generic) product sales, lets make it cars for sake of example (data is completely made up, so results might as well lack any logic). We have a sample of people which are car owners where some own particular car and the rest have any sort of other car. Goal is to estimate probabilities of selling this particular car to some potential client with given set of parameters or put in other words - try to make artificial segmentation of market by ranking these segments by sales probability. E.g. We'll define 2 parameters - age of client and age of car. And lets make it all categorical - we'll define 3 client age groups and 3 car age groups.   car: 1 0 client_age: [under 30] [31 - 40] [over 41] car_age: [under 2] [2 - 5] [over 5]   So, regression would look like car ~ client_age + car_age  And the output:  Intercept: -0.8 client_age[31 - 40]: 0.5 client_age[over 41]: -0.6 car_age[2 - 5]: 0.2 car_age[over 5]: -0.9   With all coefficients being significant at 95%.  So.. now the course of thinking. The general probability of buying a particular car would simply be the ratio of car == 1 to the size of sample. Lets make it 3%. And as I understand it - logistic regression for categorical variables shows improvement in odds (exponent of coefficient) over the baseline. e.g. client_age[31 - 40] would have exp(0.5) or 1.648 or 65% more likely to own this particular car over client_age[21 - 30]. With everything else held constant. Similarly client_age[over 41] would have exp(-0.6) or 0.54881 or 45% less likely to own this car over client_age[21 - 30]. Same applies to car age.  So, my list of questions:    What is the role/interpretation of intercept?  Would it be possible to get all combinations of categorical values and rank them by probability relative to average 3% probability of buying the car? Is there a point in using logistic regression in this particular example? By that I mean - estimating logistic regression only with  categorical variables when I can simply calculate probability for every particular subset of general data set? How would these results compare?
53724
One of the assumption of logistic regression is the linearity in the logit. So once I got my model up and running I test for nonlinearity using Box-Tidwell test. One of my continuous predictors (X) has tested positive for nonlinearity. What am I suppose to do next?   As this is a violation of the assumptions shall I get rid of the predictor (X) or include the nonlinear transformation (X*X). Or transform the variable into a categorical? If you have a reference could you please point me to that too?
109942
Is there i.i.d. assumption on the response variable of logistic regression?  For example, suppose we have $1000$ data points. It seems the response $Y_i$ is coming from a Bernoulli distribution with $p_i=\text{logit}^{-1}(\beta_0+\beta_1 x_i)$. Therefore, we should have $1000$ Bernoulli distributions, with different parameter $p$.  So, they are "independent", but are not "identical".  Am I right?    PS. I learned logistic regression from "machine learning" literature, where we optimize the objective function and check if it is good in testing data, without talking too much about assumptions.   My question started with this post  Where I try to learn more on statistical assumptions.
17858
A few days ago, a psychologist-researcher of mine told me about his method to select variables to linear regression model. I guess it's not good, but I need to ask someone else to make sure. The method is:      Look at correlation matrix between all variables (including Dependent   Variable Y) and choose those predictors Xs, that correlate most with Y.     He didn't mention any criterion.  Q: Was he right?   [I think that this selection method is wrong, because of many things, like it's the theory that says which predictors should be selected, or even  omitted variable bias (OVB).]
48674
Suppose that I have three populations with four, mutually exclusive characteristics.  I take random samples from each population and construct a crosstab or frequency table for the characteristics that I am measuring.  Am I correct in saying that:   If I wanted to test whether there is any relationship between the populations and the characteristics (e.g. whether one population has a higher frequency of one of the characteristics), I should run a chi-squared test and see whether the result is significant. If the chi-squared test is significant, it only shows me that there is some relationship between the populations and characteristics, but not how they are related. Furthermore, not all of the characteristics need to be related to the population.  For example, if the different populations have significantly different distributions of characteristics A and B, but not of C and D, then the chi-squared test may still come back as significant. If I wanted to measure whether or not a specific characteristic is affected by the population, then I can run a test for equal proportions (I have seen this called a z-test, or as prop.test() in R) on just that characteristic.   In other words, is it appropriate to use the prop.test() to more accurately determine the nature of a relationship between two sets of categories when the chi-squared test says that there is a significant relationship?
43698
Is it appropriate to do a logistic regression where both the dependent and independent variables are binary? for example the dependent variable is 0 and 1 and the predictors are contrast coded variables -1 and 1 ?
22442
I have a dataset that consists of only categorical variables and a target variable. I want to predict the (binary) target variable with the categorical variables.  I am trying to do this in Python and sklearn.  The categorical variables have many different values. I was thinking to create dummy variables for each value in all the categorical variables. However, in the end this results in a sparse data set with thousands of variables.  How would you go about to reduce the number of dummy variables? Would you use Chi2 to select useful features?  Or would you not make dummy variables at all?
113798
I know that we can use Spearman rho to measure the correlation between numerical variables. But how to measure correlation between categorical variables?
98298
Coefficients and significance of lasso/ridge  I had 628 predictors after forming dummy of all categorical variables. When I ran lot many iterations traditional logistic regression iteration, I came across 15 variables that was giving me pretty good model with good ROC, recall &amp; precision(for certain cut-off) values on test data and also all variables were significant(at p&lt;=0.05). But since it took lot of time, I tried using lasso that gave me 50 non-zero-coefficient variables after taking best lambda value post running 10 fold cross-validation. But only 5 variables were common between 15 variables of traditional method and 50 of lasso. Moreover, when I tried to calculate its SE and t-stats, I figured out that many variables are insignificant(low t-stats and high p-value). In addition to it, the AUC for ROC was less than traditional method.The ROC drops even more when I used traditional logistic regression on 50 variables that were result of lasso. Can someone help me understand the dynamics of it and how I will be able to justify the coefficients of lasso model as they are penalized(I have normalized all the variables before using lasso)?
1463
Let's say I am building a logistic regression model where the dependent variable is binary and can take the values $0$ or $1$. Let the independent variables be $x_1, x_2, ..., x_m$ - there are $m$ independent variables. Let's say for the $k$th independent variable, the bivariate analysis shows a U-shaped trend - i.e., if I group $x_k$ into $20$ bins each containing roughly equal number of observations and calculate the 'bad rate' for each bin - # observations where y = 0 / total observations in each bin - then I get a U shaped curve.  My questions are:   Can I directly use $x_k$ as input while estimating the beta parameters? Are any statistical assumptions violated which might cause significant error in estimating the parameters? Is it necessary to 'linearize' this variable through a transformation (log, square, product with itself, etc.)?
4148
I have a continuous outcome variable. I understand that if I have a binary predictor, and a continuous predictor, and an interaction, then the model looks like this:  $y_{i} = \beta_{0} + \beta_{1}x_{1} + \beta_{2}x_{2} + \beta_{3}x_{1}x_{2} + \varepsilon_{i}$  However, I'm thinking of making the binary predictor a categorical one instead, with three categories.    What will the model equation look like once I make that change? I understand that I'll need to have two dummy variables, but I can't conceptualize what the interaction will look like under that circumstance.  Does it make any difference whether I use 0,1 to code the binary predictor, or some other values like 1,2? This question also applies to the model with the categorical predictor, since in that case I'll have to decide how to code the dummy variables.  Prior to the interaction term being in my model I was encouraged to center the continuous predictor. Is centering (still) a good idea now that I have this interaction term?
64334
I am performing ordinal regression, I have 5 response categories and several predictors both continuous and categorical. I would like to add a predictor which is categorical but ordered (1, 2, 3, 4). I don't think it would be appropriate to apply the usual dummy coding for unordered categorical predictors, but when I searched for how to code this I did not find much information.  In  "linear coding" or "assuming linearity of the predictor effect" is mentioned, but without further details.  Does it mean I just use my ordered values as they are, i.e. use them as a continuous variable?
111093
I built a logistic model with 7 continuous independent variable and 8 categorical (dummy) variables in R. I tested for the linearity assumption using the Box Tidwell test and all my variables show high significance.  I am wondering now, if the assumption is violated what it would mean for the coefficient estimates and if I still can run the model with caution.   I read about the approach of using regression splines but I am confused how I would determine those and interpret the model then.
99785
I'm fairly new to Machine Learning/Modelling and I'd like some background to this problem. I have a dataset where the number of observations is $n&lt;200$ however the number of variables is $p\sim 8000$. Firstly does it even make sense to consider building a model on a dataset like this or should one consider a variable selection technique to start with such as ridge regression or Lasso? I've read that this situation can lead to over-fitting. Is that the case for all ML techniques or do some techniques handle this better than others? Without too much maths a simple explanation on why the maths start to breakdown for $p&gt;n$ would be appreciated.