If we have $Y$ as response variable which has more than 1 component, and we do want to do regression, then is there such a thing as multivariate regression, and if so, how those it differ from univariate regression, in method and result?  I picked up a book on multivariate analysis, and it did not touch upon regression and ANOVA at all, instead focusing on PCA or linear discriminant analysis and stuff like that.   So, that indicates that multivariate regression = just do univariate regression on each component.  But I am not sure.
120459
Does the following qualify as a univariate regression?  $$y=b_0+b_1x+b_2x^2+\epsilon$$  I fully comprehend the implications of adding regressors and need no background information - a "yes" or "no" will do :-)
47522
In a univariate regression, $Y=a+bX+e$, the solution for slope b is given by $COV(X,Y)/VAR(X)$.  Is there a similar expression for a bivariate regression $Y=a+bX+cZ+e$. What is the closed form solution for b and c in this case?
28105
Is it possible for more than one "linear regression" line to fit a given set of points? (i.e.... "least squares" is minimised equally in both cases)  I'm assuming a simple one-variable regression in which we have a single "dependent" variable for a given x-value. It may be that regression is forced "through the origin" for the purpose of the Q (if that matters), so, I know it wouldn't be a great fit.  Or is there something intrinsic about a regression line that means it is unique? I am interested in a 'proof' (could be just an intuitive proof) if it exists, that a regression line must be unique.
55591
I'm slightly confused if an independent variable (also called predictor or feature) in a statistical model, for example the $X$ in linear regression $Y=\beta_0+\beta_1 X$, is a random variable ?
40357
I'm doing a multivariate logistic regression where all my independent variables are categorical and binary. I have transformed all my categorical variables into dummies in order to have  reference groups and interpret my odds-ratios. However, I would like to check if there are eventually  multicollinearity issues.   I plan to calculate Spearman correlation between my independent variables and calculate VIF too. Nevertheless, I have several questions: Is Vif appropriate in this case? Do I have to calculate VIF and Spearman correlation on my categorical variables or their associated dummies?   I have tested other ways to detect multicollinearity (I check if the coefficients vary a lot if I increase my sample size or drop or add variables) and I'm pretty sure there is not a collinearity issue but I would like to have a "quantitative" proof.
83784
I have two series of measurements values, first series is X and second is Y. I need to model Y as a function of X, where I know the method that was used to measure X is two times better then the method used for measure Y in terms of error variance.  I red about regression with error term in the independent variable, but I'm not sure what to do when there is also error term in the dependent variable and they are different.  Which technique/tools/constraints should I use for this sort of case?
24294
I've already read all the pages in this site trying to find the answer to my problem but no one seems to be the right one form me...  First I explain you the kind of data I'm working with...  Let's say that I have an array vector with several names of city, one for each of 300 users. I also have another array vector with scores response to a survey of each user or a continuous value for each user.  I would like to know if exist a correlation coefficient that compute the correlation between these two variables so, between a nominal and a numeric/continuous or ordinal variables.  I've searched on the Internet and in some pages they suggest to use the contingency coefficient or Cramer's V or Lambda coefficient or Eta . For each of this measure the just say that they could be applied for such data in which we have a nominal variable and interval or numerical variable. The thing is that searching and searching, trying to understand every one of them, sometime is written or watching the examples that they are reasonable to use them if you have dichotomous nominal variable, except for Cramer's V, other time is not written any requirement for the type of data. A lot of other pages say that is right to apply regression instead, that is right, but I would just simply like to know if there is a coefficient like pearson/spearman for this kind of data.  I also think that is no so properly to use Spearman Correlation coeff since the cities are not sortable.  I have also built the function of Cramer'sV and Eta by myself (I'm working with Matlab) but for Eta they don't talk about any p-value to see if the coefficient is statistically significant...  In the matlabWorks site there is also a nice toolbox that says to compute eta^2 but the kind of input it needs is not understandable.  Is here someone that have done a test like mine? If you need more detail to understand the kind of data I'm using just ask me and I'll try to explain you better.
31278
I have eight independent variables and one dependent.  I have run a correlation matrix, and 5 of them have a low correlation with the DV.  I have then run a stepwise multiple regression to see whether any/all of the IVs can predict the DV.  The regression showed that only two IVs can predict the DV (can only account for about 20% of the variance though), and SPSS removed the rest from the model.  My supervisor reckons that I have not run the regression correctly, as due to the strength of the correlations, I should have found more predictors in the regression model. But the correlations were tiny, so my question is: if IVs and the DV hardly correlate, can IVs still be good predictors of the DV?
116621
I have 6 variables ($x_{1}...x_{6}$) that I am using to predict $y$. When performing my data analysis, I first tried a multiple linear regression. From this, only two variables were significant. However, when I ran a linear regression comparing each variable individually to $y$, all but one were significant ($p$ anywhere from less than 0.01 to less than 0.001). It was suggested that this was due to multicollinearity.  My initial research on this suggests checking for multicollinearity by using . I downloaded the appropriate package from R, and ended up with the resulting VIFs: 3.35, 3.59, 2.64, 2.24, and 5.56. According to various sources online, the point you should be worried about multicollinearity with your VIFs is either at 4 or 5.   I am now stumped about what this means for my data. Do I or do I not have a multicollinearity problem? If I do, then how should I proceed? (I cannot collect more data, and the variables are parts of a model that aren't obviously related) If I don't have this problem, then what should I be taking from my data, particularly the fact that these variables are highly significant individually, but not significant at all when combined.  Edit: Some questions have been asked regarding the dataset, and so I would like to expand...  In this particular case, we're looking to understand how specific social cues (gesture, gaze, etc) affect the likelihood of someone producing some other cue. We would like our model to include all significant attributes, so I am uncomfortable removing some that seem redundant.  There are not any hypotheses with this right now. Rather, the problem is unstudied, and we are looking to gain a better understanding of what attributes are important. As far as I can tell, these attributes should be relatively independent of one another (you can't just say gaze and gestures are the same, or one the subset of another). It would be nice to be able to report p values for everything, since we would like other researchers to understand what has been looked at.  Edit 2: Since it came up somewhere below, my $n$ is 24.
74309
I am trying see whether to go for ridge regression, LASSO, principal component regression (PCR), or Partial Least Squares (PLS) in a situation where there are large number of variables / features ($p$) and smaller number of samples ($n&lt;p$), and my objective is prediction.   This my understanding:     shrinks the regression coefficients, but uses all coefficients without making them $0$.  also shrinks the coefficients, but also makes them $0$, meaning that it can do variable selection too.  Principal component regression truncates the components so that $p$ becomes less than $n$; it will discard $p-n$ components. Partial least square also constructs a set of linear combinations of the inputs for regression, but unlike PCR it uses $y$ (in addition to $X$) for dimensionality reduction. The main practical difference between PCR and PLS regression is that PCR often needs more components than PLS to achieve the same prediction error ().   Consider the following dummy data (the actual data I am trying to work with is similar):    #random population of 200 subjects with 1000 variables   M &lt;- matrix(rep(0,200*100),200,1000) for (i in 1:200) { set.seed(i)   M[i,] &lt;- ifelse(runif(1000)&lt;0.5,-1,1) } rownames(M) &lt;- 1:200  #random yvars  set.seed(1234) u &lt;- rnorm(1000) g &lt;- as.vector(crossprod(t(M),u)) h2 &lt;- 0.5  set.seed(234) y &lt;- g + rnorm(200,mean=0,sd=sqrt((1-h2)/h2*var(g)))  myd &lt;- data.frame(y=y, M)   Implementation of four methods:      require(glmnet)   # LASSO   fit1=glmnet(M,y, family="gaussian", alpha=1)   # Ridge     fit1=glmnet(M,y, family="gaussian", alpha=0)   # PLS  require(pls)  fit3 &lt;- plsr(y ~ ., ncomp = 198, data = myd, validation = "LOO")  # taking 198 components and using leave-one-out cross validation   summary(fit3)  plot(RMSEP(fit3), legendpos = "topright")   # PCR   fit4 &lt;- pcr(y ~ ., ncomp = 198, data = myd, validation = "LOO")   The best description of the data is:    $p &gt; n$, most of times  $p&gt;10n$; Variables ($X$ and $Y$) are correlated with each other with different degrees.   My question is which strategy may be best for this situation? Why?
11648
I have a numerical dependent variable, and many independent variables. Most of my independent variables are dummy variables, but I have some categorical and numerical variables, too. I tried forward and backward model selection in R, but R returns my model empty! When I run separate simple regressions, there seems to be a significant relationship between my independent and dependent variables!  My question is: Am I going to have biased results if I run separate simple regressions with each variable, and then run a multiple regression with all the significant variables?
69179
I want to run a regression with 4 to 5 explanatory variables, but I have only 15 observations. Not being able to assume these variables are normally distributed, is there a non-parametric or any other valid regression method?
33305
I have one-sided hypotheses that I want to test with multiple regressions. Does anyone know how to perform a one-sided hypothesis testing? I heard of different positions regarding this matter ("not possible to test one-tailed with regression" and "for one-sided hypotheses just cut the p-value in half"). Or maybe I need another command in with my statistics program (I use R)!?  I didn't think this to be a difficult question but couldn't find a consistent and satsfying solution.  Thank you in advance for any helping comments.
6317
Suppose in a regression analysis in R, I have a factor type independent variable with 3 levels in my train dataset. But in the test data set that same factor variable has 5 levels. Therefore I can not predict the response values for test dataset. What should be done in this case?
112001
I have a dataset with 10 variables ...is it feasible to transform each variable differently while doing multiple regression...  for example new_V1 = log(v1)  New_V2= V2^2  New_V3= 1/V3  Likewise differently for different variables and then applying multiple regression?
76703
What is the distribution of the coefficient of determination, or R squared, $R^2$, in linear univariate multiple regression under the null hypothesis $H_0:\beta=0$?  How does it depend on the number of predictors $k$ and number of samples $n&gt;k$? Is there a closed-form expression for the mode of this distribution?  In particular, I have a feeling that for simple regression (with one predictor $x$) this distribution has mode at zero, but for multiple regression the mode is at a non-zero positive value. If this is indeed true, is there an intuitive explanation of this "phase transition"?    Update  As @Alecos showed below, the distribution indeed peaks at zero when $k=2$ and $k=3$ and not at zero when $k&gt;3$. I feel that there should be a geometrical view on this phase transition. Consider geometrical view of OLS: $\mathbf y$ is a vector in $\mathbb R^n$, $\mathbf X$ defines a $k$-dimensional subspace there. OLS amounts to projecting $\mathbf y$ onto this subspace, and $R^2$ is squared cosine of the angle between $\mathbf y$ and its projection $\hat{\mathbf y}$.  Now, from @Alecos's answer it follows that if all vectors are random, then the probability distribution of this angle will peak at $90^\circ$ for $k=2$ and $k=3$, but will have a mode at some other value $&lt;90^\circ$ for $k&gt;3$. Why?!    Update 2: I am accepting @Alecos'es answer, but still have a feeling that I am missing some important insight here. If anybody ever suggests any other (geometrical or not) view on this phenomenon that would make it "obvious", I will be happy to offer a bounty.
72888
Is a "split plot" ANOVA with two factors identical to two-way ANOVA with repeated measures in one factor? if not, what is the distinction?