From my understanding the correlation coefficient is r (the correlation between two variables) and ranges from -1,1 to tell the strength of a linear relationship. and is estimated as: $$r_{xy}=\frac{COV_{xy}}{\sigma_x\sigma_y}$$  The regression coefficient is simply the beta in a regression analysis. It tells the strength of the relationship. and is calculated through finding the slope of the line.   So...how is the covariance related to the regression coefficient?
85929
If I want to investigate how two continuous variables are linked, what is the difference between calculating the correlation coefficient (Pearson's $r$) versus calculating the (simple linear) regression coefficient?  I see people who, if the regression coefficient is significantly different from zero, talk about the two variables as if they are correlated, which is confusing as it suggests that the two coefficients (correlation, regression) are the same thing.  Having said that, isn't $r$ a measure of the (regression line) slope anyway? I'm confused!!
37282
This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this?   More formally, suppose I ran the following two regressions:  $$ y_1 = X_1\beta_1 + \epsilon_1 $$ and  $$ y_2 = X_2\beta_2 + \epsilon_2 $$ where $X_i$ refers to the design matrix of regression $i$, and $\beta_i$ to the vector of coefficients in regression $i$. Note that $X_1$ and $X_2$ are potentially very different, with different dimensions etc. I am interested in for instance whether or not $\hat\beta_{11} \neq \hat\beta_{21}$.   If these came from the same regression, this would be trivial. But since they come from different ones, I am not quite sure how to do it. Does anyone have an idea or can give me some pointers?   My problem in detail: My first intuition was to look at the confidence intervals, and if they overlap, then I would say they are essentially the same. This procedure does not come with the correct size of the test, though (i.e. each individual confidence interval has $\alpha=0.05$, say, but looking at them jointly will not have the same probability). My "second" intuition was to conduct a normal t-test. That is, take   $$ \frac{\beta_{11}-\beta_{21}}{sd(\beta_{11})} $$  where $\beta_{21}$ is taken as the value of my null hypothesis. This does not take into account the estimation uncertainty of $\beta_{21}$, though, and the answer may depend on the order of the regressions (which one I call 1 and 2).   My third idea was to do it as in a standard test for equality of two coefficients from the same regression, that is take  $$ \frac{\beta_{11}-\beta_{21}}{sd(\beta_{11}-\beta_{21})} $$   The complication arises due to the fact that both come from different regressions. Note that   $$ Var(\beta_{11}-\beta_{21}) = Var(\beta_{11}) + Var(\beta_{21}) -2 Cov(\beta_{11},\beta_{21}) $$ but since they are from different regressions, how would I get $Cov(\beta_{11},\beta_{21})$?  This led me to ask this question here. This must be a standard procedure / standard test, but I cound not find anything that was sufficiently similar to this problem. So, if anyone can point me to the correct procedure, I would be very grateful!
110290
I have a basic question.  Say I have two random variables, $X$ and $Y$.  I can standardize them by subtracting the mean and dividing by the standard deviation, i.e., $X_{standardized} = \frac{(X - E(X))}{(SD(X))}$.  Is the correlation of $X$ and $Y$, $Cor(X, Y)$, the same as the covariance of the standardized versions of $X$ and $Y$?  That is, is $Cor(X, Y) = Cov(X_{standardized}, Y_{standardized})$?
35136
I would have expected the correlation coefficient to be the same as a regression slope (beta), however having just compared the two, they are different. How do they differ - what different information do they give?
51034
I made $n$ measurements of two variables $x$ and $y$. They both have known uncertainties $\sigma_x$ and $\sigma_y$ associated with them. I want to find the relation between $x$ and $y$. How can I do it?   EDIT: each $x_i$ has a different $\sigma_{x,i}$ associated with it, and the same with the $y_i$.    Reproducible R example:  ## pick some real x and y values  true_x &lt;- 1:100 true_y &lt;- 2*true_x+1  ## pick the uncertainty on them sigma_x &lt;- runif(length(true_x), 1, 10) # 10 sigma_y &lt;- runif(length(true_y), 1, 15) # 15  ## perturb both x and y with noise  noisy_x &lt;- rnorm(length(true_x), true_x, sigma_x) noisy_y &lt;- rnorm(length(true_y), true_y, sigma_y)  ## make a plot  plot(NA, xlab="x", ylab="y",     xlim=range(noisy_x-sigma_x, noisy_x+sigma_x),      ylim=range(noisy_y-sigma_y, noisy_y+sigma_y)) arrows(noisy_x, noisy_y-sigma_y,         noisy_x, noisy_y+sigma_y,         length=0, angle=90, code=3, col="darkgray") arrows(noisy_x-sigma_x, noisy_y,        noisy_x+sigma_x, noisy_y,        length=0, angle=90, code=3, col="darkgray") points(noisy_y ~ noisy_x)  ## fit a line  mdl &lt;- lm(noisy_y ~ noisy_x) abline(mdl)  ## show confidence interval around line  newXs &lt;- seq(-100, 200, 1) prd &lt;- predict(mdl, newdata=data.frame(noisy_x=newXs),      interval=c('confidence'), level=0.99, type='response') lines(newXs, prd[,2], col='black', lty=3) lines(newXs, prd[,3], col='black', lty=3)     The problem with this example is that I think it assumes that there are no uncertainties in $x$. How can I fix this?
1643
The Pearson correlation coefficient of x and y is the same, whether you compute pearson(x, y) or pearson(y, x). This suggests that doing a linear regression of y given x or x given y should be the same, but I don't think that's the case.   Can someone shed light on when the relationship is not symmetric, and how that relates to the Pearson correlation coefficient (which I always think of as summarizing the best fit line)?
14203
Let's say I have two 1-dimensional arrays, $a_1$ and $a_2$. Each contains 100 data points. $a_1$ is the actual data, and $a_2$ is the model prediction. In this case, the $R^2$ value would be: $$ R^2 = 1 - \frac{SS_{res}}{SS_{tot}} \quad\quad\quad\quad\quad\ \ \quad\quad(1).  $$ In the meantime, this would be equal to the square value of the correlation coefficient,  $$ R^2 = (\text{Correlation Coefficient})^2 \quad (2). $$ Now if I swap the two: $a_2$ is the actual data, and $a_1$ is the model prediction. From equation $(2)$, because correlation coefficient does not care which comes first, the $R^2$ value would be the same. However, from equation $(1)$, $SS_{tot}=\sum_i(y_i - \bar y )^2$, the $R^2$ value will change, because the $SS_{tot}$ has changed if we switch $y$ from $a_1$ to $a_2$; in the meantime, $SS_{res}=\sum_i(f_i-\bar y)^2$ does not change.   My question is: How can these contradict each other?  Edit:    I was wondering that, will the relationship in Eq. (2) still stand, if it is not a simple linear regression, i.e., the relationship between IV and DV is not linear (could be exponential / log)? Will this relationship still stand, if the sum of the prediction errors does not equal zero?
82187
I am trying to get the correlation between 2 variables, given a set of data. Once in a while, in the given data set,When one of the variables has a constant value, since the standard deviation of that variable is zero, I get an NA value for the correlation. (In R). I would like to assign a value for the correlation in these scenarios explicitly or try to get some value through alternate means, so that I am able to compare this point with other times I compute the correlation. How do I go about it? (1) Should I add some noise to that variable and compute the correlation again. Would that be a meaningful thing to do?
87513
The application here is interpreting the correlation coefficient $r$ in terms of $X$'s ability to predict $Y$ for extreme values of $X$. For example, if $r = .8$, then what is $p(Y &lt; 0 | X &lt; -2)$?
105550
Recently I came across the following result in a paper, which I can't seem to find right.  Anyway the authors had cited  by Johnson &amp; Kotz, page 20 as the source. Since I do not have access to this book I am unable to check on the derivation of this result. I was hoping someone could shed some light on how to prove this.  Let $X$ and $Y$ denote a pair of bivariate log normal random variables with their means and variances given by the ordered pairs $(\zeta_{X}, \eta_{X})$ and $(\zeta_{Y}, \eta_{Y})$ and with correlation coefficient $\rho_{(X, Y)}$, derived from the bivariate normal with marginal distributions having parameters $(\xi_{X}, \nu_{X}^2)$ and $(\xi_{Y}, \nu_{Y}^2)$ and correlation coefficient $\rho_{N}$.  The expression for $\rho_{(X, Y)}$ is given as \begin{equation} \rho_{(X, Y)} = \dfrac{\mathrm e^{(\rho_{N} \nu_{X} \nu_{Y})} - 1}{\sqrt {(\mathrm e^{\nu_{X}^2} - 1)(\mathrm e^{\nu_{Y}^2} - 1)}} \end{equation}  The below is a manipulation I came up with, any error in that would be completely mine.  Upon rearranging we get \begin{equation}\begin{aligned} \rho_N &amp;= \dfrac{\ln (1 + \rho_{(X, Y)} \sqrt {(\mathrm e^{\nu_{X}^2} - 1)(\mathrm e^{\nu_{Y}^2} - 1)})}{\nu_{X}\nu_{Y}}\\ &amp;= \dfrac{\ln (1 + \rho_{(X, Y)} \frac{\eta_{X}\eta_{Y}}{\zeta{X}\zeta{Y}})}{\sqrt {\ln ((\frac{\eta_{X}}{\zeta_{X}})^2 + 1)\ln ((\frac{\eta_{Y}}{\zeta_{Y}})^2 + 1)}}. \end{aligned} \end{equation}  I am very interested in learning if this result would hold true between $X$ and $Y$'s correlations if we had a quad-variate situation as in there were 2 more random variables which are also correlated with $X$ and $Y$ would this result still be true ? I suspect it should be, but I am newbie and due to lack of a credible source I was wondering if some one would be kind to confirm it.
8470
If in a linear regression I have categorical variable... how do I know the stastical signifance of the categorical variable?  Let's say the factor $X_1$ has 10 levels... there will be 10 different resultant t-values, under the umbrella of one factor variable $X_1$...  It seems to me that the statistical signifiance is tested for each level of the factor variable? No?  @Macro: Following your suggestion, I've built the following example:  It seems that x3 is useful and must be included in the model, from the below model comparison.  But actually that's wrong...  n=100     x1=1:n x2=(1:n)^2  x3=rnorm(n) ee=rnorm(n) y=3*x1-2*x2+x3+3+ee lm1=lm(y~x1+x2+x3) summary(lm1)  lm2=lm(y~x1+x2)  summary(lm2)  anova(lm1, lm2)  &gt; anova(lm1, lm2) Analysis of Variance Table  Model 1: y ~ x1 + x2 + x3 Model 2: y ~ x1 + x2   Res.Df     RSS Df Sum of Sq      F    Pr(&gt;F)     1     96  82.782                                   2     97 146.773 -1    -63.99 74.207 1.401e-13 *** --- Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
60439
I try to understand and visualize myself covariance matrix. Supposing I have a matrix A = [ 2 3 4; 5 5 6 ], how do I calculate its covariance matrix, and what is its practical meaning? (All I was able to understand by now is that on the diagonal of the covariance matrix, variances for particular variables are placed, and on the upper and lower fields correlations between those variables.)
123696
I know correlation does not imply causation but instead the strength and direction of the relationship. Does simple linear regression imply causation? Or is an inferential (t-test, etc.) statistical test required for that?
90978
I recently learned that two independent random variables $X$ and $Y$ must have a covariance of $0$. That means that the correlation between them is also $0$. However, apparently, the converse is not true. 2 random variables $X$ and $Y$ can have a correlation of $0$, yet still be dependent. I don't understand why this is. Doesn't a correlation of $0$ imply that the random variables do not affect each other?
44585
Is it possible to obtain a positive correlation between a regressor and a response (+0,43) and, after that, obtain a negative coefficient in the fitted regression model for this regressor?  I'm not talking about changes in the sign of the regressor among some models. The coefficient sign always remains.  Could  the remaining variables of the fitted model influence the changing of the sign?
39906
What is covariance in plain language and how is it linked to the terms dependence, correlation and variance-covariance structure with respect to repeated-measures designs?
41336
I know that for the $2$-dimensional case: given a correlation $\rho$ you can generate the first and second values, $ X_1 $ and $X_2$, from the standard normal distribution. Then from there make $X_3$ a linear combination of the two $X_3 = \rho X_1 + \sqrt{1-\rho^2}\,X_2$ then take  $$ Y_1 = \mu_1  + \sigma_1 X_1, \quad Y_2 = \mu_2 + \sigma_2 X_3$$  So that now $Y_1$ and $Y_2$ have correlation $\rho$.  How would this be scaled to $n$ variables? With the condition that the end variables satisfy a given correlation matrix? I'm guessing at least n variables will need to be generated then a reassignment through a linear combination of them all will be required... but I'm not sure how to approach it.