I want to do a regression to figure out the relationship between $y$ and  $x$ given some sample data. My problem is that within the sample data, while $x_i$ is always observed, I do not directly observe $y_i$. Instead, I observe if $y_i&gt;z_i$, where $z_i$ is some observed number.   For example, I will have to train the model $y=\hat{f}(x)$ on the below sample data: $$(x_1=2, y_1&gt;1)$$ $$(x_2=3, y_2&lt;4)$$ $$(x_3=10, y_3&gt;2)$$ ...  Then I can use the trained $\hat{f}(x)$ to make predictions of $y$ on any new $x$, $y^*=\hat{f}(x^*)$  How do you think I can approach this problem?
114466
Is there a branch of statistics that deals with data for which exact values are not known, but for each individual, we know either a maximum or minimum bound to the value?  I suspect that my problem stems largely from the fact that I am struggling to articulate it in statistical terms, but hopefully an example will help to clarify:  Say there are two connected populations $A$ and $B$ such that, at some point, members of $A$ may "transition" into $B$, but the reverse is not possible. The timing of the transition is variable, but non-random. For example, $A$ could be "individuals without offspring" and $B$ "individuals with at least one offspring". I am interested in the age this progression occurs but I only have cross-sectional data. For any given individual, I can find out if they belong to $A$ or $B$. I also know the age of these individuals. For each individual in population $A$, I know that the age at transition will be GREATER THAN their current age. Likewise, for members of $B$, I know that the age at transition was LESS THAN their current age. But I don't know the exact values.  Say I have some other factor that I want to compare with the age of transition. For example, I want to know whether an individual's subspecies or body size affects the age of first offspring. I definitely have some useful information that should inform those questions: on average, of the individuals in $A$, older individuals will have a later transition. But the information is imperfect, particularly for younger individuals. And vice versa for population $B$.  Are there established methods to deal with this sort of data? I do not necessarily need a full method of how to carry out such an analysis, just some search terms or useful resources to start me off in the right place!  Caveats: I am making the simplifying assumption that transition from $A$ to $B$ is instantaneous. I am also prepared to assume that most individuals will at some point progress to $B$, assuming they live long enough. And I realise that longitutinal data would be very helpful, but assume that it is not available in this case.  Apologies if this is a duplicate, as I said, part of my problem is that I don't know what I should be searching for. For the same reason, please add other tags if appropriate.  Sample dataset: Ssp indicates one of two subspecies, $X$ or $Y$. Offspring indicates either no offspring ($A$) or at least one offspring ($B$)   age ssp offsp   21   Y     A   20   Y     B   26   X     B   33   X     B   33   X     A   24   X     B   34   Y     B   22   Y     B   10   Y     B   20   Y     A   44   X     B   18   Y     A   11   Y     B   27   X     A   31   X     B   14   Y     B   41   X     B   15   Y     A   33   X     B   24   X     B   11   Y     A   28   X     A   22   X     B   16   Y     A   16   Y     B   24   Y     B   20   Y     B   18   X     B   21   Y     B   16   Y     B   24   Y     A   39   X     B   13   Y     A   10   Y     B   18   Y     A   16   Y     A   21   X     A   26   X     B   11   Y     A   40   X     B    8   Y     A   41   X     B   29   X     B   53   X     B   34   X     B   34   X     B   15   Y     A   40   X     B   30   X     A   40   X     B   Edit: example dataset changed as it wasn't very representative
35508
I am trying for teaching purpose to explain how basically linear regression might work and how it could be extended. Since I am not the TeXpert I fear that my idea might be tricky and more difficult than it probably can be. I have seen many nice plots with tikz and pgfplots.  I searched around and found finally the following image.    This image looks quite appropriate, cause I can explain how the regression might look like. But the problem is that I cannot manipulate, e.g. draw grid, change the naming of x_0 resp \epsilon, change the red line to a dotted one, add other points than P and Q, etc. If the function itself is x^2 or some mixture, I don't mind for explanation. My main objective is to illustrate the basic ideas. And if I would have a sample, then I could extend it little bit further to mean, variance, etc.  Is anybody willing to give me a helping hand? I don't mind if somebody could point me to existing samples. Would be fine as well.
17759
I'm about to open the door to a very thorny issue in the social sciences.  How does one correctly model and test hypotheses about mediating variables using observational data?  I'm familiar with the Baron-Kenny approach to mediation , and also with structural equation modeling.  However, I've heard both approaches disparaged by more quantitative-minded social scientists than myself -- especially when one is using observational rather than experimental data.  So, let's say that I'm trying to resolve the following:   Y is a behavioral outcome. Both X and Z are observed characteristics of subjects that cannot be manipulated by an experiment. X is an attitude (something that can be changed over the long term) and Z is an unchangeable characteristic such as age, race, etc.   I hypothesize that X is a mediating variable, thus Z affects Y through the pathway of X.  While it's reasonable to suggest that Z is in some way correlated with X, my theory argues that it has no impact on Y (other than through **X).  How would one best test these hypotheses using best practices in current research?
120459
Does the following qualify as a univariate regression?  $$y=b_0+b_1x+b_2x^2+\epsilon$$  I fully comprehend the implications of adding regressors and need no background information - a "yes" or "no" will do :-)
120760
So I asked  question and didn't get a response that cleared up my understanding. I came across some material today that provokes my question again in a different context. In some slides I'm going through, the creator included this diagram:  Which was referred to as &quot;Ballentine Graph&quot;. If we run the regression $\hat y = \hat \beta_0 + \hat \beta_1 X_1$ Then we'll recover the yellow and brown portion of this graph as $\hat \beta_1$. Similarly for $X_2$. If we run the regression $\hat y = \hat \beta_0 + \hat \beta_1 X_1 + \hat \beta_2 X_2$ then $\hat \beta_1$ will now only be the yellow area. Similar results for $X_2$. My question is, what happens to the brown area when $X_1$ and $X_2$ are both included in the regression? It'd be included in the coefficient recovered if $X_1$ and $X_2$ were included separately but it's not included in either if both regressors are included. Why does brown disappear from both regressors. Where in mathematical space did it go? To me, it seems like $X_1$ and $X_2$ would have to share it but since this isn't the case, did it go into $\hat \beta_0$? Why?
15006
If granted omniscience and we know that $\beta_1$ in a multiple linear regression model is truly 0, what does that mean in words (and math notation)?  The model is: $Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \epsilon$  Does $\beta_1 = 0$ mean:   Empirically: "If we make infinite, perfect measurements of our data, $\hat\beta_1$ approaches 0 at the limit?" [Math: ??] Formally: "In the true, finite population of our data, $X_1$ has 0 association with $Y$ and their vectors of values are perfectly linearly independent, uncorrelated, and orthogonal?" [Math: ??]   What does $\beta_1 = 0$ mean, precisely?
74916
In a problem that I'm solving I find that:  "Let data (yi,xi) be sampled randomly from a two-dimensional distribution such that y|x is N(ɑ,x^2σ^2)".  Are y and x i.i.d? maybe just identically distributed but not independent?  Finally, if my model is: y = ɑ - u (where u is the error term and ɑ is a constant), and I also know that the distribution of x does not depend on either a or σ (I don't know it this assumption is needed for answer at the following question), is u independent from x?  My question is due to the fact that I thought that if the data are sampled randomly this mean that the r.v. are independent, but how is possible that the conditional distribution (his variance) depend then on x?
18045
Let's say I test how variable Y depends on variable X under different experimental conditions and obtain the following graph:    The dash lines in the graph above represent linear regression for each data series (experimental setup) and the numbers in the legend denote the Pearson correlation of each data series.  I would like to calculate the "average correlation" (or "mean correlation") between X and Y. May I simply average the r values? What about the "average determination criterion", $R^2$? Should I calculate the average r and than take the square of that value or should I compute the average of individual $R^2$'s?
35887
In a simple regression model,   $$ y = \beta_0 + \beta_1 x + \varepsilon, $$  the OLS estimators $\hat{\beta}_0^{OLS}$ and $\hat{\beta}_1^{OLS}$ are correlated.   The formula for the correlation between the two estimators is (if I have derived it correctly):  $$ \operatorname{Corr}(\hat{\beta}_0^{OLS},\hat{\beta}_1^{OLS}) = \frac{-\sum_{i=1}^{n}x_i}{\sqrt{n} \sqrt{\sum_{i=1}^{n}x_i^2} }. $$  Questions:   What is the intuitive explanation for the presence of correlation?   Does the presence of correlation have any important implications?   The post was edited and the assertion that the correlation vanishes with sample size has been removed. (Thanks to @whuber and @ChristophHanck.)
27355
Let's take the following example:  set.seed(342) x1 &lt;- runif(100) x2 &lt;- runif(100) y &lt;- x1+x2 + 2*x1*x2 + rnorm(100) fit &lt;- lm(y~x1*x2)   This creates a model of y based on x1 and x2, using a OLS regression.  If we wish to predict y for a given x_vec we could simply use the formula we get from the summary(fit).  However, what if we want to predict the lower and upper predictions of y? (for a given confidence level).  How then would we build the formula?
84058
If the best linear approximation (using least squares) of my data points is the line $y=mx+b$, how can I calculate the approximation error? If I compute standard deviation of differences between observations and predictions $e_i=real(x_i)-(mx_i+b)$, can I later say that a real (but not observed) value $y_r=real(x_0)$ belongs to the interval $[y_p-\sigma, y_p+\sigma]$ ($y_p=mx_0+b$) with probability ~68%, assuming normal distribution?  To clarify:  I made observations regarding a function $f(x)$ by evaluating it a some points $x_i$. I fit these observations to a line $l(x)=mx+b$. For $x_0$ that I did not observe, I'd like to know how big can $f(x_0)-l(x_0)$  be. Using the method above, is it correct to say that $f(x_0) \in [l(x_0)-\sigma, l(x_0)+\sigma]$ with prob. ~68%?
61941
I think this is a really simple question so please bear with me - I just had my first class in regression and I'm a little confused about nomenclature/labeling.  Does anyone recommend some good weblinks that explain beginning linear regression really well?  There's a question I've been looking at for a while and I'm not sure how to do it (although I'm sure the solution is simple):  Show that the sample estimate $\hat{\mu}(X) = \frac{1}{n} \sum X_i$ is a least square estimator of $\mu$ for a variable $X$ given $X_1, \ldots, X_n$.  My first thought was,  $\mathrm{SSE} = \sum (\mu - \hat{\mu})^2$  But I'm not sure if thats right. I'm confused about what the beta is (is it n?) and I don't know if there are enough parameters to expand it.  Thanks so much for your patience and if this doesn't make sense, I can clarify more. Thanks!
91877
I am trying to get a little into statistics, but I am stuck with something.  My data are as follows:  Year   Number_of_genes 1990          1 1991          1 1993          3 1995          4   I now want to build a regression model to be able to predict the number of genes for any given year based on the data. I did it with linear regression until now, but I have done some reading and it does not seem to be the best choice for this kind of data. I have read that Poisson regression might be useful, but I am unsure what to use. So my question is:  Is there a general regression model for this kind of data? If no, what do I have to do to find out which method is the most  appropriate to use ( in terms of what I have to find out about the data)?
26334
The OLS estimate $b$ is equal to $(X^TX)^{-1}X^Ty$ for the linear regression model. If we assume that $E(\epsilon|X)=0$ then it is easy to prove simply by taking the conditional expectation, of $b$ substituting in the expression for $y$ and simplifying.  But how do we prove it if we only know that $E(X^T\epsilon)=0$? $E(\epsilon|X)=0$ implies $E(X^T\epsilon)=0$, but not the other way around.  EDIT: Can I just get an answer, just to be sure, that even if the $u_i$'s are i.i.d., that $E(x_iu_i)=0$ does not imply unbiasedness?   So just to be absolutely clear: If $u_i$ is i.i.d., and we know that $E(x_iu_i)=0$ holds, but we don't know whether $E(u_i|x_i)=0$ holds, then OLS may be biased?
59873
In a set of  that I stumbled upon online, the author discusses a nonlinear regression model, which is linear in some parameters, like this $$ y = \theta_1 + \theta_2\exp \left( {\theta_3x} \right) + \varepsilon $$ and suggests to first perform a nonlinear regression to estimate $\theta_3$, and then use OLS to estimate the other two parameters. I am not very experienced in regression modeling, so please forgive me if I ask a very simple question, but I don't understand how you can do this consistently without also estimating the other 2 parameters? Apparently, there is some trick involved, which I don't understand...  Another question is what happens to the ability to do inferences based on parameter estimates to the complete model if you do the regression in two stages like that?  Many thanks!
116069
In Element of Statistical Learning, chapter 4-2 about linear regression of an indicator matrix, it is stated that the sum of predictor is equal to 1.  To bring a bit of context: We have $N$ training samples of size $p$ represented by a $N$ x $(p+1)$ matrix (intercept included) $\textbf{X}$. Each training sample $x_i$ belong to a category $g_i \in [| 1, .., K |]$. We use the $N$ x $K$ indicator matrix for the training samples which contains only one 1 per row. We try to fit a linear regression model for the parameter $\textbf{B}$ and we get $\hat{\textbf{B}} = (\textbf{X}^T\textbf{X})^{-1}\textbf{X}^T\textbf{Y}$.  For a new data, we compute the fitted output $\hat{f}(x)^T = (1,x^T)\hat{\textbf{B}}$ and the category is then $arg \max_{k \in [| 1, .., K |]} \hat{f}_k(x)$  They say that $\sum_{k \in [| 1, .., K |]} \hat{f}_k(x) = 1$ for any $x$  as long as there is an intercept in the model.  My question is thus why? I try to use the coefficient relation from the regression derivation ($\textbf{X}^T(\textbf{Y} - \textbf{X}\hat{\textbf{B}}) = 0$) but without success.
88319
I stumbled over a rather simple result of OLS regression which is  $$ Var[\hat\beta] = \sigma^2(X^TX)^{-1} $$ where $\sigma$ is the variance of the error term $u$ and $X$ is the regressor matrix.  I first just accepted the proof in my textbook but now I am thinking that it either uses sloppy notation or I am missing something. $\hat\beta$ is the estimated and $\beta$ is the true parameter (assuming unbiasedness).  It states that   \begin{align} Var[\hat\beta] &amp;= E[(\hat\beta - \beta)(\hat\beta-\beta)^T] \\                &amp;= E[(X^TX)^{-1}X^Tuu^TX(X^TX)^{-1}]         \\                &amp;=  (X^TX)^{-1}X^T E[uu^T] X(X^TX)^{-1} \end{align}  but $X$ was only assumed to be exogenous and not non-stochastic. Under this assumption I think $X$ cannot be dragged outside the expectation operator.   Momentarily, I think that it should be $Var[\hat\beta|X]$ to make sense. Is that the case? My web research couldn't clarify this. I only found similar derivations to the above without further explainations.