After some googling, I have found this definition of a parameter in statistics:      "a numerical characteristic of a population, as distinct from a statistic of a sample"   Or, more generally for mathematics:     "a quantity whose value is selected for the particular circumstances and in relation to which other variable quantities may be expressed"   But my understanding of parameters is that they are the betas (coefficients) that appear before the variables (which may be themselves without any transformation or may appear with an exponent, log, square root, or some other transformation,) in a regression equation, and are interpreted as the partial effects of the independent variables on the dependent variable.  A few questions: Are parameters the same as coefficients? If not, what makes them different? While linear regression uses parameters that are linear, how would non-linear parameters look and be interpreted? Are there any concrete examples of parameters in the real world, such as some constant relationship in physics?
28980
I'm relatively familiar with the distinction between the terms statistic and parameter. I see a statistic as the value obtained from applying a function to the sample data. However, most examples of parameters relate to defining a parametric distribution. A common example is the mean and standard deviation to parameterise the normal distribution or the coefficients and error variance to parameterise a linear regression.  However, there are many other values of the population distribution that are less prototypical (e.g., minimum, maximum, r-square in multiple regression, the .25 quantile, median, the number of predictors with non-zero coefficients, skewness, the number of correlations in a correlation matrix greater than .3, etc.).   Thus, my questions are:   Should any quantitative property of a population be labelled a "parameter"? If yes, then why? If no, what characteristics should not be labelled a parameter? What should they be labelled? And why?   Elaboration on confusion  The Wikipedia article on estimators states:     An "estimator" or "point estimate" is a statistic (that is, a function   of the data) that is used to infer the value of an unknown parameter   in a statistical model.   But I can define the unknown value as .25 quantile and I can develop an estimator for that unknown. I.e., not all quantitative properties of a population are parameters in the same way that say the mean and sd are parameters of a normal distribution, yet it is legitimate to seek to estimate any quantitative population property.
74044
I previously learned about sampling distributions that gave results which were for the estimator, in terms of the unknown parameter. For example, for the sampling distributions of $\hat\beta_0$ and $\hat\beta_1$ in the linear regression model $Y_i = \beta_o + \beta_1 X_i + \varepsilon_i$  $$ \hat{\beta}_0 \sim \mathcal N \left(\beta_0,~\sigma^2\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\right) $$ and $$ \hat{\beta}_1 \sim \mathcal N \left(\beta_1,~\frac{\sigma^2}{S_{xx}}\right) $$  where $S_{xx} = \sum_{i=1}^n (x_i^2) -n \bar{x}^2$  But now I have seen the following in a :     Suppose we fit the model by least squares in the usual way. Consider   the Bayesian posterior distribution, and choose priors so that this is   equivalent to the usual frequentist sampling distribution, that   is......      $$ \left( \begin{matrix} \beta_0  \\ \beta_1  \end{matrix} \right)  \sim \mathcal  N_2\left[\left(\begin{matrix} \hat{\beta}_1  \\  \hat{\beta}_2  \end{matrix} \right),~\hat{\sigma}^2  \left(\begin{matrix}   n &amp; \sum_{i=1}^{n}x_i \\   \sum_{i=1}^{n}x_i &amp;  \sum_{i=1}^{n}x_i^2  \end{matrix}  \right) ^{-1}\right] $$   This is confusing me because:     Why do the estimates appear on the left hand side (lhs) of the first 2 expressions, and the right hand side (rhs) of the last expression? Why do the beta hats in the last expression have 1 and 2 subscripts instead of 0 and 1? Are these just different representations of the same thing? If they are, could someone show my how they are equivalent? If not, could someone explain the difference? Is it the case that the last expression is the "inversion" of the first two? Is that why the 2x2 matrix in the last expression is inverted and estimates/parameters are switched from rhs$\leftrightarrow$lhs? If so could someone show me how to get from one to the others?
60217
I have noticed that such terms as model hyperparameter and model parameter have been used interchangeably on the web without prior clarification. I think this is incorrect and needs explanation. Consider a machine learning model, an SVM/NN/NB based classificator or image recognizer, just anything that first springs to mind.   What are the hyperparameters and parameters of the model? Give your examples please.
78937
Being aware of , I am curious about the question how big standardized coefficients can get. I had a discussion with my professor about that issue and she was arguing standardized coefficients (beta) in multiple linear regressions can not become greater than |1|. I have also heard that predictors with standardized coefficients greater than 1 should not be be included/appear in multiple linear regression. When I recently estimated a multiple linear regression in R using lm(), I estimated the standardized coefficients with lm.beta() function from the package 'lm.beta'. In the results I could observe a standardized coefficient greater than one. Right now I am just not sure about what is the truth.  Can standardized coefficients become greater than |1|? If yes, what does that mean and should they be excluded from the model? If yes, why?  I would be very thankful, if somebody could make this issue clear for me.
119620
Suppose, I have performed multiple regression analysis on the following data set where X1 and X2 are independent variables and Y is the dependent variable.    And achieved the following multiple regression analysis table from where it is clear that X2 is not correlated with Y as the p-value is greater than 0.05 which is 0.108 for X2 variable     So, if p value is greater than 0.05 for a variable, then why it is considered in regression equation  Y = 2.709 + 0.763(X1) + 0.463(X2) which has been shown in any online multiple regression equation calculator.  My question is Why it is not like the following Y = 2.709 + 0.763(X1) omitting the [0.463(X2)] part from the equation, as X2 is not correlated with Y  Please help to clarify the conceptual idea....
55591
I'm slightly confused if an independent variable (also called predictor or feature) in a statistical model, for example the $X$ in linear regression $Y=\beta_0+\beta_1 X$, is a random variable ?
1481
From , there are three interpretations of the degrees of freedom of a statistic:     In statistics, the number of degrees of freedom is the number of   values in the final calculation of a statistic that are free to vary.      Estimates of statistical parameters can be based upon different   amounts of information or data. The number of independent pieces of   information that go into the estimate of a parameter is called the   degrees of freedom (df). In general, the degrees of freedom of an   estimate of a parameter is equal to the number of independent scores   that go into the estimate minus the number of parameters used as   intermediate steps in the estimation of the parameter itself (which,   in sample variance, is one, since the sample mean is the only   intermediate step).      Mathematically, degrees of freedom is the dimension of the domain of a   random vector, or essentially the number of 'free' components: how   many components need to be known before the vector is fully   determined.   The bold words are what I don't quite understand. If possible, some mathematical formulations will help clarify the concept.  Also do the three interpretations agree with each other?
19575
X and Y are not correlated (-.01); however, when I place X in a multiple regression predicting Y, alongside three (A, B, C) other (related) variables, X and two other variables (A, B) are significant predictors of Y. Note that the two other (A, B) variables are significantly correlated with Y outside of the regression.   How should I interpret these findings? X predicts unique variance in Y, but since these are not correlated (Pearson), it is somehow difficult to interpret.  I know of opposite cases (i.e., two variables are correlated but regression is not significant) and those are relatively simpler to understand from a theoretical and statistical perspective. Note that some of the predictors are quite correlated (e.g., .70) but not to the extent that I would expect substantial multicollinearity. Maybe I am mistaken, though.    NOTE: I asked this question previously and it was closed. The rational was that this question is redundant with the question "". Perhaps I do not understand the other question, but I believe these are entirely separate questions, both mathematically and theoretically. My question is entirely independent from if "a regression is significant". Furthermore, several predictors are significant, while the other question entails variables not being significant, so I don't see the overlap. If these questions are redundant for reasons I do not understand, please insert a comment prior to closing this question. Also, I was hoping to message the moderator who closed the other question to avoid identical questions, but I couldn't find an option to do so.
53893
Say we fit a simple linear regression without the intercept term. I know this is inadvisable for the most part, unless there are some situations where it is reasonable to assume that when $x=0$, $y$ is also 0. I have been reading that when fitting without the intercept, it may cause the residual term to have non-zero mean. It makes sense but I can't seem to prove to myself (out of personal curiosity) why this is true. Can anyone help me understand it through a proof or some equations?
113453
I'm taking an intro to statistics class and currently we are covering linear regression. The course is good but unfortunately it doesn't always gives all the information that is needed. For example it gives formulas for the slope and y-intercept as  $b=(r)\frac{s_Y}{s_X}=\frac{n \sum{xy} - \sum{x} \sum{y}}{n \sum{x^2} - (\sum{x})^2}$ , where  $r$ is the correlation coefficient  $s_Y$ is the standard deviation of the $Y$ scores  $s_X$ is the standard deviation of the $X$ scores   and  $a = \bar{y} - b \bar{x} = \frac{\sum{y} - b \sum{x}}{n}$  However it doesn't explain why the formulas are what they are. In other words it doesn't explain how these formulas where derived. Can someone show the derivation, especially the one for the slope, $b$, i.e. why $b=(r) \frac{s_Y}{s_X}$?  Any help is appreciated.
28105
Is it possible for more than one "linear regression" line to fit a given set of points? (i.e.... "least squares" is minimised equally in both cases)  I'm assuming a simple one-variable regression in which we have a single "dependent" variable for a given x-value. It may be that regression is forced "through the origin" for the purpose of the Q (if that matters), so, I know it wouldn't be a great fit.  Or is there something intrinsic about a regression line that means it is unique? I am interested in a 'proof' (could be just an intuitive proof) if it exists, that a regression line must be unique.
61382
Beta regression (i.e. GLM with beta distribution and usually the logit link function) is often recommended to deal with response aka dependent variable taking values between 0 and 1, such as fractions, ratios, or probabilities: .  However, it is always claimed that beta regression cannot be used as soon as the response variable equals 0 or 1 at least once. If it does, one needs to either use zero/one-inflated beta model, or make some transformation of the response, etc.: .  My question is: which property of beta distribution prevents beta regression from dealing with exact 0s and 1s, and why?  I am guessing it is that $0$ and $1$ are not in the support of beta distribution. But for all shape parameters $\alpha&gt;1$ and $\beta&gt;1$, both zero and one are in the support of beta distribution, it's only for smaller shape parameters that the distribution goes to infinity at one or both sides. And perhaps the sample data are such that $\alpha$ and $\beta$ providing best fit would both turn out to be above $1$.  Does it mean that in some cases one could in fact use beta regression even with zeros/ones?   Of course even when 0 and 1 are in the support of beta distribution, probability of observing exactly 0 or 1 is zero. But so is the probability to observe any other given countable set of values, so this cannot be an issue, can it? (Cf.  by @Glen_b).  $\hskip{8em}$  In the context of beta regression, beta distribution is parameterized differently, but with $\phi=\alpha+\beta&gt;2$ it should still be well-defined on $[0,1]$ for all $\mu$.
83710
Say, $X$ is dependent on $\alpha$. Rigorously speaking,    if $X$ and $\alpha$ are both random variables, we could write $p(X\mid\alpha)$; however, if $X$ is a random variable and $\alpha$ is a parameter, we have to write $p(X; \alpha)$.   I notice several times that the machine learning community seems to ignore the differences and abuse the terms.  For example, in the famous LDA model, where $\alpha$ is the Dirichlet parameter instead of a random variable.    Shouldn't it be $p(\theta;\alpha)$? I see a lot of people, including the LDA paper's original authors, write it as $p(\theta\mid\alpha)$.
41027
I'm new to statistics and reading stats books I found different definitions for Variance.  Definition1: $ s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2 $  Definition2: $ s^2 = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 $  I would like to know why is used different definition for same thing?
78617
I Googled this, but some answers still leave me confused. One of the answers says:       "I just think of it in terms of the fitted line having 2 parameters, the intercept and the slope. But the intercept doesn't count after we adjust for the mean (which we do in obtaining SStotal), so that leaves just one df for regression."   How can you drop the intercept after adjust for the mean?
73933
I keep seeing conflicting definitions of a statistic. Is a statistic a random variable such that it is a function of the random variables of a random sample? Or is it the value of the function of the random sample, after each random variable of the random sample has taken on a specific value?   $$(1) \ S=f(X_1,X_2...X_n)$$  $$(2) \ s=f(x_1,x_2...x_n)$$  I haven't been able to get any clarification for this and I've seen the term statistic describe both situations
64218
I'm trying to justify using of GLM model in my project instead of a simple linear regression. A lot of sources that I've seen contain the statement that "GLM allow us to build regression models for response variables that are not normally distributed". I can't understand what that is supposed to mean since we don't make any assumptions about variable distribution. Could you, please, explain  why it's often more appropriate (theoretically)  to use GLM if your response variable is,  say, count.