First of all, sorry for catchy title, my question is not that broad as it suggests.  I just came to conclusion that I don't need parametric tests. Instead, I need some feedback if my reasoning makes sense. Here it is:  To use some parametric test (like t-test or ANOVA), we have to check assumptions of it (normality, heterogeneity of vaiance). But all the tests for assumptions (Shapiro-Wilk, Kolmogorov-Smirnov, Anderson-Darling, Bartlett, Snedecor's F and so on) test null hypothesis of assumption being met against alternative that it is violated.  So I can only reject null (conclude that assumption is violated) or say that I don't have enough evidence to do that. Typically, in later situation, I use parametric tests. This means that I use it when I do not actually know if assumptions are met.   My conclusion: I should use nonparametric test anyway.  My conclusion 2: I can use parametric test only when I have some "external" knowlegde about assumptions being met (like I somehow know exact distributions of my data).   That's all. I'll be greatful for any suggestions if above makes sense.  Edit: I know that parametric tests are more powerful when their assumptions are met. My concern is that we can never know that this is the case (unless we have "external" knowlegde). So, I think, using parametric tests is pointless since we don't know if we have data appropriate for them.  Analogy that comes to my mind: Imagine two stain removers: Parametric Remover is perfect for cherry juice stains and Nonparametric Remover is OK, but not perferct, for all the stains. Since I can never confirm that my stain is cherry juice, I should always use Nonparameric Remover.
89556
Can someone explain to me why would anyone choose a parametric over a nonparametric statistical method for hypothesis testing or regression analysis?   In my mind, it's like going for rafting and choosing a non-water resistant watch, because you may not get it wet. Why not use the tool that works on every occasion?
15434
This issue seems to rear its ugly head all the time, and I'm trying to decapitate it for my own understanding of statistics (and sanity!).   The assumptions of general linear models (t-test, ANOVA, regression etc.) include the "assumption of normality", but I have found this is rarely described clearly.   I often come across statistics textbooks / manuals / etc. simply stating that the "assumption of normality" applies to each group (i.e., categorical X variables), and we should we examining departures from normality for each group.   Questions:     does the assumption refer to the values of Y or the residuals of Y?  for a particular group, is it possible to have a strongly non-normal distribution of Y values (e.g., skewed) BUT an approximately (or at least more normal) distribution of residuals of Y?   Other sources describe that the assumption pertains to the residuals of the model (in cases where there are groups, e.g. t-tests / ANOVA), and we should be examining departures of normality of these residuals (i.e., only one Q-Q plot/test to run).  does normality of residuals for the model imply normality of residuals for the groups? In other words, should we just examine the model residuals (contrary to instructions in many texts)?  To put this in a context, consider this hypothetical example:   I want to compare tree height (Y) between two populations (X).   In one population the distribution of Y is strongly right-skewed (i.e., most trees short, very few tall), while the other is virtually normal  Height is higher overall in the normally distributed population (suggesting there may be a 'real' difference). Transformation of the data does not substantially improve the distribution of the first    population.  Firstly, is it valid to compare the groups given the radically different height distributions? How do I approach the "assumption of normality" here? Recall height in one population is not normally distributed. Do I examine residuals for both populations separately OR residuals for the model (t-test)?     Please refer to questions by number in replies, experience has shown me people get lost or sidetracked easily (especially me!). Keep in mind I am not a statistician; though I have a reasonably conceptual (i.e., not technical!) understanding of statistics.  P.S., I have searched the archives and read the following threads which have not cemented my understanding:
87477
Is it possible for one-way (with $N&gt;2$ groups, or "levels") ANOVA to report a significant difference when none of the $N(N-1)/2$ pairwise t-tests does?   In  @whuber wrote:      It is well known that a global ANOVA F test can detect a difference of means even in cases where no individual [unadjusted pairwise] t-test of any of the pairs of means will yield a significant result.   so apparently it is possible, but I do not understand how. When does it happen and what the intuition behind such a case would be? Maybe somebody can provide a simple toy example of such a situation?  Some further remarks:   The opposite is clearly possible: overall ANOVA can be non-significant while some of the pairwise t-tests erroneously report significant differences (i.e. those would be false positives). My question is about standard, non-adjusted for multiple comparisons t-tests. If adjusted tests are used (like e.g. Tukey's HSD procedure), then it is possible that none of them turns out to be significant even though the overall ANOVA is. This is covered here in several questions, e.g.  and . Update. My question originally referred to the usual two-sample pairwise t-tests. However, as @whuber pointed out in the comments, in the ANOVA context, t-tests are usually understood as post hoc contrasts using the ANOVA estimate of the within-group variance, pooled across all groups (which is not what happens in a two-sample t-test). So there are actually two different versions of my question, and the answer to both of them turns out to be positive. See below.
54498
Hypothesis testing seeks to reject a null hypothesis ($H_0$) on the basis of an assumption made about the sample following a certain distribution. This assumption is conditional on $H_0$ being true. So what is the flaw with the following reasoning?     If I reject $H_0$ in favor of the alternative hypothesis ($H_1$), I   must then assume that (1) $H_1$ is true, and (2) that at the   same time the variable under examination follows the distribution   used to reject $H_0$. Since we also stated that the assumed   distribution is conditional on $H_0$ being true, it appears that $H_0$   has got to be true in order to justify the assumption made about the   distribution. Logically, it would therefore seem that both $H_0$ and   $H_1$ have to be true, which is a paradox, because they are mutually   exclusive!
62105
Suppose that we have a model $Y_i = \beta_0 + \beta_1X_{i1} + \beta_2X_{i2} + \dots + \beta_kX_{ik} + \epsilon_i$.  Regression has a number of assumptions, such as that the errors $\epsilon_i$ should be normally distributed with mean zero and constant variance. I have been taught to check these assumptions using a normal QQ plot to test for normality of the residuals $e_i = Y_i - \hat{Y}_i$ and a residuals vs. fitted plot to check that the residuals vary around zero with constant variance.  However, these tests are all on the residuals, not the errors.  From what I understand, the errors are defined as the deviation of each observation from their 'true' mean value. So, we can write $\epsilon_i = Y_i - \mathbb{E}[Y_i]$. These errors cannot be observed by us. *  My question is this: how good of a job do the residuals do in mimicking the errors?   If the assumptions appear to be satisfied on the residuals, does this mean they are satisfied on the errors too? Are there other (better) ways to test the assumptions, like fitting the model to a testing dataset and getting the residuals from there?    * Furthermore, does this not require that the model is specified correctly? That is, that the response really does have a relationship with the predictors $X_1, X_2,$ etc. in the way specified by the model.  If we are missing some predictors (say, $X_{k+1}\ \text{to}\ X_p$), then the expectation $\mathbb{E}[Y_i] = \beta_0 + \beta_1X_{i1} + \beta_2X_{i2} + \dots + \beta_kX_{ik}$ would not even be the true mean, and further analysis on an incorrect model seems pointless.  How do we check whether the model is a correct one?
49409
Traditional statistical tests, like the two sample t-test, focus on trying to eliminate the hypothesis that there is no difference between a function of two independent samples. Then, we choose a confidence level and say that if the difference of means is beyond the 95% level, we can reject the null hypothesis. If not, we "can't reject the null hypothesis". This seems to imply that we can't accept it either. Does it mean we're not sure if the null hypothesis is true?   Now, I want to design a test where my hypothesis is that a function of two samples is the same (which is the opposite of traditional statistics tests where the hypothesis is that the two samples are different). So, my null hypothesis becomes that the two samples are different. How should I design such a test? Will it be as simple as saying that if the p-value is lesser than 5% we can accept the hypothesis that there is no significant difference?
64382
An addition to the common confusion about normality testing in inferential statistics for general linear models:  I understand the assumption of normality refers to the residuals in ANOVA and regression models, but what about t-tests?  Calculation of the t-statistic uses standard deviation as a measure of dispersion, rather than sums of squares for the F-statistic in ANOVA and regression. If I understand correctly, it is because of this that there is an assumption that the samples are normally distributed (i.e. the sample standard deviation provides an accurate estimate of the population standard deviation). Is this right?  What do you do if the statistical population of the dependent variable is not distributed normally (thus samples are unlikely to be normally distributed)? Is this where transformations fit in?
100987
I was reading on the computation of the unbiased estimation of standard deviation and the source I read stated      (...) except in some important situations, the task has little relevance to   applications of statistics since its need is avoided by standard   procedures, such as the use of significance tests and confidence   intervals, or by using Bayesian analysis.   I was wondering if anyone could elucidate the reasoning behind this statement, for example doesn't the confidence interval use the standard deviation as part of the calculation? Therefore, wouldn't the confidence intervals be affected by a biased standard deviation?  EDIT:  Thanks for the answers so far, but I'm not quite sure I follow some of the reasoning for them so I'll add a very simple example. The point is that if the source is correct, then then something is wrong from my conclusion to the example and I would like someone to point how how the p-value doesn't depend on the standard deviation.  Suppose a researcher wished to test whether the mean score of fifth graders on a test in his or her city differed from the national mean of 76 with a significance level of 0.05. The researcher randomly sampled the scores of 20 students. The sample mean was 80.85 with a sample standard deviation of 8.87. This means: t = (80.85-76)/(8.87/sqrt(20)) = 2.44. A t-table is then used to calculate that the two-tailed probability value of a t of 2.44 with 19 df is 0.025. This is below our significance level of 0.05 so we reject the null hypothesis.  So in this example, wouldn't the p-value (and maybe your conclusion) change depending on how you estimated your sample standard deviation?
110753
In general inference, why orthogonal parameters are useful, and why is it worth trying to find a new parametrization that makes the parameters orthogonal ?   I have seen some textbook examples, not so many, and would be interested in more concrete examples and/or motivation.
118940
I have an experiment with two groups under different conditions where a single dependent variable is measured repeatedly at multiple times.  It seems the right parametric test to use here is :     "A mixed ANOVA compares the mean differences between groups that have   been split on two "factors" (also known as independent variables),   where one factor is a "within-subjects" factor and the other factor is   a "between-subjects" factor."   However, my data is strongly non-normal (and some of the dependent variables are ordinal anyway).    What is a non-parametric equivalent test to two-factor mixed ANOVA?
96319
I am reading the chapter on Frequent Statistics from Kevin Murphy's book "Machine Learning - A Probabilistic Perspective". The section on bootstrap reads:     The bootstrap is a simple Monte Carlo technique to approximate the   sampling distribution. This is particularly useful in cases where the   estimator is a complex function of the true parameters.       The idea is simple. If we knew the true parameters $θ^∗$ , we could   generate many (say $S$) fake datasets, each of size $N$, from the true   distribution, $x_i^s \sim p (·| θ^∗ )$, for $s = 1 : S, i = 1 : N$.    We could then compute our estimator from each sample,   $\hat{\theta^s}=f (x^s_{1:N})$ and use the empirical distribution of   the resulting samples as our estimate of the sampling distribution.   Since $\theta$ is unknown, the idea of the parametric bootstrap is to   generate the samples using $\hat{\theta}(D)$ instead.       An alternative, called the non-parametric bootstrap, is to sample the   $x^s_i$ (with replacement) from the original data $D$ , and then   compute the induced distribution as before. Some methods for speeding   up the bootstrap when applied to massive data sets are discussed in   (Kleiner et al. 2011).    1. The text says:       If we knew the true parameters $\theta^*$ ... we could compute our   estimator from each sample, $\hat{\theta^s}$...     &nbsp; &nbsp; &nbsp; &nbsp; but why would I use the estimator of each sample if I already know the  true parameters $\theta^*$?    2. Also, what is the difference here between the empirical distribution and the sampling distribution? 3. Finally, I don't quite understand the difference between parametric and non-parametric bootstrap from this text. They both infer $\theta$ from the set of observations $D$, but what is exactly the difference?
38352
I do not know why testing homogeneity of variance is so important. What are the examples that require homogeneity of variance?
62920
What statistical test can I use to compare two ratios from two independent samples. The ratios are after to before results. I need to compare the after/before ratios for two independent models and show whether they are have significant difference or not. Please help!
100272
I am a little bit confused on what the assumptions of linear regression are. So far I checked whether:  all of the explanatory variables correlated linearly with the response variable. (This was the case) there was any collinearity among the explanatory variables. (there was little collinearity). the Cook's distances of the datapoints of my model are below 1 (this is the case, all distances are below 0.4, so no influence points). the residuals are normally distributed. (this may not be the case)  But I then read the following:  violations of normality often arise either because (a) the distributions of the dependent and/or independent variables are themselves significantly non-normal, and/or (b) the linearity assumption is violated.  Question 1 This makes it sound as if the independent and depend variables need to be normally distributed, but as far as I know this is not the case. My dependent variable as well as one of my independent variables are not normally distributed. Should they be? Question 2 My QQnormal plot of the residuals look like this:   That slightly differs from a normal distribution and the shapiro.test also rejects the null hypothesis that the residuals are from a normal distribution: &gt; shapiro.test(residuals(lmresult)) W = 0.9171, p-value = 3.618e-06  The residuals vs fitted values look like:   What can I do if my residuals are not normally distributed? Does it mean the linear model is entirely useless?
35041
I looked-up many references and websites and researched on how to determine if a method is between parametric or non-parametric. I came up with below definitions,     A parametric algorithm has a fixed number of parameters. In contrast, a non-parametric algorithm uses a flexible number of parameters, and the number of parameters often grows as it learns from more data.   From .  Moreover, I found,     A parametric model, we have a finite number of parameters, and in nonparametric models, the number of parameters is (potentially) infinite.    From .  And many other methods of determination, though the problem is none of them can helo ones to determine if a certain hypothetical method is parametric or not. (For instance, why k-means' number of parameters are constant but KNN is variable or basically what do we call a parameter and what we do not?)
69265
When testing the difference between 2 proportions why do we use a z-test rather than a t-test?    Further, is there a simple way to conduct an omnibus test for significant differences between more than 2 proportions (in the form of percentages). Is there an equivalent to a one-way ANOVA for this?  I imagine that you could use logistic regression (assuming you have the original data for the proportions in the form of 0s and 1s) Are there other options?
102491
Can the group recommend a good introduction text/resource to applied resampling techniques? Specifically, I am interested in alternatives to classical parametric tests (e.g. t tests, ANOVA, ANCOVA) for comparing groups when assumptions such as normality are clearly violated.   An example problem type I would like to educate myself as to a better way to solve may involve something such as:  I) 2 Groups: Treatment and Control    Dependent Var: Change in account balance dollars after intervention  Covariate: Pre intervention account balance dollars.  Issue with applying ANCOVA: Many subjects will not have any change (many zeros).   II) 2 Groups: Treatment and Control  Dependent Var: new accounts added  Covariate: Pre intervention number of accounts.  *Many subjects will not have any added account (many zeros).   Can I use a bootstrap? A permutation test? This is the type of analysis I would like to apply nonparametric resampling methods to.