I have a problem with the interpretation of a test result in which the p-value is 1.727e-05. How should it be interpreted? What is the e for?
33635
I have a problem with the interpretation of a test result in which the p-value is 8.6e-28. How should it be interpreted? What is the e for?  Specifically, I used White's Test for the homoscedasticity assumption of a linear regression:  White's general test statistic :  150.0535  Chi-sq( 9)  P-value =  8.6e-28
114341
I have these observations $(2,3.2,3.8,2.5,3.3,2.8,3.0,3.4)$ from $X \sim N(\mu,\sigma^2)$ and i want to calculate the $P$-value testing $H_0: \mu =3.2$ against $H_1 \neq 3.2$ with $\sigma = 0.6$    should i calculate $P r(X &gt; 3.3) $ and $Pr( X &lt; 3)$ and add these togheter ?   I tried it using that  $X = Y\sigma + \mu  \sim N(0,1)$ but the answer don't seem to be right according to the solution which is $P = 0.347$   Anyone can tell me what Iam doing wrong?
88380
I was wondering if anybody could give a concise rundown as to the definitions and uses of p-values, significance level and type I error.  I understand that p-values are defined as "the probability of obtaining a test statistic at least as extreme as the one we actually observed", while a significance level is just an arbitrary cutoff value to gauge if the p-value is significant or not. Type I error is the error of rejected a null hypothesis that was true. However, I am unsure regarding the difference between significance level and the type I error, are they not the same concept?  For example, assume a very simple experiment where I flip a coin 1000 times and count the number of times it lands on 'heads'. My null hypothesis, H0, is that heads = 500 (unbiased coin). I then set my significance level at alpha = 0.05.  I flip the coin 1000 times and then I calculate the p-value, if the p-value is > 0.05 then I fail to reject the null hypothesis and if the p-value is &lt; 0.05 then I reject the null hypothesis.  Now if I did this experiment repeatedly, each time calculating the p-value and either rejecting or failing to reject the null hypothesis and keeping a count of how many I rejected/failed to reject, then I would end up rejecting 5% of null hypotheses which were in actuality true, is that correct? This is the definition of type I error. Therefore, the significance level in Fisher significance testing is essentially the type I error from Neyman-Pearson hypothesis testing if you performed repeated experiments.  Now as for p-values, if I had gotten a p-value of 0.06 from my last experiment and I did multiple experiments and counted all the ones that I got a p-value from 0 to 0.06, then would I also not have a 6% chance of rejecting a true null hypothesis?
19758
I don't understand which R code I am supposed to be using to figure these problems out. I brief explanation of what the code is doing would be amazing. The problems below are two different applications I'm expected to understand and I don't get either. Thank you!  Problem 1. Let Z~N(0,1) For each of the following, find c such that the equation is true.   a) P(Z&lt;c) = 0.8413  b) P(0&lt;Z&lt;c) = 0.3051  c) P(-c&lt;Z&lt;c) = 0.8664   Problem 2. X~N(3,4) compute  a) P(X&gt;3)  b) P(1&lt;X&lt;8)  c) P(-1.5&lt;X&lt;1)
67927
I'm aware of several related questions here (e.g., , ) but I don't know the definitive answer for my question below.  Suppose a hypothesis test where we want to test whether a coin is fair or not. We have two hypotheses:  $H_0: p(head)=0.5$  $H_1: p(head)\neq0.5$  Suppose we use 5% significance level, there are two possible cases:   When we obtain the data and find that the p-value is less than 0.05, we say "With significance level 5%, we reject $H_0$."  p-value is greater than 0.05, then we say "With 5% significance level, we cannot reject $H_0$."   My question is:      In case 1, is it correct to say "we accept $H_1$"?   Intuitively, and from what I have learned in the past, I feel that "accepting" anything as a result of hypothesis testing is always incorrect. On the other hand, in this case, since the union on $H_0$ of $H_1$ covers the whole "space", "reject $H_0$" and "accepting $H_1$" look exactly the same to me. On another thought, I can also think of the following idea, which says it is incorrect to say "we accept $H_1$":     We have an evidence strong enough to believe that $H_0$ is not true, but we may not have an evidence strong enough to believe that $H_1$ is true. Therefore, "rejecting $H_0$" does not automatically imply "accepting $H_1$"   So, what is the right answer?
112414
Recently I have noticed that Type I error of a test is too much persistent. Even if all the values are almost equal (variance is extremely low) and tested to that value (true mean), still there is nearly 5% Type I error of the test. I am frightened and curious to rethink, are the people who have banned (or skeptic about) NHST (null hypothesis significance testing) truly correct? Let's have a look at the following r code:       n=10000 t1err=0 set.seed(1235) for (i in 1:n){    x=rnorm(100, 10, 20)    if (((t.test(x, mu=10))$p.value)&lt;=0.05) (t1err=t1err+1)  } cat("Type I error rate in percentage is", (t1err/n)*100,"%")   It will produce roughly 5% as Type I error. Seems ok. But let's see this:      n=10000   t1err=0   set.seed(1235)   for (i in 1:n){      x=rnorm(100, 10, 0.000001)      if (((t.test(x, mu=10))$p.value)&lt;=0.05) (t1err=t1err+1)    }   cat("Type I error rate in percentage is", (t1err/n)*100,"%")   Each values are extremely close to 10. Still it produces about 5% Type I error. What conclusion we may draw from it? Or, am I missing something?
61571
I always believed that the p value was $P(H_0|Data)$ -- which to my rude shock I recently learned was not the case -- Note: I studied traditional frequentist probability.  How can one express the p-value in terms of Bayesian concepts -- assuming one can?
71525
P-value is defined the probability of obtaining a test-statistic at least as extreme as what is observed, assuming null-hypothesis is true. In other words,  $$P( X \ge t | H_0 )$$ But what if the test-statistic is bimodal in distribution? does p-value mean anything in this context? For example, I am going to simulate some bimodal data in R:    set.seed(0) # Generate bi-modal distribution bimodal &lt;- c(rnorm(n=100,mean=25,sd=3),rnorm(n=100,mean=100,sd=5))  hist(bimodal, breaks=100)     And let's assume we observe a test statistic value of 60. And here we know from the picture this value is very unlikely. So ideally, I would want a statistic procedure that I use(say, p-value) to reveal this. But if we compute for the p-value as defined, we get a pretty high-p value  observed &lt;- 60  # Get P-value sum(bimodal[bimodal &gt;= 60])/sum(bimodal) [1] 0.7991993   If I did not know the distribution, I would conclude that what I observed is simply by random chance. But we know this is not true.   I guess the question I have is: Why, when computing p-value, do we compute the probability for the values "at least as extreme as" the observed?  And if I encounter a situation like the one I simulated above, what is the alternative solution?
17973
I recently came across the paper . The author raised a few common misconceptions regarding hypothesis testing and p-values, about which I have two specific questions:   The p-value is technically $P({\rm observation}|H_{0})$, which, as pointed out by the paper, generally does not tell us anything about $P(H_{0}|{\rm observation})$, unless we happen to know the marginal distributions, which is rarely the case in "everyday" hypothesis testing. When we obtain a small p-value and "reject the null hypothesis," what exactly is the probabilistic statement that we are making, since we cannot say anything about $P(H_{0}|{\rm observation})$? The second question relates to a particular statement from page 6(652) of the paper:      Since the p-value, or range of p-values indicated by stars, is not set a priori, it is not the long-run probability of making a Type I error but is typically treated as such.   Can anyone help to explain what is meant by this statement?
85974
Suppose that $T_1,…,T_{10}$ are iid $Exp(λ)$ and the goal is to test if $H_0:λ≤1$ versus $H_a:λ≥2$. Suppose that the test statistic is $S=\sum_{i=1}^{10}T_i$, and rejection of the null occurs when $S≤7$.  I realize that $T_1...T_{10}$ is a gamma distribution  of $S =Gamma(10, \lambda$), but I am not sure how to use this information to find the p-value for the type 1 error.
110195
The image below is from  in Psychological Science. A colleague pointed out two unusual things about it:    According to the caption, the error bars show "±2.04 standard errors, the 95% confidence interval." I've only ever seen ±1.96 SE used for the 95% CI, and I can't find anything about 2.04 SE being used for any purpose. Does 2.04 SE have some accepted meaning?  The text states that planned pairwise comparisons found significant differences for mean startle magnitude in error vs. correct predictable trials (t(30)=2.51, p&lt;.01) and error vs. correct unpredictable trials (t(30)=2.61, p&lt;.01) (the omnibus F test was also significant at p&lt;.05). However, the graph shows the error bars for all three conditions overlapping substantially. If the ±2.04 SE intervals overlap, how can the values be significantly different at p&lt;.05? The overlap is large enough that I'm assuming that the ±1.96 SE intervals also overlap.
73943
I was reading through a paper and I saw a table with a comparison between PPV (Positive Predictive Value) and NPV (Negative Predictive Value). They did some kind of statistical test for them, this is a sketch of the table:  PPV    NPV    p-value 65.9   100    &lt; 0.00001 ...   Every rows refers to a particular contingency table.   What kind of hypothesis test did they do? Thanks!
30011
I have the following probability function:  $$\text{Prob} = \frac{1}{1 + e^{-z}}$$  where  $$z = B_0 + B_1X_1 + \dots + B_nX_n.$$  My model looks like  $$\Pr(Y=1) = \frac{1}{1 + \exp\left(-[-3.92 + 0.014\times(\text{gender})]\right)}$$  I understand what the intercept (3.92) means, but I'm now sure how to interpret 0.014. Are these still log odds, odd ratios, or can I now assert that for each incremental odds change is gender, females are 0.014 more likely to win than men. Basically, how am I to interpret the 0.014?   Basically, I want to take the probability function and actually implement it in Java for a specific program that I'm writing, but I'm just not sure if I'm understanding the function correctly to implement it in Java.  Java code example:   double p = 1d / (1d + Math.pow(2.718d, -1d * (-3.92d + 0.014d * bid)));
2142
In the  in the caret package, the p-value is described as:     a one-sided test to see if the accuracy is better than the "no information   rate," which is taken to be the largest class percentage in the data.   But what precisely is "a one sided test" here? I am assuming some p-value test between the accuracy and NIR. Specifically, what does "better than the NIR" mean?  Moreover, am I correct to assume that "the largest class percentage in the data" could be computed as number of tuples with most common class label / total number of tuples?
99864
I have created a mixed distribution model comprising 80% $H_0$ plus 20% $H_1$ to illustrate the link between the expected proportions of true and false positives and negatives in the PDF, CDF and p-value distributions.  When the effect size is small (25% increase in expected correlations vs $H_0$), everything looks consistent:    However, when I increase the effect size to 75% by separating $H_1$ from $H_0$, things don't seem to add up because I get apparent false negatives (grey area) in the p-value distribution, which are no longer there in the PDF and CDF distributions:    Conversely, when I reduce the effect size to 0% (so that $H_1$ = $H_0$), the grey area disappears altogether from the p-value distribution, whereas it is even more prominent in the PDF and CDF distributions:    Now, I'm sure I have done all the maths right and the actual distribution curves are correct and consistent, but perhaps I am misinterpreting the areas between these curves?  What do these four different coloured areas mean in each chart, and how do they relate to the expected proportions of true and false positives and negatives?  How can I show the link between these proportions in the three distributions, and why don't they correspond with each other as shown?  Please help me understand the apparent discrepancy, thanks!
62396
I have been reading Geoff Cumming's 2008 paper  [~200 citations in Google Scholar] -- and  am confused by one of its central claims. This is one in the series of papers where Cumming argues against $p$-values and in favour of confidence intervals; my question, however, is not about this debate and only concerns one specific claim about $p$-values.   Let me quote from the abstract:     This article shows that, if an initial   experiment results in two-tailed $p= .05$, there is an $80\%$   chance the one-tailed $p$-value from a replication will fall in   the interval $(.00008, .44)$, a $10\%$ chance that $p &lt; .00008$,   and fully a $10\%$ chance that $p &gt; .44$. Remarkably, the   interval—termed a $p$ interval—is this wide however large   the sample size.   Cumming claims that this "$p$ interval", and in fact the whole distribution of $p$-values that one would obtain when replicating the original experiment (with the same fixed sample size), depend only on the original $p$-value $p_\mathrm{obt}$ and do not depend on the true effect size, power, sample size, or anything else:     [...] the probability distribution of $p$ can be derived without   knowing or assuming a value for $\delta$ (or power). [...] We do not assume any prior knowledge   about $\delta$, and we use only the information $M_\mathrm{diff}$ [observed between-group difference] gives about $\delta$ as   the basis for the calculation for a given $p_\mathrm{obt}$ of the distribution of   $p$ and of $p$ intervals.      $\quad\quad\quad$    I am confused by this because to me it seems that the distribution of $p$-values strongly depends on power, whereas the original $p_\mathrm{obt}$ on its own does not give any information about it. It might be that the true effect size is $\delta=0$ and then the distribution is uniform; or maybe the true effect size is huge and then we should expect mostly very small $p$-values. Of course one can start with assuming some prior over possible effect sizes and integrate over it, but Cumming seems to claim that this is not what he is doing.  Question: What exactly is going on here?    Note that this topic is related to this question:  with an excellent answer by @whuber. Cumming has a paper on this topic to:  -- but that one is clear and unproblematic.  I also note that Cumming's claim is repeated several times in the 2015 Nature Methods paper  that some of you might have come across (it already has ~100 citations in Google Scholar):     [...] there will be substantial variation in the $P$ value of repeated experiments. In reality, experiments are rarely repeated; we do not know how different the next $P$ might be. But it is likely that it could be very different. For example, regardless of the statistical power of an experiment, if a single replicate returns a $P$ value of $0.05$, there is an $80\%$ chance that a repeat experiment would return a $P$ value between $0$ and $0.44$ (and a $20\%$ change [sic] that $P$ would be even larger).   (Note, by the way, how, irrespective of whether Cumming's statement is correct or not, Nature Methods paper quotes it inaccurately: according to Cumming, it's only $10\%$ probability above $0.44$. And yes, the paper does say "20% change". Pfff.)
124081
This is clearly just a matter of definition or convention, and of almost no practical importance. If $\alpha$ is set to its traditional value of 0.05, is a $p$ value of 0.0500000000000... considered to be statistically significant or not? Is the rule to define statistical significance usually considered to be $p &lt; \alpha$ or $p \leq \alpha$??