I am using the Dunn test (the package Dunn.test in R) as a post-hoc test to my Kruskal Wallis analysis. My alpha value is 0.05, so I would reject the Ho for p - values &lt;= 0.05.   When printing my results, R prints the note:      alpha = 0.05   Reject Ho if p &lt;= alpha/2   Does anybody happen to know why R advises me that I should reject Ho if the p-value is 0.025 even at a set alpha value of 0.05? (For clarity: why is R using this particular decision rule of alpha/2 instead of just alpha?)  My guess would be that it has something to do with testing one- or two sided. I couldn't find if the default of Dunn.test and therewith the given results is for a one or two sided approach.
122400
Firstly, this is my first post to Cross Validated, so apologies in advance if I have infringed any conventions.  I am writing up a PhD thesis on aspects of biomedical research, and use R for my statistics. However, I always check some of the data against another package in case I have made a coding error.  I have been running a Kruskall-Wallis test with three groups (sample sizes of 5, 8 and 9) and using Dunn's test for the post-hoc pairwise comparisons. The p-values I get for the pairwise comparisons (unadjusted for multiple comparisons) are exactly half what I get in SPSS and GraphPad for the same data. The Kruskal-Wallis p-value reported by the dunn.test package is exactly what I get running the Kruskal-Wallis test itself in R, or in SPSS and Graphpad. There is a warning after I run my code that the dunn.test package was written on a slightly later version of R, but that seems unlikely to be the cause of a factor of 2 variation. Has anyone else come across a similar discrepancy?
71223
\documentclass[12pt,a4paper]{report} \usepackage[british]{babel} \usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} \usepackage[activate={true,nocompatibility},final,tracking=true,kerning=true,spacing=false]{microtype} \DisableLigatures[f]{encoding = T1} \microtypecontext{spacing=nonfrench} \usepackage{geometry} \geometry{     left=2cm,     right=2cm,     top=2cm,     bottom=2cm,     bindingoffset=0mm } \begin{document}     \noindent Note that the word “values” is not hyphenated as “val-ues”.     \begin{enumerate}         \item it could list every single one of them so that one knows that there are multiple mode values in a data set.         \item it could list every single one of them so that one knows that there are multiple mode val- ues in a data set.     \end{enumerate}     \noindent\rule{\textwidth}{0.1mm}\\      \noindent In the following example, the word “determined” is not hyphenated.     \vspace{3mm}      \noindent In a statistical hypothesis test, the \textbf{probability of an alpha error} (error probability) is determined. If the latter is low, $H_0$ can be rejected in favour of $H_1$!     \vspace{3mm}      \noindent In a statistical hypothesis test, the \textbf{probability of an alpha error} (error probability) is deter- mined. If the latter is low, $H_0$ can be rejected in favour of $H_1$! \end{document}
67927
I'm aware of several related questions here (e.g., , ) but I don't know the definitive answer for my question below.  Suppose a hypothesis test where we want to test whether a coin is fair or not. We have two hypotheses:  $H_0: p(head)=0.5$  $H_1: p(head)\neq0.5$  Suppose we use 5% significance level, there are two possible cases:   When we obtain the data and find that the p-value is less than 0.05, we say "With significance level 5%, we reject $H_0$."  p-value is greater than 0.05, then we say "With 5% significance level, we cannot reject $H_0$."   My question is:      In case 1, is it correct to say "we accept $H_1$"?   Intuitively, and from what I have learned in the past, I feel that "accepting" anything as a result of hypothesis testing is always incorrect. On the other hand, in this case, since the union on $H_0$ of $H_1$ covers the whole "space", "reject $H_0$" and "accepting $H_1$" look exactly the same to me. On another thought, I can also think of the following idea, which says it is incorrect to say "we accept $H_1$":     We have an evidence strong enough to believe that $H_0$ is not true, but we may not have an evidence strong enough to believe that $H_1$ is true. Therefore, "rejecting $H_0$" does not automatically imply "accepting $H_1$"   So, what is the right answer?
9302
Following on from my earlier posts, as far as I can understand, if I have three correlation coefficients, I will have to test them in pairs to see if there is a significant difference among them.   This means that I would have to use Fishers transformation to work out the z score of r and then the p value of z (which the recommended calculators in the earlier posts do, thankfully) and then ascertain if the p value is higher or lower than my alpha value (0.05) for each pair.  i.e. If 21 to 30 years olds is Age Group 1, 31 to 40 years is Age Group 2, and 41 to 50 years is Age Group 2, my comparison of the correlations between their shopping habits and weight loss would be:   Group 1 vs. Group 2 Group 1 vs. Group 3 Group 2 vs. Group 3   Instead of doing three separate calculations, is there a way to do all these calculations in a single step?
2142
In the  in the caret package, the p-value is described as:     a one-sided test to see if the accuracy is better than the "no information   rate," which is taken to be the largest class percentage in the data.   But what precisely is "a one sided test" here? I am assuming some p-value test between the accuracy and NIR. Specifically, what does "better than the NIR" mean?  Moreover, am I correct to assume that "the largest class percentage in the data" could be computed as number of tuples with most common class label / total number of tuples?
87725
Is it even possible to use a statistical test in this case? Specifically, I'm analysing cities in two different regions. The aim is to determine if there is a statistically significant difference between the amount of cities that have UK names in each region.   The data look like this:                   region1    region2 UK_named_cities  553        3255 total_cities     9155       55327   The percentages of cities with UK names:  region 1: 6.07% region 2: 5.88%   The difference is 0.19%.   My first thought was an $χ2$ test, which yielded this result:  Test Statistic (X-squared): 0.2840672425979418 p-value: 0.5940477093071286 Degrees of Freedom: 1   I'm not sure this is right though. I don't have expected and observed counts as such. And a p-value of 0.59 doesn't seem that intuitive when there is a difference (albeit a very small one). I can't see how a t-test or z-test or reporting CI's would apply either.  Fundamentally, I'm confused. Can I only report that there is indeed a difference, but not test it for significance?
118926
I'm doing a project in predicting default risk for SBA loans using R. My data has 187,986 positives and only 17796 negatives (9.46%). Clearly this is an imbalanced dataset. That's why when I run accuracy rate checking for my model using probability cutoff value=0.5, the accuracy result is way too low as below,  &gt; mytestset$LoanStatus_PIF&lt;-ifelse(mytestset$LoanStatus_PIF=="PIF",1,0) &gt; pred&lt;-predict(final,newdata=mytestset,type="response") &gt; y_pred_num&lt;-ifelse(pred&gt;0.5,1,0) &gt; mean(y_pred_num==mytestset$LoanStatus_PIF)   Result: [1] 0.009682553  When I change the cutoff value to 0.998 and up, the result is way much impressive:  &gt; y_pred_num&lt;-ifelse(pred&gt;0.5,1,0) &gt; mean(y_pred_num==mytestset$LoanStatus_PIF) Result:  [1] 0.9997752   Question: 1. Will my model be rejected because I manually choose the cut-off value?   Does 0.999 make sense for a cutoff value? As common cut-off value is usually 0.5. Thanks !
107948
I am trying to solve the following question:     Player A won 17 out of 25 games while player B won 8 out of 20 - is   there a significant difference between both ratios?   The thing to do in R that comes to mind is the following:  &gt; prop.test(c(17,8),c(25,20),correct=FALSE)      2-sample test for equality of proportions without continuity correction  data:  c(17, 8) out of c(25, 20) X-squared = 3.528, df = 1, p-value = 0.06034 alternative hypothesis: two.sided 95 percent confidence interval:  -0.002016956  0.562016956 sample estimates: prop 1 prop 2    0.68   0.40    So this test says that the difference is not significant at the 95% confidence level.  Because we know that prop.test() is only using an approximation I want to make things more exact by using an exact binomial test - and I do it both ways around:  &gt; binom.test(x=17,n=25,p=8/20)      Exact binomial test  data:  17 and 25 number of successes = 17, number of trials = 25, p-value = 0.006693 alternative hypothesis: true probability of success is not equal to 0.4 95 percent confidence interval:  0.4649993 0.8505046 sample estimates: probability of success                    0.68   &gt; binom.test(x=8,n=20,p=17/25)      Exact binomial test  data:  8 and 20 number of successes = 8, number of trials = 20, p-value = 0.01377 alternative hypothesis: true probability of success is not equal to 0.68 95 percent confidence interval:  0.1911901 0.6394574 sample estimates: probability of success                     0.4    Now this is strange, isn't it? The p-values are totally different each time! In both cases now the results are (highly) significant but the p-values seem to jump around rather haphazardly.  My questions   Why are the p-values that different each time? How to perform an exact two sample proportions binomial test in R correctly?
45552
Example code:  (pc.cr &lt;- princomp(USArrests))   summary(pc.cr) loadings(pc.cr)  ## note that blank entries are small but not zero   I am getting different outputs from each, and I am not sure I understand what the difference is.  Here is the output:  &gt; summary(pc.cr) Importance of components:                            Comp.1      Comp.2      Comp.3       Comp.4 Standard deviation     82.8908472 14.06956001 6.424204055 2.4578367034 Proportion of Variance  0.9655342  0.02781734 0.005799535 0.0008489079 Cumulative Proportion   0.9655342  0.99335156 0.999151092 1.0000000000   &gt; loadings(pc.cr)  ## note that blank entries are small but not zero  ...                 Comp.1 Comp.2 Comp.3 Comp.4 SS loadings      1.00   1.00   1.00   1.00 Proportion Var   0.25   0.25   0.25   0.25 Cumulative Var   0.25   0.50   0.75   1.00   P.S: how can I get access to the table created by summary(pc.cr)?? (I can't seem to find it in str.)
16354
I have a large dataset (500000 data, V1 column include all the data).  x &lt;- read.csv("mydata.csv", header=F) hist(x)   Which gives:    Looking at the data, I believe it is not a normal distribution. As a further check, I constructed a qqplot:  x_norm &lt;- (x$V1 - mean(x$V1))/sd(x$V1) qqnorm(x_norm); abline(0, 1)   which gave:    To check the goodness of fit of x$V1 (rawdata) to a normal distribution, I used:  rnorm &lt;- rnorm(500000, mean(x$V1), sd(x$V1)) cc &lt;- cbind(rnorm, x$V1) g &lt;- goodfit(cc, method="MinChisq") summary(g)           Goodness-of-fit test for poisson distribution               X^2 df      P(&gt; X^2) Pearson 914.5227 17 1.679266e-183 Warning message: In summary.goodfit(g) : Chi-squared approximation may be incorrect   With plot(g) giving:    Does this seem correct? Can I confidently conclude my dataset X$V1 is or is not a normal distribution?  Based on the above analysis, what other distribution should I test?
105635
I ran a linear regression of acceptance into college against SAT scores and family / ethnic background. The data are fictional. This is a follow-up on a prior question, already answered. The question focuses in the gathering and interpretation of odds ratios when leaving the SAT scores aside for simplicity.   The variables are Accepted (0 or 1) and Background ("red" or "blue"). I set up the data so that people of "red" background were more likely to get in:  fit &lt;- glm(Accepted~Background, data=dat, family="binomial") exp(cbind(Odds_Ratio_RedvBlue=coef(fit), confint(fit)))                          Odds_Ratio_RedvBlue             2.5 %       97.5 % (Intercept)             0.7088608                     0.5553459   0.9017961 Backgroundred           2.4480042                     1.7397640   3.4595454   Questions:   Is 0.7 the odd ratio of a person of "blue" background being accepted? I'm asking this because I also get 0.7 for "Backgroundblue" if instead I run the following code:  fit &lt;- glm(Accepted~Background-1, data=dat, family="binomial") exp(cbind(OR=coef(fit), confint(fit)))  Shouldn't the odds ratio of "red" being accepted ($\rm Accepted/Red:Accepted/Blue$) just the reciprocal: ($\rm OddsBlue = 1 / OddsRed$)?
60706
What conclusions can we draw if $p&gt;\alpha$? Does not rejecting the $H_0$ mean anything?
98893
I have a trained logistic regression model that I am applying to a testing data set. The dependent variable is binary (boolean). For each sample in the testing data set, I apply the logistic regression model to generates a % probability that the dependent variable will be true. Then I record whether the acutal value was true or false. I'm trying to calculate an $R^2$ or Adjusted $R^2$ figure as in a linear regression model.  This gives me a record for each sample in the testing set like:  prob_value_is_true         acutal_value    .34                          0    .45                          1    .11                          0    .84                          0     ....                        ....             I am wondering how to test the accuracy of the model. My first attempt was to use a contingency table and say "if prob_value_is_true > 0.80, guess that the actual value is true" and then measure the ratio of correct to incorrect classifications. But I don't like that, because it feels more like I'm just evaluating the 0.80 as a boundary, not the accuracy of the model as a whole and at all prob_value_is_true values.  Then I tried to just look at each prob_value_is_true discrete value, as an example, looking at all samples where prob_value_is_true=0.34 and measuring the % of those samples where the acutal value is true (in this case, perfect accuracy would be if the % of samples that was true = 34%). I might create a model accuracy score by summing the difference at each discrete value of prob_value_is_true. But sample sizes are a huge concern here, especially for the extremes (nearing 0% or 100%), such that the averages of the acutal values are not accurate, so using them to measure the model accuracy doesn't seem right.  I even tried creating huge ranges to ensure sufficient sample sizes (0-.25, .25-.50, .50-.75, .75-1.0), but how to measure "goodness" of that % of actual value stumps me. Say all samples where prob_value_is_true is between 0.25 and 0.50 have an average acutal_value of 0.45. Is that good since its in the range? Bad since its not near 37.5% (the center of the range)?  So I'm stuck at what seems like should be an easy question, and hoping someone can point me to a resource or method to calculate an accuracy stastic for a logistic regression model.
69288
I have trouble with the following problem:     We assume that X has a Exp(λ)-distribution with a unknowns value of λ.   We test H0: λ=0.2 versus H1: λ &lt; 0.2. We use X as test statistic.  a.   Calculate the critical area for alpha = 0.05 b. The measured X=10,   calculate the P-value   ans: [14.98, infinity) and p = 0.135    I know that the critical area is the collection of values for which H0 is rejected in favor of H1. In the case of a normal distribution I know how to calculate the critical region. Depending on the kind of alternate hypothesis and the significance level I can caluclate the Z-value. With the Z-value I can calculate the X.   Unfortunately I don't know how to do this with an exponential distribution. I have issues with visualising it, since a different λ gives a different curve.   Can I please get some feedback?  Thanks,  Ter
105521
In R, the function chisq.test has an option simulate.p.value. In the help page little explanation is provided but it reports to .   Could you please help me out by offering me a for dummy explanation of how p.values are being estimated when simulate.p.value is set to TRUE?  I have basic understanding of Monte Carlo procedures, there is no need to give a big intro on this subject.
59227
While formally deriving the confidence interval of an estimate, I ended up with a formula that resembles very closely the way $p$-value is computed.  Thus the question: are they formally equivalent? I.e. is rejecting an hypotheses $H_0 = 0$ with a critical value $\alpha$ equivalent to $0$ not belonging to the confidence interval with critical value $\alpha$?
33635
I have a problem with the interpretation of a test result in which the p-value is 8.6e-28. How should it be interpreted? What is the e for?  Specifically, I used White's Test for the homoscedasticity assumption of a linear regression:  White's general test statistic :  150.0535  Chi-sq( 9)  P-value =  8.6e-28