Does anyone know of good reference material on exploratory analysis and diagnostic plots for count data?
24698
What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable?  I'm especially interested in Poisson and negative binomial models, as well as zero-inflated and hurdle counterparts of each. Most of the sources I've found simply plot the residuals vs. fitted values without discussion of what these plots "should" look like.  Wisdom and references greatly appreciated. The back story on why I'm asking this, if it's relevant, is .  Related discussions:
102900
I have farms in my dataset and from each farm, I have a few animals (varies for each farm) to test for a disease. I want to know, if I tested these animals, what apparent prevalence will I be able to detect in each herd. I want to estimate this for each herd.   There is a formula for estimating the sample size for a given prevalence; but how could one estimate the prevalence, if the sample size is known?
74531
I'm interested in getting some books about multivariate analysis, and need your recommendations. Free books are always welcome, but if you know about some great non-free MVA book, please, state it.
29743
Could you recommend up-to-date and well-supported references on the topic of data imputation?
82643
Back in April, I attended a talk at the UMD Math Department Statistics group seminar series called "To Explain or To Predict?". The talk was given by  who teaches at UMD's Smith Business School. Her talk was based on research she did for a paper titled , and a follow up working paper titled .   Dr. Shmueli's argument is that the terms predictive and explanatory in a statistical modeling context have become conflated, and that statistical literature lacks a a thorough discussion of the differences. In the paper, she contrasts both and talks about their practical implications. I encourage you to read the papers.   The questions I'd like to pose to the practitioner community are:   How do you define a predictive exercise vs an explanatory/descriptive one? It would be useful if you could talk about the specific application. Have you ever fallen into the trap of using one when meaning to use the other? I certainly have. How do you know which one to use?
32558
I have a dataset with 11 variables and PCA (orthogonal) was done to reduce the data. Deciding on the number of components to keep it was evident for me from my knowledge about the subject and the scree plot (see below) that two principal components (PCs) were enough to explain the data and the remaining components were only less informative.   Scree plot with parallel analysis:  observed eigenvalues (green) and simulated eigenvalues based on 100 simulations (red). Scree plot suggests 3 PCs, whereas parallel test suggests only the first two PCs.      As you can see only 48% of the variance could be captured by the first two PCs.    Plotting observations on the first plane made by the first 2 PCs revealed three different clusters using hierarchical agglomerative clustering (HAC) and K-means clustering. These  3 clusters turned out to be very relevant to the problem in question and were consistent with other findings as well. So except the fact that only 48% of variance was captured everything else was tremendously fine.      One of my two reviewers said: one cannot rely much on these findings as only 48% of variance could be explained and it is less than required.      Question Is there any required value of how much variance should be captured by PCA to be valid? Is it not dependent on the domain knowledge and methodology in use? Can anybody judge on the merit of the whole analysis just based on the mere value of the explained variance?    Notes     Data are 11 variables of genes measured by a very sensitive methodology in molecular biology called Real-Time Quantitative Polymerase Chain Reaction (RT-qPCR).   Analyses were done using R.   Answers from data analysts based on their personal experience working on real-life problems in the fields of microarray analysis, chemometrics, spectometric analyses or alike are much appreciated.   Please consider supporting you answer with references as much as possible.
53689
I'm an undergraduate statistics student looking for a good treatment of clinical trials analysis. The text should cover the fundamentals of experimental design, blocking, power analysis, latin squares design, and cluster randomization designs, among other topics.   I have an undergraduate knowledge of mathematical statistics and real analysis, but if there's a fantastic text that requires a bit higher level of statistics or analysis, I can work up to it.
15149
I am preparing a paper in the field of Computer Science.  In order to report test results, we usually run a number of tests and report the average of those tests.  For each test, we generate random data.  Because of the randomness, at some points, the results may come out not as expected.  For instance, a graph may be like:   Usually, one should explain why on points 8, 11 and 12 there is a decrease on the plot. Probably, it is because of that randomness.  Not hand-crafting all the graph, but just manipulating a few points makes the graph acceptable:   Since three weeks or so, I work my ass off and try to figure out why my resulting graph looks like the first one. Sometimes I feel like yielding to temptation and just modify the raw data before I go crazy.  I believe, at this point the title became misleading, so let me make it clear:  I am not seeking an advice on data manipulation. I will not manipulate my data. However, I ask to myself "how the hell this can be detected?"  And now, I don't only ask to myself, but to whole community. How is this detected? For editors, referees out there, have you ever detected something like this?
82062
The ultimate goal is to show users, at a glance, if their data is normally distributed.    The first attempt is a kludge that plots the data in a frequency graph.  Then, the observed mean and standard deviation are used to build a "normal curve" graph.  The frequency chart is laid over the the normal-curve chart and put next to some key statistics.  The frequency chart also colors positive bins green and negative bins red.  It looks like this:   I understand the fallacy of this approach, but for now it's practical.   What is a better way to approach this issue?
42114
I need a graduate level, mathematically rigorous textbook on multivariate analysis and inference to enhance myself. I've been reading the  and doing the problems within, but I need a book with other focuses. Topics like famous distributions (Wishart, Wilks lambda, etc.), hypothesis testing, some theory on estimation (point, interval), and other more modern materials are welcome. I've checked , but the OP there was looking for something useful for his psychological analysis, which is not exactly what I want here. Currently I have the , so I would also like to hear the comments about this book and its exercises. Thank you.
57410
I was working on historical data looking for anomalous patterns that we would not expect to occur at random. I'd like to create a scheme to analyze data and markets to test for statistical significance and consistency over time.  Question : Are there some tools we could use to evaluate the nonrandomness of a database? What do we use usually to say that something is random or not (statistically)? Figure we work with change in some value over time.
23135
I find resources like the  and  incredibly useful. They obviously serve well as references but also help me to organize my thoughts on a subject and get the lay of the land.   Q: Does anything like these resources exist for machine learning methods?  I'm imagining a reference card which for each ML method would include:   General properties When the method works well When the method does poorly From which or to which other methods the method generalizes. Has it been mostly superseded? Seminal papers on the method Open problems associated with the method Computational intensity   All these things can be found with some minimal digging through textbooks I'm sure. It would just be really convenient to have them on a few pages.
122222
I've been reading Tukey's book "Exploratory Data Analysis".  Being written in 1977, the book emphasizes paper/pencil methods.  Is there a more 'modern' successor which takes into account that we can now instantaneosly plot large data sets?
50402
I've got a study which is kinda messed up by the design...Turns out I ended up with about 50 patients of which 80% have the outcome and 20% don't (binary outcome).  I've been turning in my bed for the last month trying to figure out what to do with this. The only real answer is "don't" but I have to deliver something in the next days as it's part of a master's thesis.  I have about 10 predictors of interest for the outcome (various variable types) and the research question is whether any of these predictors can predict the outcome. Some already have an established correlation in the literature while some are original hypothesis. Several are significant with a univariate logistic regression, but it doesn't look good in any other way than the actual p-value.  Now obviously I'm not going to be able to answer this research question sufficiently but if you were in my shoes, what sort of statistical analysis would you perform to relay to your supervisors?  EDIT: Link to my (anonymized) data (CSV):
92534
I wonder if there is any statistical test to "test" the significance of a bimodal distribution. I mean, How much my data meets the bimodal distribution or not? If so, is there any test in the R program?
98911
I am trying to determine the level of spatial autocorrelation in a point dataset.  The attribute I'm interested in is binary (presence/absence of a species), for which Moran's I is not appropriate.  On the other hand, Joint Count statistics, which are typically recommended for binary or categorical data, apparently are not appropriate for point data. What is an appropriate statistic to measure global and/or local spatial autocorrelation of points when the attribute of interest is binary?
102491
Can the group recommend a good introduction text/resource to applied resampling techniques? Specifically, I am interested in alternatives to classical parametric tests (e.g. t tests, ANOVA, ANCOVA) for comparing groups when assumptions such as normality are clearly violated.   An example problem type I would like to educate myself as to a better way to solve may involve something such as:  I) 2 Groups: Treatment and Control    Dependent Var: Change in account balance dollars after intervention  Covariate: Pre intervention account balance dollars.  Issue with applying ANCOVA: Many subjects will not have any change (many zeros).   II) 2 Groups: Treatment and Control  Dependent Var: new accounts added  Covariate: Pre intervention number of accounts.  *Many subjects will not have any added account (many zeros).   Can I use a bootstrap? A permutation test? This is the type of analysis I would like to apply nonparametric resampling methods to.