I write here to ask for a suggestion about a graduate level Bayesian statistics book. I have a bachelor degree in statistics but despite having a fairly solid background on frequentist and non parametric statistics, I do not know much about Bayesian statistics.  In particular I am undecided between these two books:   Doing Bayesian Data Analysis  Statistical Rethinking    Which one is right for me?  I know R very well, so the presence of examples on R is an added value.
5730
Which is the best introductory textbook for Bayesian statistics?  One book per answer, please.
47005
The Bayesian bootstrap was introduced by  as a Bayesian analog of the original bootstrap. Given dataset $X=\{x_1, \dots, x_N\}$, instead of drawing weights $\pi_{n}$ from the discrete set $\left\{0, \frac{1}{N}, \ldots, \frac{N}{N}\right\},$ the Bayesian approach treats the vector of weights $\pi$ as unknown parameters and derives a posterior distribution for $\pi$. Rubin (1981) used an improper, non-informative prior, $\prod_{i=1}^{n} \pi_{i}^{-1},$ which when combined with the multinomial likelihood, leads to a Dirichlet(1,...,1) distribution for the posterior distribution of $\pi$. In other words, our prior is \begin{equation}     p(\boldsymbol{\pi}) =Dirichlet(\boldsymbol{\alpha}), \quad \text{with}\ \boldsymbol{\alpha} = [0,\dots,0]. \end{equation} and posterior is \begin{equation}     p(\boldsymbol{\pi}|\boldsymbol{x}) =Dirichlet(\boldsymbol{\alpha}), \quad \text{with}\ \boldsymbol{\alpha} = [1,\dots,1]. \end{equation} Now my questions are:  I was asked the following questions which I was not able to answer: How can you have a posterior distribution that a/ does not depend on data and b/ is a uniform distribution?  Are both the prior and the posterior non-informative? I understand that the posterior is a uniform distribution, which is non-informative. Also, I see that the prior is referred to as non-informative prior. Does that mean it's flat?   I believe that section 5 in  addresses these questions, but I do not comprehend that discussion. Any clarification or pointing out what I may be misunderstanding would be appreciated.  EDIT: I just noticed one more issue when computing the posterior. Let $d=\left(d_{1}, \ldots, d_{K}\right)$ be the vector of all possible distinct values of $X$, and let $\pi=\left(\pi_{1}, \cdots, \pi_{K}\right)$ be the associated vector of probabilities $$ P\left(X=d_{i} \mid \pi\right)=\pi_{i}, \quad \sum \pi_{i}=1 $$ Let $x_{1}, \ldots, x_{n}$ be an i.i.d. sample from the equation above and let $n_{i}$ be the number of $x_{j}$ equal to $d_{i}$. If we use improper prior above over the sampling weights $\pi$, we can compute the posterior over $\pi$ \begin{align*}     p(\boldsymbol{\pi}|X) &amp;\propto p(X|\boldsymbol{\pi})p(\boldsymbol{\pi})\\     &amp; \propto \prod_{i}\pi_i^{n_i}\prod_{i}\pi_{i}^{\alpha_i-1}\\     &amp; \propto \prod_{i}\pi_i^{n_i}\prod_{i}\pi_{i}^{-1}\\     &amp; \propto \prod_i\pi_i^{n_i-1}. \end{align*} How does this yield a flat Dirichlet posterior? Are we assuming $n_i=1$ for $i=1,\dots,K$? In that case, is the vector of all possible observations $d=\left(d_{1}, \ldots, d_{K}\right)$ (the original sample that we resample from) our observation?
33080
I am reading Chris Bishop's Pattern Recognition and Machine Learning textbook. I came across the term probabilistic inference several times. I have a couple of questions.   Is probabilistic inference only applicable in a graphical modelling context? What's the distinction between traditional statistical inference (p-values, confidence intervals, Bayes factors etc.) and probabilistic inference? Is this a term that's specific to the CS community or is it widely used in the statistics community as well?
98091
From :           To demonstrate the concept of Naïve Bayes Classification, consider the   example displayed in the illustration above. As indicated, the objects   can be classified as either GREEN or RED. My task is to classify new   cases as they arrive, i.e., decide to which class label they belong,   based on the currently exiting objects.      Since there are twice as many GREEN objects as RED, it is reasonable   to believe that a new case (which hasn't been observed yet) is twice   as likely to have membership GREEN rather than RED. In the Bayesian   analysis, this belief is known as the prior probability. Prior   probabilities are based on previous experience, in this case the   percentage of GREEN and RED objects, and often used to predict   outcomes before they actually happen.      Thus, we can write:            Since there is a total of 60 objects, 40 of which are GREEN and 20   RED, our prior probabilities for class membership are:             Having formulated our prior probability, we are now ready to classify   a new object (WHITE circle). Since the objects are well clustered, it   is reasonable to assume that the more GREEN (or RED) objects in the   vicinity of X, the more likely that the new cases belong to that   particular color. To measure this likelihood, we draw a circle around   X which encompasses a number (to be chosen a priori) of points   irrespective of their class labels. Then we calculate the number of   points in the circle belonging to each class label. From this we   calculate the likelihood:            From the illustration above, it is clear that Likelihood of X given   GREEN is smaller than Likelihood of X given RED, since the circle   encompasses 1 GREEN object and 3 RED ones. Thus:                  Although the prior probabilities indicate that X may belong to GREEN   (given that there are twice as many GREEN compared to RED) the   likelihood indicates otherwise; that the class membership of X is RED   (given that there are more RED objects in the vicinity of X than   GREEN). In the Bayesian analysis, the final classification is produced   by combining both sources of information, i.e., the prior and the   likelihood, to form a posterior probability using the so-called Bayes'   rule (named after Rev. Thomas Bayes 1702-1761).            Finally, we classify X as RED since its class membership achieves the   largest posterior probability.   This is where the difficulty of my maths understanding comes in.     p(Cj | x1,x2,x...,xd) is the posterior probability of class membership, i.e., the probability that X belongs to Cj but why write it like this?   Calculating the likelihood?    Posterior Probability?    I never took math, but my understanding of naive bayes is fine I think just when it comes to these decomposed methods confuses me. Could some one help with visualizing these methods and how to write the math out in an understandable way?
20659
There are several threads on this site for book recommendations on  and  but I am looking for a text on advanced statistics including, in order of priority: maximum likelihood, generalized linear models, principal component analysis, non-linear models. I've tried  by A.C. Davison but frankly I had to put it down after 2 chapters. The text is encyclopedic in its coverage and mathematical treats but, as a practitioner, I like to approach subjects by understanding the intuition first, and then delve into the mathematical background.   These are some texts that I consider outstanding for their pedagogical value. I would like to find an equivalent for the more advanced subjects I mentioned.   , D. Freedman, R. Pisani, R. Purves. , R. Hyndman et al. , T. Z. Keith , Rand R. Wilcox , Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani , Hastie, Tibshirani and Friedman (2009)
34941
I've been researching the Bayesian take on classical statistics lately. After reading about the Bayes factor I've been left wondering if power analysis is a necessity in this view of statistics. My main reason for wondering this is the Bayes factor really just appears to be a likelihood ratio. Once it's like 25:1 it seems like I can call it a night.  Am I far off? Any other reading I can do to learn more? Currently reading this book: , by W.M. Bolstad (Wiley-Interscience; 2nd ed., 2007).
10199
During the first half of 2015 I did the  (by Andrew Ng, GREAT course). And learned the basics of machine learning (linear regression, logistic regression, SVM, Neuronal Networks...)  Also I have been a developer for 10 years, so learning a new programming language would not be a problem.  Lately, I have started learning R in order to implement machine learning algorithms.  However I have realized that if I want to keep learning I will need a more formal knowledge of statistics, currently I have a non-formal knowledge of it, but so limited that, for example, I could not properly determine which of several linear models would be better (normally I tend to use R-square for it, but apparently that is not a very good idea).   So to me it seems pretty obvious that I need to learn the basics of statistics (I studied that in uni but forgot most of it), where should I learn, please note that I don't really need a fully comprehensive course, just something that within a month allows me to know enough so I can get eager and learn more :).  So far I have read about "", any other suggestion?
89556
Can someone explain to me why would anyone choose a parametric over a nonparametric statistical method for hypothesis testing or regression analysis?   In my mind, it's like going for rafting and choosing a non-water resistant watch, because you may not get it wet. Why not use the tool that works on every occasion?
23808
I'm looking for a full online course on college-introductory level statistics (and beyond). I've seen a few options on statistics.com ($600usd online courses ouch), and coursera (starts over a month from now).   I wonder if there are other good resources that could constitute a course on statistics (books seem to supplement courses well, but not replace them quite perfectly). Academic credit &amp; certification options are a plus, but not necessary.  Thoughts?
64733
I have created a bayesian model that estimates 6 parameters using rjags from R. Now i want to do some predictions based on new data in R. Can anyone help me with an example.  model {    B.m[1] &lt;- Kb # mean B[1]    B[1] ~ dnorm( B.m[1], tau_B)    for(t in 2 : 10) { # Process model       B.m[t]&lt;- max( B[t-1] + rB[t-1](1 -B[t-1]/K) - C[t-1] , 0.0)       B[t] ~ dnorm(B.m[t], tau_B) } # Observation mode       for (t in 1: 10) {          LogC_mean[t] &lt;- log( max(qsegB[t]Eff[t], 0.000001))          C[t] ~ dlnorm(LogC_mean[t], tau_c) } # Prior          b~dbeta(1, 1)          cv &lt;- 1          K.mean &lt;- 500000          K.v &lt;- pow(cvK.mean, 2)          K.tau &lt;- 1/K.v          K ~ dnorm(K.mean, K.tau)          r ~ dlnorm(0.0, 1.4)          qseg ~ dlnorm(-15.4, 1.44)          tau_c ~ dgamma(0.001,0.001)          tau_B ~ dgamma(0.01,0.01)        }     }
71957
I am in the market for a new system (probably a laptop) that would be be used primarily for Bayesian/MCMC analyses.  If I had unlimited funds I would obviously buy very high end hardware and be done with it.  Unfortunately, this is the real world and I have to make my budget stretch.  I am looking for advice about the type of system I should get particularly in terms of processor(s) and amount of RAM.  Please note that I am not looking for specific recommendations regarding RAM and processor.  I am looking for general guidance about how to financially prioritize the various aspects of the computational hardware.  I am fine with a PC or a Mac.  If it is relevant, most of my computations would involve running R and/or C.
41273
I'm trying to put together a self-directed math curriculum to prepare for learning data mining and machine learning. This is motivated by starting  on Coursera and feeling that before proceeding I needed to improve my math skills. I graduated from college a while ago so my algebra and statistics (specifically from political science/psychology classes) are rusty.  The answers in the thread  only suggest books or classes directly related to machine learning; I have already looked into some of those classes and books and do not know exactly what math subject to study (for instance: what field[s] of math address deriving an equation to "minimize a cost function"?). The other thread suggested ()   only mentions broad categories of skills needed for analyzing data. The thread  does not apply because I do not already have a degree in math; a similar thread  has an incredible list of stats books, but again, I'm looking at starting math from a rusty recollection of algebra and moving up from there.  So, for those that work in machine learning and data mining, what fields of math do you find essential to do your job? What math subjects would you suggest to prepare for data mining and machine learning, and in what order? Here is the list and order I have so far:   Algebra Pre-calculus Calculus Linear algebra Probability Statistics (many different sub-fields here, but don't know how to break them out)   As for the data mining and machine learning, through my current job I have access to records on website/app activity, customer/subscription transactions, and real estate data (both static and time-series). I'm hoping to apply the data mining and machine learning to these datasets.  Thank you!  EDIT:  For posterity's sake, I wanted to share a helpful  for Geoffrey Gordon's/Alex Smola's  class at CMU.
40102
I have been trying to understand the idea of conjugate priors in Bayesian statistics for a while but I simply don't get it. Can anyone explain the idea in the simplest possible terms, perhaps using the "Gaussian prior" as an example?
23558
To give you some background, I have a grasp on the basics of statistics and probability theory and even remember touching Bayes theorem at the university data mining course. But being a few years away from the university made my math got extremely rusty (so much for last-minute pre-exam cramming). While I remember various random basic concepts, there are a lot of gaps in my understanding of them.  What would be a good material (a book, a site, or otherwise equally accessible medium) to revise the fundamentals and go beyond basics? I'd like a book that can be actually read as a book (most statistics books are really dry and are close to being reference material, rather than a book).
94749
This question has been posted before, but I need book with specific qualifications. I do not need books for engineers, book that is centered around calculations and stuff. I need to find a book that is theoretical, proves the statements and has good presentation of the theoretical structure. I have had the book by Tenenbaum, I did not like it. I would be very very thankful if someone shared their knowledge with us about this matter.
88663
I'm a frequentist but have been exploring Bayesian techniques lately. One thing that bothers me is that I'm never completely sure that I've implemented them correctly. To be specific, I'm not talking about convergence, burn-in time, or autocorrelation. I'm talking about, 'did I code the formulas correctly?'.  What I do now is to jack up the sample size and watch for the posterior to hug the true parameter values (with simulated data). But that's completely ignoring the power of the prior and the great small sample properties that Bayesians love to talk about.   Frequentist techniques make testable predictions in the form of relative frequencies. But in the Bayesian context, we lose the relative frequency-based interpretation of probability. So how does one programmatically test a Bayesian method's software implementation?
24143
An investigator wishes to produce a combined analysis of several datasets.  In some datasets there are paired observations for treatment A and B.  In others there are unpaired A and/or B data.  I am looking for a reference for an adaptation of the t-test, or for a likelihood ratio test, for such partially paired data.  I am willing to (for now) to assume normality with equal variance and that the population means for A are the same for each study (and likewise for B).