In a MCMC implementation of hierarchical models, with normal random effects and a Wishart prior for their covariance matrix, Gibbs sampling is typically used.  However, if we change the distribution of the random effects (e.g., to Student's-t or another one), the conjugacy is lost. In this case, what would be a suitable (i.e., easily tunable) proposal distribution for the covariance matrix of the random effects in a Metropolis-Hastings algorithm, and what should be the target acceptance rate, again 0.234?  Thanks in advance for any pointers.
123626
In a MCMC implementation of hierarchical models, with normal random effects and a Wishart prior for their covariance matrix, Gibbs sampling is typically used.  However, if we change the distribution of the random effects (e.g., to Student's-t or another one), the conjugacy is lost. In this case, what would be a suitable (i.e., easily tunable) proposal distribution for the covariance matrix of the random effects in a Metropolis-Hastings algorithm, and what should be the target acceptance rate, again 0.234?  Thanks in advance for any pointers.
48148
I was wondering which software statistical package do you guys recommend for performing Bayesian Inference.  For example, I know that you can run openBUGS or winBUGS as standalones or you can also call them from R. But R also has several of its own packages (MCMCPack, BACCO) which can do bayesian analysis.  Does anybody have any suggestions as to which bayesian statistics package in R is best or about other alternatives (Matlab or Mathematica?)  The main features I am looking to compare are performance, ease of use, stability and flexibility
123615
I want to simulate a correlation matrix which has some off-diagonal structures and also should have some hierarchical structures. For simulating correlation matrices which contain hierarchical structures, I am using . But, I don't know how to add off-diagonal structures to it. I want to simulate a correlation matrix which somewhat have off-diagonal structures as below image    An example of a hierarchical structure in the correlation matrix is below image   Does anyone have a suggestion on how to simulate such correlation matrices?
69973
My understanding is that when using a Bayesian approach to estimate parameter values:   The posterior distribution is the combination of the prior distribution and the likelihood distribution. We simulate this by generating a sample from the posterior distribution (e.g., using a Metropolis-Hasting algorithm to generate values, and accept them if they are above a certain threshold of probability to belong to the posterior distribution). Once we have generated this sample, we use it to approximate the posterior distribution, and things like its mean.   But, I feel like I must be misunderstanding something. It sounds like we have a posterior distribution and then sample from it, and then use that sample as an approximation of the posterior distribution. But if we have the posterior distribution to begin with why do we need to sample from it to approximate it?
90072
I have a set of data with an ordinal response ranging from 1-5 (worst to best) and a categorical predictor with five unordered levels. The experiment is a language experiment whereby subjects are asked to rate different sentence types. In the literature it seems that people fit lmer() most of the time by scaling the ordinal response per subject (i.e. taking the mean response and standard deviation of that subject and dividing the every single response by it.). On the other hand, it seems a reasonable approach to use ordered linear mixed effects probit models or ordered linear mixed effects logit models as implemented in e.g. MCMCglmm() or clmm().  (1) I find myself asking what the best method would be. When is one method to be prefered over the other in an experiment as I have sketched above?  (2) Why do people prefer fitting an lmer() model with a scaled response? What is the advantage?  (3) If it is advisable to use an ordered probit or logit model, how do I decide between an ordered probit or ordered logit model?  Thanks for any help!
102052
I am looking for uninformative priors for beta distribution to work with a binomial process (Hit/Miss). At first I thought about using $\alpha=1, \beta=1$ that generate an uniform PDF, or Jeffrey prior $\alpha=0.5, \beta=0.5$. But I am actually looking for priors that have the minimum effect on posterior results, and then I thought about using an improper prior of $\alpha=0, \beta=0$. The problem here is that my posterior distribution only works if I have at least one hit and one miss. To overcome this I then thought about using a very small constant, like $\alpha=0.0001, \beta=0.0001$, just to assure that posterior $\alpha$ and $\beta$ will be $&gt;0$.  Does anyone knows if this approach is acceptable? I see numerical effects of changing these prior, but someone could give me a sort of interpretation of putting small constants like this as priors?
46319
I have been trying to derive the posterior distribution in the case of weighted Bayesian regression in the case of multivariate normal distribution for a few days and have been stuck. I am not sure if this is due to lack of my linear algebra skills or if I have done something really wrong.  In my case, the distributional assumptions are:  $$ y = N (B x, W^{-1} \nabla) $$  The parameters $B$ have a normal prior.   $$ P(B) = N (B_0, \Sigma_{B}) $$  The conditional posterior distribution for $B$ is proportional to $P(y|B) P(B)$. This is proportional to:  $$ \exp \big((B - B_0)^T \Sigma (B - B_0) + (y - Bx)^T \nabla (y - Bx)\big) $$  I have seen the textbooks mention something called completing the square which as far as I can tell equates to matching like terms. Even though I could work out for scalar variance terms, I am having trouble figuring out how to deal with the precision matrices and how this can be decoupled.   [EDIT]:  So, I tried to make a run with Glen_b's suggestion and came this far:  Original expression without the exponential (I will simplify notation slightly):  $$ (B - B_0)^T \Sigma (B - B_0) + ( y - Bx)^T \nabla (y - Bx) $$ Expanding:  $$ B^T\Sigma B - 2 B_0^T \Sigma B + B_0^T \Sigma B_0 + y^T \nabla y - 2 y^T \nabla B x + (Bx)^T \nabla Bx $$  $$ = B^T(\Sigma + x^T \nabla x)B - 2 (B_0^T \Sigma B + y^T \nabla B x) + C $$  So I am not sure how to make further progress with the second term. I guess I have to make it in the form 2(some_term)B but not sure how that can be done.
80386
I'm not a statistician. So, please bear with my blunders if any.  Would you please explain in a simple manner how simulation is done? I know that it picks some random sample from a normal distribution and use for simulating. But, don't understand clearly.
20533
I'm interested in finding out a method for generating correlated, non-normal data. So ideally some sort of distribution that takes in a covariance (or correlation) matrix as a parameter and generates data that approximates it. But here's the catch: the method I'm trying to find should have the flexibility to also control its multivariate skewness and / or kurtosis.   I'm familiar Fleishman's method and the use of the power method of normal variates, but I believe most of those extensions only allow the user for certain combinations of marginal skewness and kurtosis, leaving the multivariate skewness / kurtosis just out there. What I was wondering is if there is a method that helps specify the multivariate skewness and / or kurtosis, alongside with some correlation / covariance structure.   About a year ago I took a seminar on copula distributions and I remember the professor casually mentioning that through the use of vine copulas, one could generate data which is, say, symmetric in each one of its 1-D marginals but jointly skewed and vice-versa. Or, even further, that any lower-dimensional margins could have some skewness or kurtosis while keeping the highest dimensions symmetric (or not). I was marveled by the idea that such flexibility could exist I've been trying to find some sort of article or conference paper that describes said method but I have been unsuccessful :(. It doesn't have to be through the use of copulas, I'm open to anything that works.  Edit: I have added some R code to try to show what I mean. So far I am only well-acquainted with Mardia's definition of multivariate skewness and kurtosis. When I first approached my problem I naively thought that if I used a symmetric copula (Gaussian in this case) with skewed marginals (beta, in this example), univariate tests on the marginals would yield significance but Mardia's test for multivarite skewness/kurtosis would be non-significant. I tried that and it didn't come out as I had expected:  library(copula) library(psych) set.seed(101)  cop1 &lt;- {mvdc(normalCopula(c(0.5), dim=2, dispstr="un"),              c("beta", "beta"),list(list(shape1=0.5, shape2=5),              list(shape1=0.5, shape2=5)))}              Q1 &lt;- rmvdc(cop1, 1000)             x1 &lt;- Q1[,1]             y1 &lt;- Q1[,2]   cop2 &lt;- {mvdc(normalCopula(c(0.5), dim=2, dispstr="un"),              c("norm", "norm"),list(list(mean=0, sd=1),              list(mean = 0, sd=1)))}              Q2 &lt;- rmvdc(cop2, 1000)             x2 &lt;- Q2[,1]             y2 &lt;- Q2[,2]  mardia(Q1)    Call: mardia(x = Q1)  Mardia tests of multivariate skew and kurtosis Use describe(x) the to get univariate tests n.obs = 1000   num.vars =  2  b1p =  10.33   skew =  1720.98  with probability =  0 small sample skew =  1729.6  with probability =  0 b2p =  22.59   kurtosis =  57.68  with probability =  0  mardia(Q2) Call: mardia(x = Q2)  Mardia tests of multivariate skew and kurtosis Use describe(x) the to get univariate tests n.obs = 1000   num.vars =  2  b1p =  0.01   skew =  0.92  with probability =  0.92  small sample skew =  0.92  with probability =  0.92 b2p =  7.8   kurtosis =  -0.79  with probability =  0.43   Upon inspecting the contours for 'cop1' VS 'cop2' as well as the empirical bivariate density plots, I can also see that none of them look symmetric at all. That's when I realized this is probably a little more complicated than I thought.  I know that Mardia's is not the only definition of multivariate skewness/kurtosis, so I'm not limiting myself to finding a method that only satisfies Mardia's definitions.  thank you!
56210
I know that one of the advantages of mixed models is that they allow to specify variance-covariance matrix for the data (compound symmetry, autoregressive, unstructured, etc.) However, lmer function in R does not allow for easy specification of this matrix. Does anyone know what structure lmer uses by default and why there is no way to easily specify it?
66536
Background  I am designing a Monte Carlo simulation that combines the outputs of series of models, and I want to be sure that the simulation will allow me to make reasonable claims about the probability of the simulated outcome and the precision of that probability estimate.  The simulation will find the probability that a jury drawn from a specified community will convict a certain defendant. These are the steps of the simulation:   Using existing data, generate a logistic probability model (M) by regressing “juror first ballot vote” on demographic predictors. Use Monte Carlo methods to simulate 1,000 versions of M (i.e., 1000 versions of the coefficients for the model parameters). Select one of the 1,000 versions of the model (Mi). Empanel 1,000 juries by randomly selecting 1,000 sets of 12 “jurors” from a “community” (C) of individuals with specified demographic characteristic distributions. Deterministically calculate the probability of a first ballot guilty vote for each juror using Mi. Render each "juror’s" probable vote into a determinate vote (based on whether it is greater or less than randomly selected value between 0-1). Determine each "jury’s" “final vote” by using a model (derived from empirical data) of the probability a jury will convict, conditional on the proportion of jurors voting for conviction on the first ballot. Store the proportion of guilty verdicts for the 1000 juries (PGi). Repeat steps 3-8 for each of the 1,000 simulated versions of M. Calculate the mean value of PG and report that as the point estimate of the probability of conviction in C. Identify the 2.5 &amp; 97.5 percentile values for PG and report that as 0.95 confidence interval.   I am currently using 1,000 jurors and 1,000 juries on the theory that 1,000 random draws from a probability distribution&mdash;demographic characteristics of C or versions of M&mdash;will fill out that distribution.   Questions  Will this allow me to accurately determine the precision of my estimate? If so, how many juries do I need to empanel for each PGi calculation to cover C's probability distribution (so I avoid selection bias); may I use fewer than 1,000?  Thank you so much for any help!
412
I'm currently estimating parameters of a model defined by several ordinary differential equations (ODEs). I try this with a bayesian approach by approximating the posterior distribution of the parameters given some data using Markov Chain Monte Carlo (MCMC).   A MCMC sampler generates a chain of parameter values where it uses the (unnormalized) posterior probability of a certain parameter value to decide (stochastically) whether it will add that value to the chain or add the previous value again. But, it seems to be the practice that the actual posterior probabilities do not need to be saved, rather is a n-dimensional histogram of the resulting parameter values generated and summary statistics like highest density regions (HDRs) of a parameter posterior ditribution is calculated from this histogram. At least that's what I think i learned from .  My Question: Wouldn't it be more straightforward to save the posterior probabilities of the sampled parameter values along with these and approximate the posterior distribution from these values and not from the frequencies of parameter values in the MCMC chain? The problem of the burn-in phase would not arise as the sampler would initially still sample low probability regions more often than they would "deserve" by their posterior probabilities but it would not be anymore the problem of giving unduly high probability values to these.
108874
According to the , we have the scenario shown below, but how can ABC generate simulation datasets from samples of $\theta$ without knowing or evaluating the likelihood function?  For example, how would this method work for a normal $N(\mu, \sigma)$ distribution? Assuming that the parameter to estimate is $\mu$ and that $\sigma$ is known, how would it generate the simulations without evaluating $N(\mu, \sigma)$?
84589
Suppose that we have a posterior distribution $p(\theta\mid y)$ and we wish to define a transformation on $\theta$ such that $\phi = f(\theta)$. I know that generally such transformations will not affect the MLE as it is on the data space, but will result in non-invariance on the Maximum A Posteriori (MAP) estimate as it is a function of the parameter space. I am wondering if there is an example to illustrate this? Thanks.
118642
I'm looking to generate a set of 5 random variables and enforce a dependence structure between them and onto a dependent variable $Y$.  I understand how to generate correlated random variables for multivariate normal, but not when mixing different types.  Below is a little more than I need, but I'm hoping someone can give me a general way of solving this problem...   $X_1$ and $X_2$ need to be highly correlated Bernoulli variables.  $X_3$ needs to take one of 5 categorical values, call them "A"..."E".  $X_4$ needs to be normal, and negatively correlated with $X_1$, $X_2$.  $X_5$ needs to approximate test scores from $0$ to $100$ with a high skew, so gamma probably.  $X_5$ needs to be positively correlated with $X_1$, $X_2$, $X_4$.   Each of these variables must impact a "success/occurrence" Bernoulli distributed variable $Y$.  How would I begin?  I would like to enforce correlation both between the values of $X$, and also between each $X$ and $Y$.  (The categorical correlations seem particularly confusing to me.)
40102
I have been trying to understand the idea of conjugate priors in Bayesian statistics for a while but I simply don't get it. Can anyone explain the idea in the simplest possible terms, perhaps using the "Gaussian prior" as an example?
102747
While trying Gaussian Mixture Models , I found these 4 types of covariances.   'full' (each component has its own general covariance matrix), 'tied' (all components share the same general covariance matrix), 'diag' (each component has its own diagonal covariance matrix), 'spherical' (each component has its own single variance).   I googled a lot to find more details about each of these types but found very high-level descriptions (such as ) only.   Appreciate if someone can help me understand these, or at least direct me to somewhere I can read about these.