I have a variable which seems to be a mix of two Gaussian distributions (it is bi-modal with each mode looking normally distributed).   I would like to identify anomalous samples.   So my idea is to fit a GMM. Than I would like to see which points have a very low probability of coming from that model.  Does this make sense?   If yes, I'm aware that there could be different ways to do this python. Which would you recommend?  EDIT  I don't mean tests that would reject a uni-modal distribution. I am, actually, assuming that the process generating the data is multi-modal.  More specifically:   (1) I assume that data is coming from a finite Gaussian mixture (likely to be just two)  (2) I'm thinking of fitting such a model to the data (either by assuming that it is just 2 Gaussians or by estimating the number of components from the data)  (3) Then my thinking is to test which samples do not fit that hypothesis  Does this make sense?   If yes, which python packages would you suggest for such a pipeline?
92534
I wonder if there is any statistical test to "test" the significance of a bimodal distribution. I mean, How much my data meets the bimodal distribution or not? If so, is there any test in the R program?
58601
I have two independent variables: $X$ follows from standard Gaussian distribution $N(0,\sigma^2)$; $Y$ follows from Rademacher distribution, i.e., $Y$ can be either $-1$ or $1$ with the same probability.  My question is:  For a new random variable of $Z=XY$, What is the probability density function?   Or can we have some concentration inequalities for $Z$ (e.g., show how $Z$ approaches its expectation, etc.)?
66693
I was reading this  where it had a gaussian distribution model.   I mean gaussian is given by  $P(y) = \frac{e^{-\frac{1}{2}(y -\mu)^T \Sigma^{-1}(y -\mu)}}{2\pi^{n/2}|\Sigma|^{1/2}}$  But is   $\frac{e^{-(y -\mu)^T \Sigma^{-1}(y -\mu)}}{2\pi^{n}|\Sigma|}$ Gaussian as well?  The paper used their model saying the gaussian had the above form. It is valid?
57809
I would like to find a computationally quick and sufficiently accurate way (i.e., either analytically or numerical approximation) to compute a conditional probability for a specific multi-variate normal distribution, which is defined as follows:  Let $X_1, \dots, X_n$ be n independent (univariate) normal random variables. And let $A_1, \dots, A_k$ be defined as sums of subsets of those random variables, i.e.,: $\forall i: A_i = \sum_{j \in S} X_j$, where $S \subseteq \{1,\dots, n\}$.  I am looking for the following conditional probabilities of $A_i$:  Pr[ $A_i \leq x$ | $A_j \leq t$ $\forall j \in C$ and $j \neq i$ ], where $x$ is any number smaller than $t$, and $C \subseteq \left\{ 1,...,k \right\}$ is a subset of all possible variable indices on which the probability is conditioned. So the condition is on several $A_j$'s.  To the best to my knowledge, the conditional density of a multi-variate normal density is generally not a (uni-variate) normal distribution. I tried via Monte-Carlo simulation, but approximation is not sufficiently accurate when using 500,000 samples, 2000 bins (to classify the results) for a "problem" with 50 random variables, i.e. n=50.  Analytically that doesn't seem to be trivial. I have asked quite a few Statisticians around me and so far we haven't found a solution.
67906
I have just started reading about GPs and analogous to the regular Gaussian distribution it is characterized by a mean function and the covariance function or the kernel. I was at a talk and the speaker said that the mean function is usually quite uninteresting and all the inference effort is spent on estimating the correct covariance function.   Can someone explain to me why that should be the case?
76076
We want to apply PCA to monitor our process testing data. After we plot the density of the testing data and do normality tests, we found them to be definitely not normal, but still symmetrical and long tailed.  What distribution can these be?  If they are not normal, is PCA still be applicable? If not, what other monitoring technique could be used?   We have too many of those testing data, so eyeballing them 1 by 1 is definitely not a good choice.    shapiro test   ARM_Z  RAMP_Z  DISC_Z RAMP_Z8  ARM_Z1  ARM_Z2   0.0000  0.0032  0.0000  0.0724  0.0246  0.0000   adf test   ARM_Z  RAMP_Z  DISC_Z RAMP_Z8  ARM_Z1  ARM_Z2     0.01    0.01    0.01    0.01    0.01    0.01   jarque-bera test   ARM_Z.X-squared  RAMP_Z.X-squared  DISC_Z.X-squared RAMP_Z8.X-squared  ARM_Z1.X-squared  ARM_Z2.X-squared             0.0000            0.4580            0.0003            0.8591            0.0029            0.0000    They are more peaked than corresponding fitted normal distributions.
19394
I am building a classification model (let's say logistic regression) using 2 different data sets. The data sets do not come from the same population. Let's say the data sets comes from different space-time coordinates (Asia in Fall vs North America in Winter). I have been told that simply mixing the data sets into one and applying a classification model is not right (closest thing I understood was concerns around bias). Are there any techniques or models that I may use to analyze these 2 data sets as one? Note that the variables/features all align (including the target variable, which is binary).
60830
We know if two RVs are marginally Gaussian and uncorrelated, they can be not jointly Gaussian. How about two correlated RVs which are marginally Gaussian? Are they Jointly Gaussian or no, there are occasions when they are not Jointly Gaussian?
109602
This post relates to my original question , but this time focusing on a more fundamental misunderstanding of how MCMC actually works.  When using Bayesian MCMC for parameter inference with a single model, it is usual to ignore the "evidence" integral and write:  $$P(\theta | D) \propto P(D | \theta) P(\theta)$$  For numerical reasons, it is also common to work with logarithms:  $$ln[P(\theta | D)] \propto ln[P(D | \theta)] + ln[P(\theta)]$$  I've used this successfully on a number of occasions, typically using Python and . However, for reasons explained in my , I've recently been wondering about "normalising" my likelihood by dividing it by the number of observations (I realise this probably sounds odd, but please see my other question for why I thought it was worth a try).  My likelihood assumes i.i.d. Gaussian errors, so my log-likelihood is just the sum of the individual data likelihoods. My prior is uniform and I'm currently using an improper log-prior function, which simply returns zero if the parameter values are "within range" and '-inf' otherwise.   I initially assumed that dividing by the number of data points wouldn't make any difference, because the RHS is still proportional to the log-posterior:  $$ln[P(\theta | D)] \propto \frac{1}{n}ln[P(D | \theta)] + ln[P(\theta)]$$  (My prior only ever returns 0 or '-inf', so multiplying that by $1/n$ makes no difference).  However, when I re-run my MCMC, I get very different results. As a sense check, I tried a simple OLS linear regression using the same code.   With my original (not normalised) likelihood, I get sensible results: the MCMC recovers the "true" parameters and the 95% CI broadly agrees with the Frequentist results from 'statsmodels':    However, with the "normalised" likelihood, I get very different (and obviously wrong) results:    I think I understand why this is, at least from the point of view of a simple Metropolis algorithm. The probability of accepting the proposed jump, $x_{i+1}$, compared to the current location, $x_i$, is:  $$\alpha = \frac{P(x_{i+1})}{P(x_i)}$$  Which, re-writing in terms of my log-posterior, $L$, becomes:  $$\alpha = \frac{exp(L(x_{i+1}))}{exp(L(x_i))} = exp(L(x_{i+1}) - L(x_i))$$  On the other hand, for my "normalised" version, all the likelihoods are divided by $n$, so this becomes:  $$\alpha = [exp(L(x_{i+1}) - L(x_i))]^\frac{1}{n}$$  which is different, and might explain why my uncertainty bounds suddenly become much larger. (I understand that emcee is more sophisticated than just basic Metropolis, but I guess a similar principle applies?).  My question:  What am I missing/misunderstanding to reconcile these two results, please? On the one hand, it's OK to ignore the "normalising constant" in Bayes' equation, because anything proportional to the log-posterior is sufficient. On the other hand, multiplying the log-posterior by a constant $\frac{1}{n}$ seems to have a dramatic effect on my results?  Thank you!
104338
I have a generalized linear model that adopts a Gaussian distribution and log link function. After fitting the model, I check the residuals: QQ plot, residuals vs predicted values, histogram of residuals (acknowledging that due caution is needed). Everything looks good. This seems to suggest (to me) that the choice of a Gaussian distribution was quite reasonable. Or, at least, that the residuals are consistent with the distribution I used in my model.   Q1: Would it be going too far to state that it validates my choice of distribution?  I chose a log link function because my response variable is always positive, but I'd like some sort of confirmation that it was a good choice.   Q2: Are there any tests, like checking the residuals for the choice of distribution, that can support my choice of link function? (Choosing a link function seems a bit arbitrary to me, as the only guidelines I can find are quite vague and hand-wavey, presumably for good reason.)
41586
I am trying to come up with a model by using negative binomial regression (negative binomial GLM). I have a relatively small sample size (greater than 300), and the data are not scaled. I noticed that there are two ways to measure goodness of fit - one is deviance and the other is the Pearson statistic. How can I determine which goodness-of-fit measure to use? Are there some criteria that I can take a look at in selecting the goodness-of-fit measure?
48038
I got two distribution which are not pairs. It's difficult to say that they are also from a normal distribution because they have different length.  The distributions of the vectors are, for gas: 65  65  65  65  74  74  74  74  74  77  78  81  81  83  83  83  85  85  85  85  85  87  87 89  89  90  91  91  91  91  91  91  94  94  94  95  95  95  95  98 101 101 101 102 102 102 102 102 103 103 103 103 103 104 104 104 104 104 104 106 106 106 106 107 108 108 110 110 113 113 115 115 115 118 118 118 118 119 119 121 122 122 122 125 125 125 128 128 128 128 128 129 129 134 134 134 134 134 134 137 137 137 142 145 145 148 148 148 148 150 150 150 150 150 150 150 153 153 154 154 154 158 158 161 161 161 161 161 161 161 161 164 164 168 168 168 168 168 186 188 188 192 192 194 194 197 197 231 256  while for the diesel vector distribution is:  65  91  91  93  93  93  93  94  94  95 122 128 161 161 161   I made the t.test and the wilcox.test and I got these results:  Welch Two Sample t-test  data:  diesel_type and gas_type t = -1.7104, df = 18.149, p-value = 0.1042 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -31.873874   3.256425 sample estimates: mean of x mean of y  109.0000  123.3087    and Wilcoxon test is:  data:  diesel_type and gas_type W = 843, p-value = 0.1179 alternative hypothesis: true location shift is not equal to 0   I could accept the null hypothesis in the t.test, saying that the means are equals because the p-value is over the 5%. Should I accept the null hypothesis in the Wilcoxon test too? Are the medians equals? What could I say about these results?
6395
I want to estimate parameters of Dirichlet mixture models using Gibbs sampling and I have some questions about that:   Is a mixture of Dirichlet distributions equivalent to a Dirichlet process? What is their main differences if is not? Also, if I want to estimate a single Dirichlet distribution's parameters, which distribution for parameters should be selected as priors in Bayesian framework?   In all of the papers I found an estimation of a multinomial distribution using Dirichlet priors. I need estimation of a Dirichlet distribution using multinomial priors, perhaps.   Is the posterior function also in the form of DIRICHLET(α+N) similar to the case “estimation of multinomial distribution using Dirichlet priors”? as the multiplication of probability density function for iid samples are not considered in the definition of the likelihood function. I again cannot understand why.  e.g. as expressed in this paper:   or      so thanks for your attention  my data is Hyperion (a kind of hyperspectral remote sensing imagery) and i want to perform hyperspectral unmixing using mixture of Dirichlet sources and i will apply Gibbs sampling method for parameter estimation. my data is in dimension (614*512*224) which is commonly available AVIRIS sensor data for Cuprite Nevada district and is almost 200MB. also this data is available via (). unfortunately i don't know how can i sent my data.   i just ask you to help me in statistical modelling tasks for my PHD thesis. i will be so grateful if you help me to solve my confusions in modelling.  all the best solmaz
26074
I have two independent gaussian distributions to combine and I have a doubt.  Let's say we have $X \sim N(\mu_x,\sigma^2_x)$ and $Y \sim N(\mu_y,\sigma^2_y)$. I want to mix the two variables with a 0.5 weight each, so I obtain a new variable $Z=0.5*X+0.5*Y$ with mean $\mu_z=0.5*\mu_x+0.5*\mu_y$.  Is it right to say that $\sigma^2_z=0.5^2*\sigma^2_x+0.5^2*\sigma^2_y$?  Because I clearly obtain less variance than intuitively expected when combining distributions.
94866
I'm hoping someone can help straighten out a point of confusion for me. Say I want to test whether 2 sets of regression coefficients are significantly different from each other, with the following set up:   $y_i = \alpha + \beta x_i + \epsilon_i$, with 5 independent variables. 2 groups, with roughly equal sizes $n_1, n_2$ (though this may vary) Thousands of similar regression will be done simultaneously, so some kind of multiple hypothesis correction has to be done.   One approach that was suggested to me is to use a Z-test:  $Z = \frac{b_1 - b_2}{\sqrt(SEb_1^2 + SEb_2^2)}$  Another I've seen suggested on this board is to introduce a dummy variable for grouping and rewrite the model as:  $y_i = \alpha + \beta x_i + \delta(x_ig_i) + \epsilon_i$, where $g$ is the grouping variable, coded as 0, 1.  My question is, how are these two approaches different (e.g. different assumptions made, flexibility)? Is one more appropriate than the other? I suspect this is pretty basic, but any clarification would be greatly appreciated.
54849
As a part of my studies, I’m trying to cluster co-occurrences of URLs and tags in data from Delicious. I found a promising method for this in a paper called “” (pages 6-13). It used a Separable Mixture Model (SMM, described in the paper “” pages 2-4) to model the data and an adapted EM-algorithm to fit the known data to the model.  I coded the algorithm with Java and ran it with a little piece of real data from Delicious. Unfortunately, the results did not seem correct. The results showed that each tag had equal (although varying from tag to tag) possibility to belong to each concept.  Now, while this problem could have came from me simply coding the adapted EM-algorithm wrong, I would also like to rule out the possibility of incorrectly initialized variables. This time, since I didn’t know any better way to do it, I simply initialized all the $R_{r\alpha}$ (variables that denote the possibility of co-occurrence $r$ to have raised from concept $\alpha$) to be equal, $1/K$ ($K$ being the number of concepts).  My question is two-fold. Could the flat results come from the flat variable initialization? How should I initialize the variables from the EM-algorithm in this case?
121301
I have a couple of quick questions about PCA:    Does the PCA assume that the dataset is Gaussian?   What happens when I apply a PCA to inherently non-linear data?   Given a dataset, the process is to first mean-normalize, set the variance to 1, take an SVD, reduce rank, and finally map the dataset into the new reduced-rank space. In the new space, each dimension corresponds to a "direction" of maximal variance.     But is the correlation of that dataset in the new space always zero, or is that only true for data that is inherently Gaussian?   Suppose I have two datasets, "A" and "B", where "A" corresponds to randomly sampled points taken from a Gaussian, while "B" corresponds to points randomly sampled from another distribution (say Poisson).    How does the PCA(A) compare to the PCA(B)?   By looking at the points in the new space, how would I determine that the PCA(A) corresponds to points sampled from a Gaussian, while PCA(B) corresponds to points sampled from a Poisson?   Is the correlation of the points in "A" 0?   Is the correlation of points in "B" also 0?   More importantly, am I asking the "right" question?   Should I look at the correlation, or is there another metric that I should consider?