I've learnt that for parametrical distributions you can describe the family of statistical model with the parameters, one such example has been the uniform distribution. I just came across a text saying that the uniform distribution is &quot;inherently non-parametric&quot;. What is really the difference between parametric and non-parametric distributions? (I have already read the previous answers on this topic). edit: link  I don't have a background in statistics/mathematics/machine learning.
122180
Does a family of a distribution have a different definition for statistics than in other disciplines?  In general, a  is a set of curves, each of which is given by a function or parametrization in which one or more of the parameters is varied. Such families are used, for example, to .  For statistics, a family according to  is the result of varying the shape parameter. How then can we understand that the gamma distribution has a shape and scale parameter and only the generalized gamma distribution has, in addition, a location parameter? Does that make the family the result of varying the location parameter? According to @whuber the meaning of a family is implicitly A "parameterization" of a family is a continuous map from a subset of ℝ$^n$, with its usual topology, into the space of distributions, whose image is that family.  What, in simple language, is a family for statistical distributions?  A question about relations among of the statistical properties of distributions from the same family has already generated considerable controversy for a  so it seems worthwhile to explore the meaning.  That this is not necessarily a simple question is born out by its use in the phrase , which has nothing to do with a family of curves, but is related to changing the form of the PDF of a distribution by reparameterization not only of parameters, but also substitution of functions of independent random variables.
74342
I'm studying machine learning and every book I open I bump into chi-squared distribution, gamma-function, t-distribution, Gaussian, etc.   Every book I have opened so far only defines what the distributions are: they don't explain or give the intuition on where the specific formulas for the functions come from.   For example, why is chi-squared distribution the way it is? What is the t-distribution? What is the intuition behind the distribution? Proofs? etc.   I would like to have a clear and fundamental understanding of the most commonly used distributions so that every time later on when I see them, I truly understand what is a t-distribution, what is a Gaussian distribution and most importantly why are they the way they are.   It would be nice if the books / tutorials  can explain the concepts to a layman so that in order to understand them you don't already need to understand them x) Many books are like this, they don't fit for beginners :(
99395
How does the expected value of a continuous random variable relate to its arithmetic mean, median, etc. in a non-normal distribution (eg. skew-normal)?  I'm interested in any common/interesting distributions (eg. log-normal, simple bi/multimodal distributions, anything else weird and wonderful).  I'm looking mostly for qualitative answers, but any quantitative or formulaic answers are also welcome. I'd particularly like to see any visual representations that make it clearer.
82643
Back in April, I attended a talk at the UMD Math Department Statistics group seminar series called "To Explain or To Predict?". The talk was given by  who teaches at UMD's Smith Business School. Her talk was based on research she did for a paper titled , and a follow up working paper titled .   Dr. Shmueli's argument is that the terms predictive and explanatory in a statistical modeling context have become conflated, and that statistical literature lacks a a thorough discussion of the differences. In the paper, she contrasts both and talks about their practical implications. I encourage you to read the papers.   The questions I'd like to pose to the practitioner community are:   How do you define a predictive exercise vs an explanatory/descriptive one? It would be useful if you could talk about the specific application. Have you ever fallen into the trap of using one when meaning to use the other? I certainly have. How do you know which one to use?
17516
Let $(X_1, X_2, \ldots, X_i, \ldots, X_k)$ be $k$ independent, normally distributed random variables with means $\mu_i$ and variances $1$. Then the random variable $$ \sum_{i=1}^k X_i^2$$ is distributed according to the noncentral chi-squared distribution with  parameters: $k$   and $ \lambda=\sum_{i=1}^k \mu_i^2$.  If $(X_1', X_2', \ldots, X_i', \ldots, X_k')$ are also independent normal distributed with variances $1$, and means $\mu'_i$'s s.t. $\sum_{i=1}^k \mu_i'^2 = \lambda$, why are $$ \sum_{i=1}^k X_i'^2$$ and $$ \sum_{i=1}^k X_i^2$$ identically distributed? Thanks.
63006
The question asks us to classify all distributions in $\mathbb{R}^2$, say $u$, such that $$(xy)u(x,y) = (x^2-y^2)u(x,y)= 0.$$ I am genuinely stuck, any help is extremely appreciated.
26874
I'm trying to come up with a metric for measuring non-uniformity of a distribution for an experiment I'm running.  I have a random variable that should be uniformly distributed in most cases, and I'd like to be able to identify (and possibly measure the degree of) examples of data sets where the variable is not uniformly distributed within some margin.    An example of three data series each with 10 measurements representing frequency of the occurrence of something I'm measuring might be something like this:  a: [10% 11% 10%  9%  9% 11% 10% 10% 12%  8%] b: [10% 10% 10%  8% 10% 10%  9%  9% 12%  8%] c: [ 3%  2% 60%  2%  3%  7%  6%  5%  5%  7%]   &lt;-- non-uniform d: [98% 97% 99% 98% 98% 96% 99% 96% 99% 98%]   I'd like to be able to distinguish distributions like c from those like a and b, and measure c's deviation from a uniform distribution.  Equivalently, if there's a metric for how uniform a distribution is (std. deviation close to zero?), I can perhaps use that to distinguish ones with high variance.  However, my data may just have one or two outliers, like the c example above, and am not sure if that will be easily detectable that way.   I can hack something to do this in software, but am looking for statistical methods/approaches to justify this formally.  I took a class years ago, but stats is not my area.  This seems like something that should have a well-known approach.  Sorry if any of this is completely bone-headed.  Thanks in advance!
12694
I know very little of Probability and Statistics, and am wishing to learn.  I see the word "distribution" used all over the place in different contexts.  For example, a discrete random variable has a "probability distribution."  I know what this is. A continuous random variable has a probability density function, then for $x\in\mathbb{R}$, the integral from $-\infty$ to $x$ of the probability density function is the cumulative distribution function evaluated at $x$.   And apparently just "distribution function" is synonymous with "cumulative distribution function," at least when talking about continuous random variables (question: are they always synonyms?).  Then there are many famous distributions.  $\Gamma$ distribution $\chi^2$ distribution, etc.  But what exactly is a $\Gamma$ distribution?  Is it the cumulative distribution function of a $\Gamma$ random variable?  Or the probability density function of a $\Gamma$ random variable?  But then a frequency distribution of a finite data set appears to be a histogram.  Long story short: in Probability and Statistics, what is the definition of the word "distribution"?    I know the definition of distribution in Mathematics (an element of the dual space of the collection of test functions equipped with the inductive limit topology), but not Probability and Statistics.
113164
I had the understanding that the major difference between machine learning and statistical model is, the later "assumes" certain type of distribution of data &amp; based on that different model paradigm as well as statistical results we obtain (e.g. p-values, F-statistics, t-stat, etc.). But in case of machine learning, we don't bother about distribution of data and more interested in prediction.  When I was going through Mllib doc, I found for linear regression we are specifying a distribution. But Mllib is a machine learning package. So, I've the following questions:  1) Is my understanding between ML &amp; statistical method is wrong?  2) Is spark is using statistical modeling for linear regression and GLMs?  Thanks!  Note: There are lot of wonderful post regarding the difference between machine learning and statistical method. But this more related to spark MLLIB.
29221
A recent question () asked about the distribution of the ratio of two random variables, and the answer accepted there was a reference to Wikipedia which (in simplified and restated form) claims that if $X$ is a Gamma random variable with parameters $(n, 1)$ and $Y$ is an exponential random variable with parameter $1$, then $Y/X$ is a Pareto random variable with parameters $(1, n)$.  Presumably $X$ and $Y$ need to be independent for this result to hold.  But, Wikipedia's page on Pareto random variables doesn't seem to include a statement as to what $(1, n)$ means, though based on what it does says, a reasonable interpretation is that the Pareto random variable takes on values in $(1,\infty)$ and its complementary CDF decays away as $z^{-n}$.  My question is:  what is the intuitive explanation for the ratio $Y/X$ to have value $1$ or more?  It would seem that all positive values should occur, and indeed the event  $\{Y &lt; X\}$ should have large probability since the Gamma random variable has larger mean than the exponential random variable.  I did work out the complementary CDF of $Y/X$ and got $(1+z)^{-n}$ for $z &gt; 0$ which is not quite what Wikipedia claims.  Added Note:  Thanks to Sasha and Didier Piau for confirming my calculation that for $z &gt; 0$,  $P\{Y/X &gt; z\} = (1+z)^{-n}$ which of course implies that  $$P\{Y/X + 1 &gt; z\} = P\{Y/X &gt; z-1\} = (1 + z - 1)^{-n} = z^{-n}~ \text{for}~ z &gt; 1$$ and thus it is $Y/X + 1$ which is a Pareto random variable, not $Y/X$ as claimed by Wikipedia.  This leads to a simple answer to  a  posed by S Huntsman:  Are there any (pairs) of simple distributions that give rise to a power law ratio?     If $W$ and $X$ are the $(n+1)$-th and $n$-th arrival times in a (homogeneous) Poisson process, then $W/X$ is a Pareto $(1,n)$ random variable:  $P\{W/X &gt; a\} = a^{-n}$ for $a &gt; 1$.   I suspect that this result is quite well known in the theory  of Poisson processes but I don't have a reference for it.
88502
Question  Suppose we have a multinomial distribution with $N$ possible outcomes, with probabilities $p_1,\ldots,p_N$. We sample this $n$ times, and denote the observed frequency of the $i$th outcome as $\xi_i$. In [1] the author claims that the distribution of the $\xi_i$ in the limit of large $n$ is:  $$p(\xi_1,\ldots,\xi_N)\propto\exp\left(-\frac{n}{2}\sum_{i=1}^N\frac{(\xi_i-p_i)^2}{p_i}\right).\;\;\;\;\;(1)$$  We can see immediately that this must be an approximation, as it assigns nonzero probabilities for $\xi_1+\cdots+\xi_N&gt;1$. However we can see that these have vanishing probability in the limit $n\rightarrow\infty$. My question is how do we derive (1) from the multinomial distribution, and show that they match in the $n\rightarrow\infty$ limit?  My thoughts  My first thought would be to appeal to the central limit theorem. The multinomial distribution has mean $\mu_i=p_i$ and covariance matrix $\Sigma_{ij}=\delta_{ij}p_i-p_ip_j$, so we would expect this in the large $n$ limit to be described by a multivariate Gaussian with mean $\mu$ and covariance $\frac{1}{n}\Sigma$. However, things are complicated by the fact that the multinomial covariance is singular (since $\xi_N$ is determined by the other $\xi_i$s), and so the multivariate Gaussian is not defined.  To address this, we may try and consider only the first $\xi_1,\ldots,\xi_{N-1}$, which have a non-singular covariance matrix and hence well-defined multivariate Gaussian distribution. Let's take the Binomial distribution $N=2$. The frequency $\xi_1$, this has mean $p_1$ and variance $p_1(1-p_1)$, so this would be described the the Gaussian: $$\propto\exp\left(-\frac{n}{2}\frac{(\xi_1-p_1)^2}{p_1(1-p_1)}\right).\;\;\;\;\;(2)$$ The expression (1) gives: $$\propto\exp\left(-\frac{n}{2}\left(\frac{(\xi_1-p_1)^2}{p_1}+\frac{(\xi_2-p_2)^2}{p_2}\right)\right).\;\;\;\;\;(3)$$ If we substitute $\xi_2\rightarrow 1-\xi_1$, $p_2\rightarrow 1-p_1$ into (3), we can verify that this gives the same answer as (2). I have verified that this also works for $N=4$.   I'm sure that if I just bashed out the algebra for general $N$ we would get agreement between the central limit theorem and (1) when we restrict the latter to $\xi_1+\cdots+\xi_N=1,p_1+\cdots+p_N=1$. However, how can we start with the multinomial distribution and derive (1) as a limit which is valid everywhere? One idea would be to say that (1) goes to zero as $n\rightarrow\infty$ when you are not on that plane, however I am a bit uncomfortable with this as it goes to zero everywhere except the mean as $n\rightarrow\infty$, so I don't know if that argument is good enough.  [1] Wootters, William K. "Statistical distance and Hilbert space." Physical Review D 23.2 (1981): 357.
116557
Would you please explain me the difference between Probability distribution and Sampling distribution easily ?  Is that the difference : in probability distribution we have probability for every individual whereas in sampling distribution we get probability for statistic ?  Does sampling distribution require more than 2 observations ?
32582
What is the intuitive difference between a random variable converging in probability versus a random variable converging in distribution?  I've read numerous definitions and mathematical equations, but that does not really help. (Please keep in mind, I am undergraduate student studying econometrics.)  How can a random variable converge to a single number, but also converge to a distribution?
90539
Apparently, the solution to the  is that a uniform probability distribution over the positive real numbers doesn't exist. Can anyone explain why this is the case and what probability distributions can exist over the positive real numbers (it seems that this would be quite limited, given that such a simple distribution is impossible)?
113413
Let $X$ be a random variable that follows $Uniform(-1,2)$ distribution.   Then what will be the cdf of $X^2$?   What i did was $$P(X^2\le y)=P(X\le y^{0.5})= \int_{-1}^{y^{0.5}}(1/3)dx=(1/3)(y^{0.5}+1) $$ $0\lt y \lt 4$ But this cdf doesnt give the probability $0$ at $y=0$ also the pdf which i get doesn't integrate to 1 What is wrong can somebody explain?? Am i doing something wrong??
32777
Definition of "Consistent nearest neighbour", runs our usual KNN classifier but instead of viewing k as a hyper-parameter it always sets k = ceil[log(n)].  So far, I looked-up many references and websites and researched on how to determine if a method is between parametric or non-parametric. I came up with below definitions,     A parametric algorithm has a fixed number of parameters. In contrast, a non-parametric algorithm uses a flexible number of parameters, and the number of parameters often grows as it learns from more data.   From .  Moreover, I found,     A parametric model, we have a finite number of parameters, and in nonparametric models, the number of parameters is (potentially) infinite.    From .  My question is, why do we count it as a Non-parametric model, while    the number of parameters can increase to infinity in the mentioned    method and it will gets more complicated by increasing test data?
70944
I often find myself asking questions like, "I know this variable $x$  lies in $(0,1)$ and most of the mass lies in $(0,.20)$ and then declines continuously towards 1.  What distribution can I use to model it?"  In practice, I wind up using the same few distributions over and over again simply because I know them.  Instead, I'd like to look them up in a more systematic way.  How do I go about accessing the wealth of work that probabilitists have done developing all of these distributions?  Ideally I'd like a reference organized by properties (region of support, etc.), so I can find distributions by their characteristics and then learn more about each distribution based on the tractability of the pdf/cdf and how closely the theoretical derivation fits the problem I'm working on.  Does such a reference exist, and if not, how do you go about choosing distributions?