I have two Binomial distributed random variables. $$X_1\sim\text{Bi}\left(50,\theta_1\right)$$ $$X_2\sim\text{Bi}\left(50,\theta_2\right)$$ I sample from each distribution. $$X_1=x_1=15$$ $$X_2=x_2=26$$ Question  formulate the joint likelihood function using the given information  I am curious to know what a joint likelihood function is? Attempt 1 In this attempt I calculated the likelihood for each observation separately and multiplied them together. I am curious to know if I am on the right track?  First likelihood $$L\left(X_1=x_1|\theta_1,n\right)= {n\choose x_1}\theta_{1}^{x_1}(1-\theta_1)^{n-x_1} $$ $$n=50,x_1=15$$ $$L\left(X_1=15|\theta_1,n\right)= {50\choose 15}\theta_{1}^{15}(1-\theta_1)^{50-15} $$ $$L\left(X_1=15|\theta_1,n\right)= (2250829575120) \theta_{1}^{15}(1-\theta_1)^{35} $$  similarly, the second likelihood is: $$L\left(X_2=26|\theta_2,n\right)= {50\choose 26}\theta_{2}^{26}(1-\theta_2)^{50-26} $$  Is the joint likelihood their product? $$L_{\text{joint}}=L\left(X_1=15|\theta_1,n\right)\times L\left(X_2=26|\theta_2,n\right)$$  Further context This is from a question in a 2nd year undergraduate statistics course which I am struggling with.
122302
In Likelihood and All That Ben Bolker states " the joint likelihood of the whole data set is the product of the likelihoods of each individual observation". Is there a way that someone could explain to me, in simple terms, how we obtain the likelihood of an individual observation?
68054
Let $X_1,X_2,X_3 ...X_n$ be a random sample from Bernoulli   distribution with parameter $\theta$.Find UMVUE of $\theta(1-\theta)$.   I know that $T=\sum_{i=1}^{n}X_i$ is complete sufficient statistic for our paramenter $\theta$. I am trying to find out function of T which is a unique unbiased estimator of $\theta(1-\theta)$. Now $T\sim Bin(n,\theta)$  $E(T^2)-E(T)^2=V(T)$  $n\theta(1-\theta)+n^2\theta^2-n^2\theta^2=n\theta(1-\theta)$  $\implies\dfrac{1}{n}(T^2-\bar{T}^2)$ is umvue of $\theta(1-\theta)$     If I have sample size $n=10$ with observations $1,1,1,1,1,0,0,0,0,0$   obtain the value of this estimator.   Now I am stuck at this point that is $T^2$ is $T^2=\sum_{i=1}^{n}X_i^2$ or $T^2=(\sum_{i=1}^{n}X_i)^2$. Can someone help me and tell at what point I am doing things wrongly?
23640
I'm reading  where the authors are leading from a discussion of maximum likelihood estimation to Bayes' Theorem, ostensibly as an introduction for beginners.  As a likelihood example, they start with a binomial distribution:  $$p(x|n,\theta) = \binom{n}{x}\theta^x(1-\theta)^{n-x}$$  and then log both sides  $$\ell(\theta|x, n) = x \ln (\theta) + (n-x)\ln (1-\theta)$$  with the rationale that:     "Because the likelihood is only defined up to a multiplicative constant of proportionality (or an additive constant for the log-likelihood), we can rescale ... by dropping the binomial coefficient and writing the log-likelihood in place of the likelihood"   The math makes sense, but I can't understand what is meant by "the likelihood is only defined up to a multiplicative constant of proportionality" and how this allows dropping the binomial coefficient and going from $p(x|n,\theta)$ to $\ell(\theta|x,n)$.  Similar terminology has come up in other questions ( and ), but it still not clear what, practically, likelihood being defined or bringing information up to a multiplicative constant means. Is it possible to explain this in layman's terms?
118606
How do I complete the square from the point I have left off at, and is this correct so far?   I have a normal prior for $\beta$ of the form $p(\beta|\sigma^2)\sim \mathcal{N}(0,\sigma^2V)$, to get:  $p(\beta|\sigma^2)=(2\pi\sigma^2V)^\frac{p}{2}\exp[-\frac{1}{2\sigma^2}\beta^T\beta]$  where $\beta^T\beta$ is   $\sum\limits_{i=1}^p \beta_i^2$.  My likelihood has a normal distribtuion for the data points y of the form $p(y|\beta,\sigma^2)\sim\mathcal{N}(B\beta,\sigma^2I)  $  $p(y|\beta,\sigma^2)=(2\pi \sigma^2V)^\frac{n}{2}\exp[-\frac{1}{2\sigma^2}({\bf y}-{\bf B}{\bf \beta})^T({\bf y}-{\bf B}{\bf \beta})]$  (Note that $\beta$ is a matrix/vector too, \bf does not work.)  To get my posterior for $\beta$ I combined the above, took the exponential parts only, and then expanded to get:  $\exp[-\frac{1}{2\sigma^2}({\bf y}^T{\bf y}-{\bf y}^T{\bf B}\beta-\beta{\bf B}^T{\bf y}-\beta^T{\bf B}^T{\bf B}\beta)]\exp[-\frac{1}{2\sigma^2}({\bf \beta}^T{\bf B})]$.  I dropped the $({\bf y}^T{\bf y})$ term, as is not a function of $\beta$.  Putting into one expression without the exponential:  $-\frac{1}{2\sigma^2}(-{\bf y}^T{\bf B}\beta-\beta{\bf B}^T{\bf y}-\beta^T{\bf B}^T{\bf B}\beta+{\bf \beta}^T{\bf B})$.  I know I need to combine the similar terms and get into the form of the multivariate normal distribution, which is what I am aiming for, but I am unsure how to do this? I probably have to add an extra term to the expression to get it into the correct form?  Note: This is not homework, it's a project, but my Bayesian working knowledge is not good at all and so I need to understand the working out. I intend to integrate out the $\beta$ and then the $\sigma^2$ after getting into the multivariate form.
20477
Some sources say likelihood function is not conditional probability, some say it is. This is very confusing to me.  According to most sources I have seen, the likelihood of a distribution with parameter $\theta$, should be a product of probability mass functions given $n$ samples of $x_i$:  $$L(\theta) = L(x_1,x_2,...,x_n;\theta) = \prod_{i=1}^n p(x_i;\theta)$$  For example in Logistic Regression, we use an optimization algorithm to maximize the likelihood function (Maximum Likelihood Estimation) to obtain the optimal parameters and therefore the final LR model. Given the $n$ training samples, which we assume to be independent from each other, we want to maximize the product of probabilities (or the joint probability mass functions). This seems quite obvious to me.  According to , "likelihood is not a probability and it is not a conditional probability". It also mentioned, "likelihood is a conditional probability only in Bayesian understanding of likelihood, i.e., if you assume that $\theta$ is a random variable."  I read about the different perspectives of treating a learning problem between frequentist and Bayesian.   According to a source, for Bayesian inference, we have a priori $P(\theta)$, likelihood $P(X|\theta)$, and we want to obtain the posterior $P(\theta|X)$, using Bayesian theorem:  $$P(\theta|X)=\dfrac{P(X|\theta) \times P(\theta)}{P(X)}$$   I'm not familiar with Bayesian Inference. How come $P(X|\theta)$ which is the distribution of the observed data conditional on its parameters, is also termed the likelihood? In , it says sometimes it is written $L(\theta|X)=p(X|\theta)$. What does this mean?  is there a difference between Frequentist and Bayesian's definitions on likelihood??  Thanks.    EDIT:  There are different ways of interpreting Bayes' theorem - Bayesian interpretation and Frequentist interpretation (See: ).
37318
I am trying to solve this complicated Generalised Likelihood Ratio Test problem for 2 samples as part of my independent study course. However, the book that I am using does not talk about 2 samples at all; all the examples provided are 1 sample questions.  The question:      Assume that $X_1, ...,X_m$ is a random sample from $\theta_1e^{-\theta_1x}I_{(0,\infty)}(x)$ and $Y_1, ...,Y_n$ is a random sample from $\theta_2e^{-\theta_2y}I_{(0,\infty)}(y)$. If we assume the samples are independent. What is the GLR for testing   $H_0:\theta_1=\theta_2$  vs. $H_0:\theta_1\ne\theta_2$?   So far I just got to $$\Lambda=\frac{L(\theta,\theta)}{L(\theta_1,\theta_2)}=\frac{\theta^{m+n}\exp[-\theta\sum x_i+\sum y_i]}{\theta_1^m\exp[-\theta_1\sum x_i]\theta_2^n \exp[-\theta_2\sum y_i]}$$  However, I don't know what I should do for the next step. They say it should be an F test but I just can't see it.   I appreciate your time and help!
59756
Say we have $f(x , \theta) = \frac{1}{2}e^{-|x-\theta|}$ Lets assume for simplicity, we only have 1 sample. We find that the log-likelihood for this distribution is:  $$ l(\theta , x) = -log(2) + (\theta - x) I_{(x &lt; \theta)}+(x-\theta)I_{(x \geq \theta)}. $$  Now in general, Call the score function:  $$ S(\theta , X) = Dl = \frac{Dg_{\theta}}{g_{\theta}} $$ by noticing the log-likelihood is expressed as :  $$ l(\theta, X) = log (g_{\theta}) $$  Where g is the likelihood and assume $g$ is 3 times differentiable. Furthermore, we find :  $$ DS = \frac{D^2g_\theta}{g_\theta}- SS'. $$ Which finally allows us to obtain that :  $$ var_\theta S = -E_\theta DS $$ by noticing that $E_\theta S = 0$  Now in this case:  $$ Dl = \frac{dl}{d\theta} = I_{(x &lt; \theta)} - I_{(x \geq \theta)} $$  So indeed  $$ E_\theta S = P{(x &lt; \theta)} - P{(x \geq \theta)} = 0 $$ And then $\frac{dS}{d\theta} = 0$ so $-E_\theta \frac{dS}{d\theta} = 0$  Now computing  $$ ES^2 = E( [I_{(x &lt; \theta)} - I_{(x \geq \theta)}]^2) = E(I_{(x &lt; \theta)} + I_{(x \geq \theta)}) = 1. $$ This does not match the case above, however here do not have differentiability. Is what I did correct, or plain fiction?
26986
In general we maximize a function   $$ L(\theta; x_1, \ldots, x_n) =  \prod_{i=1}^n f(x_i \mid \theta) $$  where $f$ is probability density function if the underlying distribution is continuous, and a probability mass function (with summation instead of product) if the distribution is discrete.  How do we specify the likelihood function if the underlying distribution is a mixture between a continuous and a discrete distribution, with the weights on each depending on $\theta$ ?
119933
Find the MLE of $p$ where $f(y;p)=2p^2y^{-3}$.   Attempt:  Method: find the likelihood function, differentiate with respect to $p$ then set to zero and solve for $p$.  $L(p;y)=\prod\limits_{i=1}^n [2p^2y^{-3}_i] =\prod\limits_{i=1}^n[2] \prod\limits_{i=1}^n[p^2] \prod\limits_{i=1}^n [y^{-3}_i] = 2^n p^{2n} \prod\limits_{i=1}^n [y^{-3}_i]$  Differentiating the log likelihood gives $2n/p$ (i think) and the $y_i$ all disappear. So what happens when you set this to zero? Because if $0=2n/p$ then surely $0=2n$ and $n=0$.  Have I misstated $L(p;y)$? Or is something going wrong with my differentiation? (Or both?)
54467
I am trying to find the MLE of $\theta=(\theta_1,\theta_2)$ in a random sample $\{X\}_{i=1}^n$ with the following pdf $$f(x\mid\theta)= \begin{cases} (\theta_1+\theta_2)^{-1}\exp\left(\frac{-x}{\theta_1}\right) &amp;,  x&gt;0\\  (\theta_1+\theta_2)^{-1}\exp\left(\frac{x}{\theta_2}\right) &amp;,  x\le0\\ \end{cases} $$ If I let $\bar{X}_1$ be the average of the $n_1$ values where $X_1&gt;0$ and $\bar{X}_2$ the average of $n_2$ values where $X_i\le 0$ and $n_1+n_2=n$ Then the likelihood function is: $$L(\theta\mid  X)=\left(\frac 1 {\theta_1+\theta_2}\right)^n\exp\left(\frac{-n_1\bar{X}_1}{\theta_1}+\frac{n_2\bar{X}_2}{\theta_2}\right)$$ but I am having trouble maximizing this function.
31178
$X_1, X_2, . . . , X_n$ are i.i.d. observations from a uniform distribution on the interval $[\theta − \frac{1}{2}, \theta + \frac{1}{2}]$.  Show that any $\theta$ between $X_{max} − \frac{1}{2}$ and $X_{min} + \frac{1}{2}$  maximizes the likelihood, and therefore, can be taken as the MLE.  I am confused...because $f(x|\theta) = \frac{1}{(\theta+\frac{1}{2}) -(\theta -\frac{1}{2})} = 1$ for $\theta-\frac{1}{2} \le x \le\theta+\frac{1}{2}$ and 0 otherwise... Isn't the likelihood function $L(x|\theta) = 1$?  The how can it be maximized between $X_{max} − \frac{1}{2}$ and $X_{min} + \frac{1}{2}$ ?
52202
According to Bayes' theorem, $P(y|\theta)P(\theta) = P(\theta|y)P(y)$. But according to my econometric text, it says that $P(\theta|y) \propto P(y|\theta)P(\theta)$. Why is it like this? I don't get why $P(y)$ is ignored.
4118
I've recently encountered the bivariate Poisson distribution, but I'm a little confused as to how it can be derived.  The distribution is given by:  $P(X = x, Y = y) = e^{-(\theta_{1}+\theta_{2}+\theta_{0})} \displaystyle\frac{\theta_{1}^{x}}{x!}\frac{\theta_{2}^{y}}{y!} \sum_{i=0}^{min(x,y)}\binom{x}{i}\binom{y}{i}i!\left(\frac{\theta_{0}}{\theta_{1}\theta_{2}}\right)^{i}$  From what I can gather, the $\theta_{0}$ term is a measure of correlation between $X$ and $Y$; hence, when $X$ and $Y$ are independent, $\theta_{0} = 0$ and the distribution simply becomes the product of two univariate Poisson distributions.  Bearing this in mind, my confusion is predicated on the summation term - I'm assuming this term explains the the correlation between $X$ and $Y$.  It seems to me that the summand constitutes some sort of product of binomial cumulative distribution functions where the probability of "success" is given by $\left(\frac{\theta_{0}}{\theta_{1}\theta_{2}}\right)$ and the probability of "failure"  is given by $i!^{\frac{1}{min(x,y)-i}}$, because $\left(i!^{\frac{1}{min(x,y)-i!}}\right)^{(min(x,y)-i)} = i!$, but I could be way off with this.  Could somebody provide some assistance on how this distribution can be derived? Also, if it could be included in any answer how this model might be extended to a multivariate scenario (say three or more random variables), that would be great!  (Finally, I have noted that there was a similar question posted before (), but the derivation wasn't actually explored.)
76822
Question: Let $X$ and $Y$ be two independent and identically distributed exponential random variables with parameter $\lambda&gt;0$. Compute the probability density function of $X+Y$.   My Answer: I have found the joint probability density function of $X$ and $Y$ to be $f_{X,Y}(x,y)=\lambda^2e^{-\lambda x - \lambda y}$. I then let $Z=X+Y$ and calculated $F_Z(z)=\lambda^2e^{-\lambda z}$.  I know I need to integrate $F_Z(z)$ to calculate the density function but am unsure on what the limits should be or if I found $F_Z(z)$ correctly. I was also wondering if there is a quicker way to tackle such questions or if this method is okay.
40192
I am a bit confused about the derivation of MLE of Uniform$(0,\theta)$.  I understand that $L(\theta)={\theta}^{-n}$ is a decreasing function and to find the MLE we want to maximize the likelihood function.  What is confusing me is that if a function is decreasing, then wouldn't the function be maximized at the smallest input rather than the largest?  Thank you in advance for your help.
75632
Let $Y_1$ and $Y_2$ be independent random variables with   $Y_1\sim N(1,3)$ and $Y_2 \sim N(2,5).$      If $W_1=Y_1+2Y_2$ and $W_2=4Y_1-Y_2$   what is the joint distribution of $W_1$ and $W_2$?   Is my procedure correct?  $E(W_1)=E(Y_1+2Y_2)=E(Y_1)+2E(Y_2)=1+2\cdot2=5$  $Var(W_1)=Var(Y_1+2Y_2)=Var(Y_1)+4 Var(Y_2)=3+4\cdot 5=23$  $f_{W_1}(w_1)=\frac{1}{\sqrt{2\pi}23}\epsilon^{\frac{-1}{2}(\frac{x-5}{23})^2}$  I can't conclude.
46353
Assume the standard situation, that is, let $X_1, \ldots , X_n$ be independent and identically distributed with $X_k ∼ P_\theta(x; \theta)$ , where $P_\theta(x; \theta) = 2x/\theta^2$  if $0\le x \le\theta$ and $0$ otherwise. It is required to estimate $\theta$. Show that the maximum likelihood estimator for $\theta$ is $\hat{\theta} = \max{[X_1, \ldots , X_n]}$ and then show that the cumulative distribution function of $\hat{\theta}$ is $F_\theta(z) = z^{2n}/\theta^{2n}$. Here's what i did so far: Maximum Likehood estimator: $L_x(θ) = \prod_{i = 1}^{n} P_θ(x_k)$ Here we have $P_θ(x_1,...,x_n;θ) = P_θ(x_1,θ).P_θ(x_2,θ)...P_θ(x_n,θ)$ Likehood = $L_{x,θ}(θ) =P_θ(x_1,θ).P_θ(x_2,θ)...P_θ(x_n,θ)= 2x_1/θ^2. 2x_2/θ^2.. 2x_n/θ^2 = [2^n.\prod_{i = 1}^{n}x_i]/θ^{2n}$ Log-likehood: $\sum_{i = 1}^{n}log(P_θ(x_1,...x_n;θ))= \sum_{i = 1}^{n}log(2x_i/θ^2)$ Is this correct so far? Im still not sure how to get to $\hat{θ} = max{[X_1, . . . , X_n]}$ As for the the cumulative distribution part to show $F_θ(z) = z^{2n}/θ^{2n}$: $F(z) = P(max({x_k})&lt;z) = P(x_1&lt;z).P(x_2&lt;z)..P(x_n&lt;z) = 2x_1/θ^2.2x_2/θ^2...2x_n/θ^n = 2^n.\prod_{i = 1}^{n}x_i/θ^2n$ Not sure if this is correct. Would really appreciate some help. Edit: From the answers below, we can deduce the the estimator is biased. What estimator would be unbiased? How can i find it?