Given probability density function  $$p(x|\phi)=\begin{cases}\dfrac{\phi}{x^2}&amp;\phi&lt;x&lt;\infty\\0&amp;\text{others}\end{cases}.$$  I want to find an estimation for the $\phi$ parameter with the maximum likelihood estimator.  First, I find the likelihood function as below $$L(\phi)=\dfrac{\phi}{{x_1}^2}\dfrac{\phi}{{x_2}^2}\ldots\dfrac{\phi}{{x_n}^2}=\dfrac{\phi^n}{\prod\limits_{i=1}^n{x_i}^2}.$$  Then I determine the log likelihood $\ln L(\phi)$,  $$\ln L(\phi)=n\ln \phi-2\ln\prod\limits_{i=1}^n{x_i}.$$  Now, I try to find the maximum of the log likelihood,  $$\dfrac{d}{d\phi} \ln L(\phi)=\dfrac{n}{\phi}=0.$$  Then I have $n=0$.     Problem: I can't find the $\hat\phi$. How to find the parameter   estimation of $\phi$ using maximum likelihood estimator?   Please help me. Thanks before.
123839
I am trying to get the maximum likelihood estimate for the parameter $p$. The distribution is the following:   $$ f(x\mid p) = \begin{cases} \frac{p}{x^2} &amp;\text{for} \ p\leq x &lt; \infty  \ \\ 0 &amp;\text{if not} \end{cases} $$  The sample has size $n$.  The problem is, when I try to estimate it by the procedure I know, I would have to estimate the likelihood function and obtain the derivative of the log-likelihood. We'd have:   $$ L(p; x) = \frac{p^n}{\prod_{i=1}^{n} x^2_i}$$  $$ \ln L(p;x) = n \ln(p) - \sum_{i=1}^{n} \ln(x_i^2) $$   For the derivative:  $$ l'(p;x) = \frac{n}{p} = 0$$   And I am stuck because it has no solution for $p$. How do I evaluate this?  Thanks!   EDIT:  So in this case I can use the indicator variable to write:   $$ L(p;x) = \frac{p^n}{\prod_{i=1}^{n}x_i^2} I_{(x_i \geq p)}$$   for $i = 1,2, \ldots n$ in the indicator variable. So the "closest" non-null value of $x \in X$ to $p$ is $min(X_1, \ldots X_n)$. Is that the point?
65089
Hi I'm solving some exercise problems in my text : "A Course in Mathematical Statistics".  I'm in the chapter "Point estimation" now, and I want to find a UMVUE of $\theta$ where $X_1 ,...,X_n$ are i.i.d random variables with the p.d.f $f(x; \theta)=\theta e^{-\theta x}, x\gt0$.   I know that $E(X_i)=1/\theta,$ for each $i$, and also have that $\bar{X}$ (or equivalently $\sum_1^n X_i$) is a complete sufficient statistic for $\theta$. But I cannot go any further here. Somebody can help me?
61359
I believe that the MLE of parameter $p$ in the geometric distribution, $\hat p = 1/(\bar x +1)$, is an unbiased estimator for $p$ and would like to prove it. So far, I have:  $E[\bar x + 1] = E[\bar x] + 1 = \frac{1}{p} - 1 + 1 = \frac{1}{p}$  I can see that the relationship is likely there, but I don't know how to work with the $(\bar x + 1)$ being in the denominator.
31178
$X_1, X_2, . . . , X_n$ are i.i.d. observations from a uniform distribution on the interval $[\theta − \frac{1}{2}, \theta + \frac{1}{2}]$.  Show that any $\theta$ between $X_{max} − \frac{1}{2}$ and $X_{min} + \frac{1}{2}$  maximizes the likelihood, and therefore, can be taken as the MLE.  I am confused...because $f(x|\theta) = \frac{1}{(\theta+\frac{1}{2}) -(\theta -\frac{1}{2})} = 1$ for $\theta-\frac{1}{2} \le x \le\theta+\frac{1}{2}$ and 0 otherwise... Isn't the likelihood function $L(x|\theta) = 1$?  The how can it be maximized between $X_{max} − \frac{1}{2}$ and $X_{min} + \frac{1}{2}$ ?
88907
This is for my hw, and if anyone can solve the first part of the question it will be great. Here is the question:  Assume a two-class problem with equal a priori class probabilities and Gaussian class-conditional densities as follows:  $$p(x\mid w_1) = {\cal N}\left(\begin{bmatrix} 0 \\ 0 \end{bmatrix},\begin{bmatrix} a &amp; c \\ c &amp; b \end{bmatrix}\right)\quad\text{and}\quad p(x\mid w_2) =  {\cal N}\left(\begin{bmatrix} d \\ e \end{bmatrix},\begin{bmatrix} 1 &amp; 0 \\ 0 &amp; 1 \end{bmatrix}\right)$$  where $ab-c^2=1$.   Find the equation of the decision boundary between these two classes in terms of the given parameters, after choosing a logarithmic discriminant function. Determine the constraints on the values of a, b, c, d and e, such that the resulting discriminant function results with a linear decision boundary.
47871
Suppose we have a random variable $X \sim f(x|\theta)$. If $\theta_0$ were the true parameter, the the likelihood function should be maximized and the derivative equal to zero. This is the basic principle behind the maximum likelihood estimator.   As I understand it, Fisher information is defined as   $$I(\theta) = \Bbb E \Bigg[\left(\frac{\partial}{\partial \theta}f(X|\theta)\right)^2\Bigg ]$$  Thus, if $\theta_0$ is the true parameter, $I(\theta) = 0$. But if it $\theta_0$ is not the true parameter, then we will have a larger amount of Fisher information.   my questions    Does Fisher information measure the "error" of a given MLE? In other words, doesn't the existence of positive Fisher information imply my MLE can't be ideal?  How does this definition of "information" differ from that used by Shannon? Why do we call it information?
83455
Assume we have $n$ distinct points $x_1,x_2,...,x_n\in \Bbb{R}$, and observations which are independent random variables $y_1,y_2,...,y_n$ depending on formula: $$ y_i=\theta_0+\theta_1x_i^2+\theta_3e^{x_i}+\varepsilon_i $$ where $\varepsilon_i$ are distributed as $N(0,\sigma^2)$. Derive the maximum likelihood estimator for $\theta_{0},\theta_{1},\theta_{2}$ .  I know how to find the MLE for a known distribution like normal or Poisson, but for this question, I don't how I can find the likelihood function, should I consider it as normal distribution? And I don't know what the mean and variance would be since I can't figure out the distribution and likelihood function. Also, should I consider $\varepsilon_i$ as a useful variable, or just treat it as an error in estimation? Thanks for any help.
80120
Find the maximum likelihood estimator for $\theta$ when $f(x)=\frac{2x}{\theta^2}, 0\leq x \leq \theta$.  This should be a really easy question but I somehow cannot seem to get the right answer. My likelihood function is $\prod\limits_{i=1}^n \frac{2x_i}{\theta^2}$ which gives me after some simplification $n\ln(2\theta^{-2}) + \sum\limits_{i=1}^n ln(x_i)$ as my log-likelihood function. Then the derivative becomes $\frac{-2n}{\theta}$. If I equate this to $0$ to obtain the maximum I obtain $\hat{\theta}\rightarrow\infty$. However, this answer does not seem plausible. What did I do wrong?
84761
Assume ${{X}_{1}}$, ${{X}_{2}}$, ${{X}_{3}}$,...,${{X}_{n}}$  are i.i.d. samples from distribution with density f, and cdf F.  Let V=min( ${{X}_{1}}$, ${{X}_{2}}$, ${{X}_{3}}$,...,${{X}_{n}}$)  To find the minimum $${B}_{v}={\{V\ge{v}\}}=\{{X}_{1}\ge{v}\}\cap\{{X}_{2}\ge{v}\}\cap...\cap\{{X}_{n}\ge{v}\}$$  So, $$P(V\ge{v})=[1-F(v)]^{n}$$  The cdf is  $$F_{v}(V)=1-[1-F(v)]^{n}$$   The density is $${f}_{v}(V)=\frac{d}{dv}[1-F(v)]^{n}=nf(v)[1-F(v)]^{n-1}$$   To find the density, why did we use $[1-F(v)]^{n}$ instead of $1-[1-F(v)]^{n}$, shouldn't we used the cdf to derive the pdf?
69991
I am learning logistic regression from The elements of statistical learning: data mining, inference, and prediction, by Trevor Hastie, Robert Tibshirani, Jerome H. Friedman.  Suppose $G$ is a random variable representing response taking class label in $\{1,\ldots,K \}$, and $X$ is a random vector representing predictor taking values in $\mathbb{R}^n$.  After modeling $P(G = k|X = x; \theta), k=1,\ldots,K, x \in \mathbb{R}^n$ by logistic model in , the parameter $\theta$ is estimated from samples $\{(x_i,k_i), i=1,\ldots,N \}$ in section :     Logistic regression models are usually ﬁt by maximum likelihood, using   the conditional likelihood of $G$ given $X$. Since $P(G|X)$ completely   specifies the conditional distribution, the multinomial distribution   is appropriate. The log likelihood for $N$ observations is  $$ℓ(θ) = \sum_{i=1}^N \log P(G = k_i|X = x_i; \theta) $$   My question is how the estimation method is maximum likelihood.   Generally speaking, MLE is to estimate parameter $\theta$ of a single distribution $p(z;\theta)$ from its i.i.d. samples $\{z_i, i=1,\ldots,N \}$ by maximizing $\log p(x_1,\ldots, x_N; \theta) = \sum_{i=1}^N \log p(x_i; \theta) $, the log of joint distribution of the samples.  The estimation for logistic regression as in the quote seems not using MLE, because:   $P(G = k|X = x_i; \theta), i=1,\ldots,N $ are $N$ different distributions, instead of just one distribution as for MLE; in $ℓ(θ) = \sum_{i=1}^N \log P(G = k|X = x_i; \theta) = \log     \prod_{i=1}^N P(G = k|X = x_i; \theta)$, I wonder how to explain that "${ (k_i|x_i), i=1,\ldots,N }$" are independent and therefore $\prod_{i=1}^N P(G = k_i|X = x_i; \theta)$ is their joint distribution, as I don't know how to understand $(k_i|x_i)$ which seems not make sense.   My guess answer is that $\{(x_i,k_i), i=1,\ldots,N \}$ are assumed to be i.i.d. samples of the single distribution for $(X,G)$. So MLE for the distribution of $(X,G)$ is to maximize $$ \log P((x_1,k_1),\ldots,(x_N,k_N))  = \log \prod_{i=1}^N P(x_i,k_i)  $$ $$ = \log \prod_{i=1}^N P(k_i | x_i) P(x_i) = \log \prod_{i=1}^N P(k_i | x_i) + \log \prod_{i=1}^N P(x_i) $$ $P(k_i | x_i)$ is parameterized by $\theta$ as $P(k_i | x_i;\theta$, while $P(x_i)$ is assumed to be independent of $\theta$, so to solve for $\theta$ by MLE for the distribution of $(X,G)$ is equivalent to maximize $\log \prod_{i=1}^N P(k_i | x_i; \theta)$. I wonder if this is a reasonable explanation, and what yours are?  Thanks and regards!
87051
I can follow the proofs in which the asymptotic normal-distribution of a maximum likelihood estimator $\tilde{\theta}_n$ is derived.   however, does this already imply that the maximum likelihood estimates are asymptotically unbiased, i.e. do we have $$E(\tilde{\theta}_n) \to \theta \text{ as } n \to \infty?$$  Since I am aware that in general it is not true that convergence in distribution implies convergence in moments, an explanation would be nice.  The result is often stated e.g.  as "...it means that the bias of the maximum likelihood estimator is equal to zero up to the order $n^{-1/2}$")  1.) Are  some kind of regularity conditions from the mle theory used to establish this result?  2.) Or is a $\sqrt{n}$-convergence of an estimator (to a normal distribution) in general already enough to establish convergence of its moments?  Note: the Wikipedia article mentions Cox, David R.; Snell, E. Joyce (1968).  , as a source where the order of the bias is derived (formula (12) or (20)).   However in this paper I can't follow the arguments 100%, since their Taylor approximation of $L'(\widehat{\beta})$ is lacking the remainder term. What is the argument used here to discard it completely?
22014
I was doing my homework and the following problem came up! We have the CDF of an exponential distribution that is shifted $L$ units where $L&gt;0$ and $x&gt;=L$. The CDF is:  $$1-e^{-\lambda(x-L)}$$  The question says that we should assume that the following data are lifetimes of electric motors, in hours, which are:  $$\begin{align*} 153.52,103.23,31.75,28.91,37.91,7.11,99.21,31.77,11.01,217.40 \end{align*}$$  Please note that the $mean$ of these numbers is: $72.182$  Now the question has two parts which I will go through one by one:  Part1: Evaluate the log likelihood for the data when $\lambda=0.02$ and $L=3.555$. Now the way I approached the problem was to take the derivative of the CDF with respect to $\lambda$ to get the PDF which is:  $$(x-L)e^{-\lambda(x-L)}$$  Then since we have $n$ observations where $n=10$, we have the following joint pdf, due to independence:  $$(x_i-L)^ne^{-\lambda(x_i-L)n}$$ which can be rewritten as the following log likelihood:  $$n\ln(x_i-L)-\lambda\sum_{i=1}^n(x_i-L)$$ Is this the correct approach? Because it would take quite a while and be pretty cumbersome to evaluate $n\ln(x_i-L)$ for every observation?  Part2: The question also asks for the ML Estimate of $L$. So assuming the log likelihood is correct, we can take the derivative with respect to $L$ and get:  $\frac{n}{x_i-L}+\lambda=0$ and solve for $L$? Is this correct? Because I am not quite sure on how I should proceed?  Thanks so much for your help! I greatly appreciate it :)
42104
Let $X_1, X_2, \ldots , X_n$ be a random sample from a distribution with the following pdf  $$f(x|\theta) =  \begin{cases} 1/(\theta_2−\theta_1), &amp;\quad\text {for}\quad \theta_1 \leq x\leq \theta_2\\ 0 &amp;\quad\text { otherwise}\quad \end{cases}$$ Suppose that $\theta_1$ and $\theta_2$ are unknown.  How would I go about writing the likelihood function for this distribution on $\theta_1$ and $\theta_2$.
29256
Inspired by my other , I would like to ask how does one find the mode of a probability density function (PDF) of a function $f(x)$?  Is there any "cook-book" procedure for this? Apparently, this task is much more difficult than it seems at first.
40192
I am a bit confused about the derivation of MLE of Uniform$(0,\theta)$.  I understand that $L(\theta)={\theta}^{-n}$ is a decreasing function and to find the MLE we want to maximize the likelihood function.  What is confusing me is that if a function is decreasing, then wouldn't the function be maximized at the smallest input rather than the largest?  Thank you in advance for your help.
48669
Let $X_1,X_2,\dots, X_n$ be a random sample form a distribution $f(x)=e^{\delta-x}$ for $x\geq\delta$. Find MLE of $\delta$.   My solution:  $$L(\hat\gamma)=\prod_{i=1}^nf(x_i\mid \delta)=\prod_{i=1}^ne^{\delta-x} =e^{n\delta}e^{\sum_{i=1}^nx_i}$$  The log likelihood is $l(\hat\delta)=n\delta-\sum_{i=1}^n x_i$. To find the $\max$ we take the 1st derivative with respect to $\delta$: $$l'(\hat\delta)=n$$ the first derivative cannot be zero. I am not sure how to proceed.
87869
I have an answer but I'm not sure if I'm doing it right: $f(x)$ is the pdf $f(x)=2e^{-2x}$, then, if the random variable $Y=\ln(X)$, then we simply substitute:  $P(Y \leq y)$=  $P(2e^{-2X} \leq y)$  $P(-4X \leq y)$ since $y=ln(x)$  $P(X \leq - \dfrac {1} {4} y)$  This is the distribution function, right?  Thanks!