I would like to know that to determine Fisher information from the Likelihood model, why do we take the log of the likelihood first instead of using normal likelihood ?
I'm trying to understand at a deeper level the ubiquity of log-likelihood (and perhaps more generally log-probability) in statistics and probability theory.  Log-probabilities show up all over the place: we usually work with the log-likelihood for analysis (e.g. for maximization), the Fisher information is defined in terms of the second derivative of the log-likelihood, entropy is an expected log-probability, Kullback-Liebler divergence involves log-probabilities, the expected diviance is an expected log-likelihood, etc.    Now I appreciate the many practical and convenient reasons.  Many common and useful pdfs are from exponential families, which leads to elegantly simplified terms when log-transformed.  Sums are easier to work with than products (esp for differentiating).  Log-probs have a great floating point advantage over straight probs.  Log-transforming a pdf often converts a non-concave function into a concave function.  But what is the theoretical reason/justification/motivation for log-probs?  As an example of my perplexity, consider the Fisher information (FI).  The usual explanation for intuiting the FI is that the second derivative of the log-likelihood tells us how "peaked" the log-likehood is:  a highly peaked log-likelihood means the MLE is well-specified and we are relatively sure of its value, while a nearly flat log-likehood (low curvature) means many different parameter values are nearly as good (in terms of the log-likelihood) as the MLE, so our MLE is more uncertain.   This is all well-and-good, but isn't it more natural to just find the curvature of the likelihood function itself (NOT log-transformed)?  At first glance the emphasis on the log-transform seems arbitrary and wrong.  Surely we are more interested in the curvature of the actual likelihood function.  What was Fisher's motivation for working with the score function and the Hessian of the log-likelihood instead?  Is the answer simply that, in the end, we have nice results from the log-likelihood asymptotically? E.g., Cramer-Rao and normality of the MLE/posterior.  Or is there a deeper reason?
What is taking up so much space on my disk, beside the filesystem? I have only one disk in my computer, a 80 GB SSD. It is formatted as a single ext4 partition (no swap), and all the usual folders are installed on it (I keep lots of data on external media, but /home and all the rest is on the SSD).   Today I booted it and I got a message that the drive is full. I opened Disk Usage Analyzer to look at what takes up space. It insists that 67.8 GB of the disk are used, and that / is taking up 36.4 GB. Which leads to the question, where are the missing 30 GB, if not on the file system?!    Just to have a comparison, I added up all the sizes of all readable folders listed in Nautilus as subfolders of / (including the hidden ones). I got 20.9 GB. Trash was unreadable, but I know it has 16.2 GB, so the sum is 36.1 GB, around the same as what Disk Usage Analyzer is reporting. There were some system folders which were unreadable, like proc, but I doubt that they add up to 30 GB - else it wouldn't be possible to install Ubuntu on small disks, and I've seen it run on 2.5 GB. I think they must be the 0.3 GB difference between my calculation and the Disk Usage Analyzer's report.   So I'd like to know, what is eating up these 30 GB and how do I get it to free them?     Edit with answers to CYREX's questions   Booted from Ubuntu 10.10 64-bit live CD (same as my system).  fsck result:   root@ubuntu:~# fsck /dev/sda1 fsck from util-linux-ng 2.17.2 e2fsck 1.41.12 (17-May-2010) /dev/sda1: clean, 265956/4890624 files, 18073343/19537408 blocks  Booted from the SSD. Emptied the trash, and the 16 GB from there are now free. The 30 GB are still missing.    I'm not experienced enough to know what counts as weird in a log. Here are my messages log and my syslog since last boot. Maybe you can find something in them:       Here it got really strange. I plugged in an external 500 GB HDD. Disk Usage Analyzer overreported the available space, then showed space missing (76 GB used, but only 24 in folders).      Booted again from the LiveCD, started the Disk Usage Analyzer from it and got the same results as from the installed Ubuntu, within 1-2 GB.   Edit with answer to CodeMonk:  That would have been a nice solution, but the partition is really 80 GB - I mean 74 GB + marketing "error", anyway the whole disk is formatted. It also shows that over 50 GB are used, so where are the 30 GB if not in files and folders?    GParted also reports the correct size of the external HDD. I have a problem understanding the proof of Rencontres numbers (Derangements) Pull-up and Pull-down Resistor Usage on Input or Output MCU Pins Are pull-up/down resistors (whether internal or external) only needed for MCU INPUT pins? In contrast, an MCU pin configured as an OUTPUT "knows what level it's at" because it does the driving - a "floating" MCU OUTPUT pin tied to some input of another circuit doesn't make sense, because the state of the MCU pin can only be high or low... do I have this right? Now, upon MCU bootup or failure, it may be beneficial to have a pull-up/down tied to this "MCU output to IC input" line to ensure that the input to some IC is never floating.  Maybe I just answered my own question here... pull-up/down resistors can be used on both input and output pins, depending on application? What are the correct version numbers for C#? Create an object with dynamic property names These days, the standard way to present differential calculus is by introducing the Cauchy-Weierstrass definition of the limit. One then defines the derivative as a limit, proves results like the Leibniz and chain rules, and uses this machinery to differentiate some simple functions such as polynomials. The purpose of my question is to see what creative alternatives people can describe to this approach. The nature of the question is that there is not going to be a single best answer. I have several methods that I've collected which I'll put in as answers to my own question.  It's not reasonable to expect answers to include an entire introductory textbook treatment of differentiation, nor would anyone want to read answers that were that lengthy. A sketch is fine. Lack of rigor is fine. Well known notation and terminology can be assumed. It would be nice to develop things to the point where one can differentiate a polynomial, since that would help to illustrate how your method works and demonstrate that it's usable. For this purpose, it suffices to prove that if $n&gt;0$ is an integer, the derivative of $x^n$ equals $0$ at $0$ and equals $n$ at $1$; the result at other nonzero values of $x$ follows by scaling. Doing this for $n=2$ is fine if the generalization to $n&gt;2$ is obvious. How do I send a file with FileZilla from the command line? I have a batch file that builds an application, and then I want to upload it to an FTP server from the command line.   Considering I do the upload manually with , is there a command line for it to upload files?  I am limited to Windows. Induction problem: "if the group has at least one player who is better than Messi, then all the members of the group are better than Messi" I'm currently using  but it sometimes falls over when the RSS or Atom feed isn't well formed. Are there any other options for parsing RSS and Atom feeds with PHP? What's the best way to check if a String represents an integer in Java? I normally use the following idiom to check if a String can be converted to an integer.  public boolean isInteger( String input ) {     try {         Integer.parseInt( input );         return true;     }     catch( Exception e ) {         return false;     } }   Is it just me, or does this seem a bit hackish?  What's a better way?    See my answer (with benchmarks, based on the  by ) to see why I've reversed my position and accepted  to this problem.  I think this original code will be used by most people because it's quicker to implement, and more maintainable, but it's orders of magnitude slower when non-integer data is provided. Which site deals with career development questions for programmers? Is there any site in the network where career development questions about the programming profession can be asked ? I thought Programmer's was the site for it, but that doesn't appear to be the case. What registers to save in the ARM C calling convention? It's been a while since I last coded arm assembler and I'm a little rusty on the details. If I call a C function from arm,  I only have to worry about saving r0-r3 and lr, right?  If the C function uses any other registers, is it responsible for saving those on the stack and restoring them? In other words, the compiler would generate code to do this for C functions.   For example if I use r10 in an assembler function, I don't have to push its value on the stack, or to memory, and pop/restore it after a C call, do I?  This is for arm-eabi-gcc 4.3.0. Giving edit reputation bonus for same user for same post multiple times On Stack Overflow, I saw some users editing the same question multiple times, and getting the 2 point reputation bonus for each time.  I saw it during review of a suggested edit (some months ago, I forgot the question that's why I'm unable to paste the link here). I rejected it and gave the reason in the "Other" reason field.  Most of the time I saw that users first formats the code, and then after approval they edit the title and fix the grammar.  Why is this allowed?  My point is, if an user gained 2 reputation points for a particular edit, then why give another 2 points again for the same post? Because the second edit on same question/answer means that there were some mistakes or misses in their previous edit. Local polynomial regression: Why does the variance increase monotonically in the degree? How can I show that the variance of local polynomial regression is increasing with the degree of the polynomial (Exercise 6.3 in Elements of Statistical Learning, second edition)?  This question has been asked  but the answer just states it follows easliy.  More precisely, we consider $y_{i}=f(x_{i})+\epsilon_{i}$ with $\epsilon_{i}$ being independent with standard deviation $\sigma.$  The estimator is given by  $$ \hat{f}(x_{0})=\left(\begin{array}{ccccc} 1 &amp; x_{0} &amp; x_{0}^{2} &amp; \dots &amp; x_{0}^{d}\end{array}\right)\left(\begin{array}{c} \alpha\\ \beta_{1}\\ \vdots\\ \beta_{d} \end{array}\right) $$ for $\alpha,\beta_{1},\dots,\beta_{d}$ solving the following weighted least squares problem $$ \min\left(y_{d}-\underbrace{\left(\begin{array}{ccccc} 1 &amp; x_{1} &amp; x_{1}^{2} &amp; \dots &amp; x_{1}^{d}\\ \vdots\\ 1 &amp;  &amp;  &amp;  &amp; x_{n}^{d} \end{array}\right)}_{X}\left(\begin{array}{c} \alpha\\ \beta_{1}\\ \vdots\\ \beta_{d} \end{array}\right)\right)^{t}W\left(y-\left(\begin{array}{ccccc} 1 &amp; x_{1} &amp; x_{1}^{2} &amp; \dots &amp; x_{1}^{d}\\ \vdots\\ 1 &amp;  &amp;  &amp;  &amp; x_{n}^{d} \end{array}\right)\left(\begin{array}{c} \alpha\\ \beta_{1}\\ \vdots\\ \beta_{d} \end{array}\right)\right) $$ for $W=\text{diag}\left(K(x_{0},x_{i})\right)_{i=1\dots n}$ with $K$ being the regression kernel. The solution to the weighted least squares problem can be written as $$ \left(\begin{array}{cccc} \alpha &amp; \beta_{1} &amp; \dots &amp; \beta_{d}\end{array}\right)=\left(X^{t}WX\right)^{-1}X^{t}WY. $$ Thus, for $l(x_{0})=\left(\begin{array}{ccccc} 1 &amp; x_{0} &amp; x_{0}^{2} &amp; \dots &amp; x_{0}^{d}\end{array}\right)\left(X^{t}WX\right)^{-1}X^{t}W$ we obtain $$ \hat{f}(x_{0})=l(x_{0})Y $$ implying that  $$ \text{Var }\hat{f}(x_{0})=\sigma^{2}\left\Vert l(x_{0})\right\Vert ^{2}=\left(\begin{array}{ccccc} 1 &amp; x_{0} &amp; x_{0}^{2} &amp; \dots &amp; x_{0}^{d}\end{array}\right)\left(X^{t}WX\right)^{-1}X^{t}W^{2}X\left(X^{t}WX\right)^{-1}\left(\begin{array}{ccccc} 1 &amp; x_{0} &amp; x_{0}^{2} &amp; \dots &amp; x_{0}^{d}\end{array}\right)^{t}. $$ My approach: An induction using the formula for the inverse of a block matrix but I did not succeed.  The paper  by D. Ruppert and M. P. Wand derives an asymptotic expression for the variance for $n\rightarrow\infty$ in Theorem 4.1 but it is not clear that is increasing in the degree. Make WCF service be accessible through Internet Change password on root user and user account If I change the password for the root user is that going to automatically change the password for the user account?   I found this link for root:    How do I change the user account password? I was told that the following does not work:     Employees having children will be granted a day off.   However, this sounds better:     Having children, those employees will be granted a day off.   What is wrong with the first clause? I got this $$ f=a(x-1)(x-2)...(x-101) $$ knowing a and that $1 ,2 \ldots ,101$ are it's roots ... I need the roots of $f'$ , how should I do it ? More specifc how can I derivate a polynomial like that, is there a formula or something ? I have a dozen or so image files on my iMac's desktop.  Most of the time, after booting up my computer, it displays a thumbnail of each image (as it should).  But sometimes, the thumbnails won't show up (see screenshot below).    It appears to happen randomly.  I estimate that it happens about 20% of the time.  Why is this, and how can I prevent it from happening?  I am running OS X 10.9.4.  Thanks. Is this intended behavior (with plain XeTeX):  \input tikz \tikz\draw[dashed] node {$3T\over4$}; \bye   It renders as: !?  The fraction line is dashed, too! Is this intentional? Is there something I can do to change that? I know I can separate the fraction into its own node, but other than that?