Can someone please explain why/how Count encoding of categorical features improve accuracy in classification  when compared to simply label encoding them ? I found one explanation in kaggle &quot; Rare values tend to have similar counts (with values like 1 or 2), so you can classify rare values together at prediction time. Common values with large counts are unlikely to have the same exact count as other values. So, the common/important values get their own grouping. &quot; which doesn't seem convincing or I don't understand the reasoning . Can someone please explain why it performs better than label encoding ? Label encoding can also find frequent patterns to correlate with the target variable , right ?
Frequency encoding is a widely used technique in Kaggle competitions, and many times proves to be a very reasonable way of dealing with categorical features with high cardinality. I really don't understand why it works.   Does it work in very specific cases where frequencies are correlated with the target or is it more general? What's the rationale behind it?
I am working on parameter estimation and one of the estimators involves a summation of $_nC_k$ ($n$ choose $k$) expressions. For some iterations, I need to compute expressions like $_0C_1$, $_0C_2$, etc.  In general how do we compute $_nC_k$ when $n$ is less than $k$? Do we still use the formula $\frac{n!}{(n-k)!k!}$ and use the gamma function to compute the negative factorial?  Thanks! Looking for a short story about a man who is visiting a planet where you get your position/job through playing a game I read this story 8-15 years ago in German. I'm not sure but I think the book had 2 short, totally different stories in it.  The story was about a man who was sent (I think kinda forced to go) to a planet where he should play a game everyone on that planet plays. I think the goal was to beat their leader and conquer the world like this?  The better you are at the game, the higher is your position in the society. I remember that at the end he was playing against the leader and they had a long back and forth game, but I'm not sure who won.  They played the game on a different planet that had only one stripe of forest around the whole planet and a fire that goes around it. But thats only the location they were playing.  The man was chosen because he was the best player in every game if I remember correctly.  The game was a board like game where you have troops or armies that you move around. It was not an easy game everyone just could play. How to run scripts without typing the full path? I'd like to know if there is a way to run  program/shell script without typing full path:  /opt/idea/bin/idea.sh How to compare (probability) predictive ability of models developed from logistic regression? I know some well-known measures are $c$ statistic, Kolmogorov-Smirnov $D$ statistic. However, as far as I know, those statistics take into account only of the rank order of the observations, and is invariant under changing the intercept of the logistic regression model (e.g. in oversampling-correction exercise).  In my current application, I need to depend on the accuracy of the logistic regression to predict probability of event. I know only of qualitative way of assessing models for probability prediction ability, namely by plotting "QQ-plot" of the actual vs predicted probability of event:   Score the validation dataset using the developed model. Rank the observations according to the predicted probability and group into $n$ buckets according to their rank of predicted probability. (First 1/n would go to the first bucket, next 1/n would go to the next ...) Calculate the average predicted and actual probability of Event for each bucket. Create a scatter plot of Predicted vs Actual - one point for each bucket.   I am wondering:   Is the "Q-Q plot" I mentioned above a legitimate way to assess predictive performance of models developed from logistic regression? If so, where may I find more reference for that? Is there any known quantitative way to assess the probability prediction ability of this kind of model? Get table names using SELECT statement in MySQL I cannot make xmodmap run on login in Ubuntu 13.04. I have searched around, including in askubuntu and all answers I found do not work.  I have a file named .Xmodmap in my home folder which contains the desired mappings. If I run it manually it works fine, so there is no error there. In order to make it run on startup, I have tried:     Using various .xinitrc containing one of the following commands:      xmodmap .Xmodmap xmodmap ~/.Xmodmap   /usr/bin/xmodmap /home/NAME/.Xmodmap if [ -s ~/.Xmodmap ]; then xmodmap ~/.Xmodmap fi  Using .Xresources instead of .xinitrc with the same variations. Setting it as a startup application.   None of the above works in Ubuntu 13.04 (64-bit). I also saw a few ideas about globally changing the keyboard mappings, but most of them were not applicable in Ubuntu 13.04 (i.e. the corresponding files did not exist where specified) and I prefer not to touch the global settings anyway. Do conjugate priors just lead to a posterior that is a modification of the parameters of the prior? We know, for example, that the conjugate relationship between the classic beta-binomial is as follows:  \begin{align}  y &amp;∼ \mathcal{Bin}(n,\ θ)  \\  θ &amp;∼ \mathcal{Beta}(α,\ β)  \\ θ|y &amp;∼ \mathcal{Beta}(y + α,\  n − y + β) \end{align}  Notice how the posterior is just a change in parameterization compared to the prior. Is the posterior of conjugate priors just a change in parameters? Show that  $$t(n) = 1 + \sum_{ j=0}^{n-1} t(j)$$  is the same as  $$t(n) = 2^n$$  Initial condition $t(0) = 1$ I have an iPhone 6 128 GB which is not jailbroken. The current version of iOS on the phone is 8.1.3 and yesterday I wanted to update to iOS 8.2.  The download went through without any problem but while installing, it gave the error:     Unable to Install Update   An error occurred installing iOS 8.2   Tried installing after switching Off and ON and also tried Hard Reset (Holding Home button and On/off switch).  Same error displays every time.  Also the Touch ID has stopped working and I find that the Settings - Touch ID &amp; Passcode - iPhone Unlock is OFF.  When I try to switch it ON, the system displays the error     Failed   Unable to complete Touch ID setup. Please go back and try again.   Is anyone facing this problem?  I live in India. How to open terminal, split to 9 terminals and switch between them using one script? I can reproduce that every time now, I go to a site, open an article, then to the next site, open another article. The Feed page pops up for a second, the article page lays over this with some some transparent areas (see picture). The same app-version is ok on a Smartphone with Android 4.4.2. I also sent in some bug reports.    Edit  1.0.71 isn't the solution yet. I go to Meta SE, then to an article. When I want back to Meta SE again, it crashes. Much earlier than before. I already sent in a bug report for this.  Another Edit  I can describe it better now. From the questions-page of a site I go to a post. When I use the back button, everything is ok, I can go to the next site. When I try to go to the next site directly from the post, it crashes. This only happens on the Nexus 10 with Android 5.1.1, App version 1.0.71. Hope this helps.  Edit3  This is fixed in 1.0.73 now. Thanks again. Attack moving or Attack Canceling I watch streams sometimes and i see the high elo Ranged AD move between each shot. Do they gain anything with this? And if so whats a good way to practice it. Can sum of a rational number and its reciprocal be an integer? Can sum of a rational number and its reciprocal be an integer?   My brother asked me this question and I was unable to answer it.   The only trivial solutions which I could think of are $1$ and $-1$.  As to what I tried, I am afraid not much. I have never tried to solve such a question, and if someone could point me in the right direction, maybe I could complete it on my own.  Please don't misunderstand my question.   I am looking for a rational number $r$ where $r + \frac{1}{r}$ is an integer. Understanding a derivation of bias correction for the Adam optimizer I'm reading  about the Adam optimizer and went up until the bias-correction section; in the paper they estimate the bias of the moving average of the squared gradient. These are the equations they use:    Where $v_{t}$ is the moving average of gradient squared, $g_{i}$ is gradient at the $i^{th}$ iteration and $\beta_{2}$ is a moving average parameter.  I don't understand how they make any of these transitions from the first row to the second and on to the third one. Could someone explain what rules are allowing them to do so ? Fourier series identity I need to prove that $\dfrac{a \sin(bx)}{1 - 2a \cos(bx) + a^2} = \sum_{n=1}^\infty a^n \sin(nbx)$ where $|a| &lt; 1$.  It seems that this can be proved by using Euler's formula identities for $\cos(bx)$ and $\sin(bx)$ and substituting $z = e^{ibx}$. From that, I get $$ \dfrac{a \sin(bx)}{1 - 2a \cos(bx) + a^2} = \dfrac{a (z - 1/z)}{2i(1 - a(z + 1/z) + a^2)}$$ $$= \dfrac{a (z^2 -1)}{2i(z - a(z^2 +1) + za^2)}$$ $$= \dfrac{a (z^2 -1)}{-2ai(z^2 - z(1-a^2)/a + 1)}$$  I don't understand what I am supposed to do next. Suppose Alice chooses a random Prime $p$ and a random private Key $a \in \mathbb{Z}^*_p$. By accident, she also chooses a random number $g \in \mathbb{Z}^*_p$, which is not a generator of $\mathbb{Z}^*_p$ and therefore  $$\langle g\rangle \subset \mathbb{Z}^*_p$$  as opposed to $\langle g\rangle = \mathbb{Z}^*_p$, which would yield a valid key. Alice then computes $A \equiv g^a \pmod{p} $ and publishes the Tuple $(p,g,A)$ as her public key.  Bob now encrypts a message $M$ using Alices public key by computing  $$C_1 = g^b \pmod{p}$$ $$C_2 = M \cdot A^b \pmod{p}$$  Is it possible for an attacker (Eve) to distinguish a “real” ciphertext $(C_1,C_2)$ from a random ciphertext $(Z_1,Z_2) \stackrel{$}{\longleftarrow} \mathbb{Z}^*_p$ with significant advantage (say $Adv \geq \frac12$)?    Edit: I guess it should be enough to show, that the ciphertext cannot be random, if  $$\langle g \rangle \neq \langle C_1 \rangle \quad(\text{or $Z_1$, respectively})$$  But how can Eve check for this property efficiently? Can someone point me in the right direction here?    Edit: A real-world situation where this flaw occurred Determining a matrix from its characteristic polynomial Missing entry in flagging history I happened to glance upon my flagging history and I realized that there was an entry missing from the spam section in it. I had marked 6 posts as spam, out of which 4 were deemed helpful and 1 was declined. I'm not sure what happened to the remaining 1 entry and why it is missing from the history summary. Here is a snapshot of the spam section of my flagging history.    Is this behavior a  or a ?  Note: I've checked my flagging history completely and there is no spam flag which is still in the active state. Why is the sum of two random variables a convolution? Topological Circuit Simulator Does something like  exist for topological (eg. braided) circuits?   Alternatively, any ideas on how @ is getting  (or something similar)?