Unstable projection in LDA space in $n<p$ situation I'm trying to classify (LDA) few samples (n=12) in a high dimensional feature space (p=24) into 3 classes.  First I reduced the dimension of my initial dataset with a PCA, keeping only the first two Eigen vectors. Update: turns out, I was actually using all 11 PCs for the LDA. Then I had a look at the projection of my n x 2 n x 11 dataset in the LDA space (1st vs 2nd Eigen vector) and I obtained the following:   I was quite happy because the LDA found a strong separation between the 3 classes.  So I tried a leave-one-out cross validation to evaluate the LDA. I trained the classifier with 11 samples and tested it with the last one, and looped around.  The problem is the classifier performs at chance level (30% success rate).  I noticed that the LDA space changes drastically between each iteration, depending on the 11 samples used to compute it. Moreover, when I project the tested sample in the corresponding LDA space, it falls quite far away from what should be its group, explaining the poor success rate.   My questions are: is it normal that such a (visually) nice separation between classes leads to such a poor classification? Is it due to the small number of samples? Is there anything I can do to improve the situation?
Does it make sense to combine PCA and LDA? Assume I have a dataset for a supervised statistical classification task, e.g., via a Bayes' classifier. This dataset consists of 20 features and I want to boil it down to 2 features via dimensionality reduction techniques such as Principal Component Analysis (PCA) and/or Linear Discriminant Analysis (LDA).  Both techniques are projecting the data onto a smaller feature subspace: with PCA, I would find the directions (components) that maximize the variance in the dataset (without considering the class labels), and with LDA I would have the components that maximize the between-class separation.  Now, I am wondering if, how, and why these techniques can be combined and if it makes sense.  For example:   transforming the dataset via PCA and projecting it onto a new 2D subspace transforming (the already PCA-transformed) dataset via LDA for max. in-class separation   or    skipping the PCA step and using the top 2 components from a LDA.   or any other combination that makes sense.
getOnClickListener() in Android views How do I effectively defeat the Wall of Flesh? Convert a string to an enum in C# Convert list of ints to one number? I have a list of integers that I would like to convert to one number like:  numList = [1, 2, 3] num = magic(numList)  print num, type(num) &gt;&gt;&gt; 123, &lt;type 'int'&gt;   What is the best way to implement the magic function?  EDIT  I did find , but it seems like there has to be a better way. "The train will leave" vs. "is going to leave" vs. "leaves" vs. "is leaving" From the grammatical point of view all are correct, just the meaning are different, please bring your clarification, thank you.        The Train will leave at 10:00 tomorrow morning.   The Train is going to leave at 10:00 tomorrow morning.   The Train leaves at 10:00 tomorrow morning.   The Train is leaving at 10:00 tomorrow morning. Using setminus notation with set elements The "correct" way to write a set without a specific element is as follows: $S \setminus \{s\}$  But in some contexts this is cumbersome to write/type or read, and it detracts from the flow of the writing. Is it acceptable to just use $S \setminus s$?  Edit: By "cumbersome" i mean they look bad when you're trying to inline formulas in LaTeX. How old is Chewbacca? I am running Ubuntu GNOME 15.10 with GNOME 3.18, and I would like to know if I can do any of the following and if so how:   Make it so that a folder is not indexed but leave its contents indexable (so that if I search for that folder's name in my system nothing will come up, but if I search for its contents, that will come up) Make it so that a folder is not indexable and nor is any of its contents (and if it contains sub-folders within it then it should recursively not allow any contents in them to be indexed nor the contents of any sub-sub-folders in the sub-folders etc, nor any of those sub-folders themselves) Make it so that individual files in a folder are not indexable, though not having any impact on other contents nor the parent-folder   So, are any of these variants possible? I believe the indexing program is called Tracker which indexes files and folders and their contents and allows for one to search for them on the system. How to rename wp-includes folder? Is there a way that we could rename wp-includes folder   I have found out there are ways to do for other folders. But how do we do it for wp-includes folder? How to horizontally center an element This is problem 3.18 from Patrick Morandi's Field and Galois Theory.     Give an example of fields $k \subseteq K \subseteq L$ and $l \subseteq L$ for which $l/k$ and $L/K$ are algebraic, $k$ is algebraically closed in $K$, and $lK = L$, but $l$ is not algebraically closed in $L$.   This question has been asked twice, namely  and , though neither had any answers, and only some inconclusive comments. Normally I wouldn't ask the question again, but after struggling with it for weeks at this point, I emailed Morandi, and he gave me some hints.  He told me that he remembers finding an example, but that he isn't sure. He told me that it would need to be over characteristic $p$ and that the extensions (at least some of them I suppose) could not be separable. Moreover, he told me that $K = k(x)(a)$, where $a$ is a root of an irreducible polynomial $f(x,y) \in k(x,y)$ (I'm not 100% what the root of a two variable polynomial is). Finally, he said that $l$ is an algebraic extension of $k$, where $f$ becomes reducible over $l$. Still however, I can't come up with anything. I was hoping that with these extra hints, someone would be able to figure it out.  He did point out that there is a chance that there is an error in the question, but I haven't managed to prove that such an example cannot exist. How do you set, clear, and toggle a single bit? Why does speed of light have to be constant? Proof by induction that $\sum_{i=1}^n \frac{i}{(i+1)!}=1- \frac{1}{(n+1)!}$ Prove via induction that $\sum_{i=1}^n \frac{i}{(i+1)!}=1- \frac{1}{(n+1)!}$ Having a very difficult time with this proof, have done pages of work but I keep ending up with 1/(k+2).  Not sure when to apply the induction hypothesis and how to get the result $1- \frac{1}{(n+2)!}$.  Please help! thanks guys, youre the greatest! If $k$ is a positive integer, how to prove that there is no solution for $n$ in positive integers except $n = 1$ for $2^n = nk + 1$ How to change color of the text in TextView on hover like in css? can my transit visa take me through munich and frankfurt? So I am in an introductory linear algebra(with proofs) class, and I came across the following theorem:  If $S$ is a subspace of a vector space $V(F)$ and is generated by $\{a_1,...,a_n\}$ then any subset $\{b_1,...,b_m\}$ of $S$ with $m&gt;n$ is linearly dependent.   The proof in the textbook is done through induction (as most proofs regarding finite sets are) but I really do not follow it and I feel there is a simpler way of doing it. A hint would be appreciated. In the  there are lots of numbers on display, including 1000, 2500 or 10000. The display is erratic though:    Same number, two displays. Both appear valid to me: 1000 or 1,000 but please, make it consistent. My eyes hurt! :) I just found , but there it's discussing the whole network while I'm asking just for that specific page. For a start. Why does `Position` by default return a list of lists of positions instead of a simple list of positions?