I am trying to segment an image by first dividing it into superpixel, and than classifying those superpixel to merge them together. If I am doing it for example with a SVM or a Decision Tree, it works fairly well, but is still doing some mistakes.  Now I wanted to use an Neural Network, to do the task, but after a few iterations it almost always gets stuck in the local minima and "maps every input to the same class".   It behaves in general really weird, because sometimes the accuracy first increases, but than steadily decreases again, till it reaches one of the minima from above (I am talking about the train accuracy, not the test accuracy, so overfitting is not an explanation, I also don't use Stochastic gradient descent but use all train data for an weight update, so fluctuations cant really explain the decreasing accuracy)  That looks like this:  617/617 [==============================] - 1s - loss: 1.1174 - acc: 0.2690       Epoch 2/1000 617/617 [==============================] - 0s - loss: 0.8662 - acc: 0.8169      Epoch 3/1000 617/617 [==============================] - 0s - loss: 0.5829 - acc: 0.8136      Epoch 4/1000 617/617 [==============================] - 0s - loss: 6.3685 - acc: 0.9060      Epoch 5/1000 617/617 [==============================] - 0s - loss: 9.0494 - acc: 0.4036       Epoch 6/1000 617/617 [==============================] - 0s - loss: 4.4840 - acc: 0.1297      Epoch 7/1000 617/617 [==============================] - 0s - loss: 1.9398 - acc: 0.1167      Epoch 8/1000 617/617 [==============================] - 0s - loss: 1.8809 - acc: 0.1167    (It stays at the same loss and acc from there)  The NN I am using looks like this (even though I also tried many others):  from keras.models import Sequential from keras.layers import Dense, Dropout from keras.optimizers import SGD enc = OneHotEncoder() labelOH = enc.fit_transform(label.reshape(-1,1)).A  clfN = Sequential() clfN.add(Dense(32, input_dim=feature_array.shape[1])) clfN.add(Dense(64, activation="relu")) clfN.add(Dense(64, activation="relu")) clfN.add(Dense(32, activation="relu")) clfN.add(Dense(labelOH.shape[1]))  sgd = SGD(lr=0.0001, decay=1e-10, momentum=0.9, nesterov=True) clfN.compile(loss='categorical_crossentropy',               optimizer=sgd,               metrics=['accuracy']) clfN.summary() clfN.fit(feature_array, labelOH,  epochs=1000, batch_size=617)   What could be the cause for this behaviour, and what could I do about it?
I'm training a neural network but the training loss doesn't decrease. How can I fix this?  I'm not asking about overfitting or regularization. I'm asking about how to solve the problem where my network's performance doesn't improve on the training set.    This question is intentionally general so that other questions about how to train a neural network can be closed as a duplicate of this one, with the attitude that "if you give a man a fish you feed him for a day, but if you teach a man to fish, you can feed him for the rest of his life." See this Meta thread for a discussion:   If your neural network does not generalize well, see:
How to check if a string starts with a specified string? Prove that $f(X)$ and $g(Y)$ are independent if $X$ and $Y$ are independent Let $X$ and $Y$ be independent random variables. Prove that $f(X)$ and $g(Y)$ are independent for any choice of measurable functions $f$ and $g$.   This sounds very obvious, but I have no idea how to approach it.  EDIT: Two random variables are independent if $\Pr\{X = x \text{ and } Y = y\} = \Pr\{X = x\} \cdot \Pr\{Y = y\}$ What are the minimum content and traffic requirements for Google AdSense? How to set systemd for automatically force fsck disks after crash (hard poweroff)?  When I used sysvinit (in Arch Linux) as /sbin/init I used the hack: in the rc.local I create /forcefsck file; in the rc.local.shutdown I remove it. At boot-up rc.sysinit enable force fsck if /forcefsck exists.  How to do same in systemd? Maybe it has built-in facilities for automatic fsck after crush? Prove that if a function $f$ has a jump at an interior point of the interval $[a,b]$ then it cannot be the derivative of any function. Prove that if a function $f$ has a jump at an interior point of the interval $[a,b]$ then it cannot be the derivative of any function.  I know that for $f$ is differentiable in $(a,b)$ and that it has one-sided derivative $f_+' (a)≠f_-' (b)$ at the endpoints. If $C$ is a real number between $f_+' (a)$  and $f_-' (b)$, then there exists $c∈(a,b)$ such that $f' (c)=C $. How can I use this to prove the above? This question has come up    ( ), but I'm finding the answers to be generally incomplete. The general question is "Why does/doesn't my job get killed when I exit/kill ssh?", and here's what I've found.  The first question is:  How general is the following information?  The following seems to be true for modern Debian linux, but I am missing some bits; and what do others need to know?   All child processes, backgrounded or not of a shell opened over an ssh connection are killed with SIGHUP when the ssh connection is closed only if the huponexit option is set: run shopt huponexit to see if this is true. If huponexit is true, then you can use nohup or disown to dissociate the process from the shell so it does not get killed when you exit.  Or, run things with screen. If huponexit is false, which is the default on at least some linuxes these days, then backgrounded jobs will not be killed on normal logout. But even if huponexit is false, then if the ssh connection gets killed, or drops (different than normal logout), then backgrounded processes will still get killed.  This can be avoided by disown or nohup as in (2). There is some distinction between (a) processes whose parent process is the terminal and (b) processes that have stdin, stdout, or stderr .  I don't know what happens to processes that are (a) and not (b), or vice versa.   Final question: How can I avoid behavior (3)?  In other words, by default in Debian backgrounded processes run along merrily by themselves after logout but not after the ssh connection is killed.  I'd like the same thing to happen to processes regardless of whether the connection was closed normally or killed.  Or, is this a bad idea?  Edit: Another, important way to keep jobs of being killed, that works (?) in either case is to run them through .  But, the question is more about understanding when things get killed and when they don't: sometimes people want the jobs to be killed on logout, for instance.  More threads: -  -  -  -  -  -  -  - Finding my house in minecraft Search and replace a line in a file in Python I want to loop over the contents of a text file and do a search and replace on some lines and write the result back to the file. I could first load the whole file in memory and then write it back, but that probably is not the best way to do it.  What is the best way to do this, within the following code?  f = open(file) for line in f:     if line.contains('foo'):         newline = line.replace('foo', 'bar')         # how to write this newline back to the file Show that conguence $\bmod m\,$ is an equivalence relation How to write (La)TeX (with parentheses) [or any other TeX-related logo] How do I troubleshoot random rebooting? If I create just a basic hello.tex file an compile it using   pdflatex hello.tex   The pdflatex program is creating a plethora of file which is a little annoying. Currently it writes all those files about which I really don't care:  hello.aux, hello.log, hello.nav, hello.out, hello.snm, hello.toc   Is there a way to tell pdflatex to only write hello.pdf if the build is successful ? If not, can I specify the output directory of those files ? Maybe I could just write them in /tmp. Do any clones refuse Order 66? When the clones are issued      telling them to turn on the Jedi,    do any of them refuse to carry out the order? Add Appendix with correct page to ToC Do the 3 Doctors remember that they saved Gallifrey after 'Day of the Doctor'? I've read several posts and I've seen one that says before it was Eleven's turn to take part in the events of 'Day of the Doctor', Gallifrey was most certainly destroyed, and when it came to his turn he re-wrote history and saved Gallifrey. But I've rewatched 'Day of the Doctor' recently and the War Doctor asks if he'll remember saving Gallifrey and the Doctor says no. So I'm a bit confused, I don't want to think that the War Doctor or the 10th Doctor have lost their memories after saving Gallifrey and their life of saying they destroyed Gallifrey was a lie. So did they really remember? I have read , but it was asked in the context of servers not desktops or notebooks.  If a good amount of RAM is installed on a notebook, say 8 GB, do we still need a swap space? Let $R$ be a ring such that $x^4=x$ for every $x\in R$. Is this ring commutative? Effective Refractive Index in Paper I am reading the paper, entitled "Artificial uniaxial and biaxial dielectrics with use of two-dimensional subwavelength binary gratings" by Eric B. Grann, M. G. Moharam, and Drew A. Pommet.   In the paper they mention this formula:        Do you know how it can be derived and where it comes from ? String concatenation: concat() vs "+" operator Assuming String a and b:  a += b a = a.concat(b)   Under the hood, are they the same thing?  Here is concat decompiled as reference. I'd like to be able to decompile the + operator as well to see what that does.  public String concat(String s) {      int i = s.length();     if (i == 0) {         return this;     }     else {         char ac[] = new char[count + i];         getChars(0, count, ac, 0);         s.getChars(0, i, ac, count);         return new String(0, count + i, ac);     } } Every time I render an image and try to save it, it just disappears. How do you exactly save a render? I am using a mac.