How to prevent the keras convolutional neural network model to over-fit? I want to build a convolutional neural network and train it to recognise whether the digit is 0 or 1.  Example of my training data is a 800 * 600 gray scale image containing a digit one: I have 22 such images, 11 containing zeroes, 11 containing ones.  I now build a convolutional nerual network:  def addConvolutionalLayer(layer, newDepth = None):     print(layer.get_shape().as_list())     if newDepth is None:         newDepth = layer.get_shape().as_list()[3] * 2     newLayer = Conv2D(newDepth, kernel_size = (3, 3),                 activation = 'relu', padding='same') (layer)     newLayer = Dropout(0.2) (newLayer)     return Conv2D(newDepth, kernel_size = (3, 3),         activation = 'relu', padding='same') (newLayer)  def buildModel():     s = Input((600, 800, 1))     d1 = addConvolutionalLayer(s, 32)     d2 = addConvolutionalLayer(MaxPooling2D(4)(d1))     d3 = addConvolutionalLayer(MaxPooling2D(4)(d2))     d4 = addConvolutionalLayer(MaxPooling2D(4)(d3))     f = Flatten()(d4)     dense = Dense(64, activation='relu') (f)     o = Dense(1, activation='sigmoid') (dense)     model = Model(inputs = [s], outputs = [o])     model.compile(optimizer='adam', loss='binary_crossentropy',               metrics=['accuracy'])     return model  model = buildModel() model.compile(optimizer='adam',         loss='binary_crossentropy',         metrics=['accuracy'])   In short, a network consists of multiple convolutional layers (with MaxPooling included), ending with a dense layer.  I then split the data into training and test sets:  X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)   X_train, X_test are arrays of float values, each value corresponding to the pixel in the image (1.0 for white, 0.0 for black).  y_train, y_test are lists containing 0 or 1 depending on the digit in the corresponding image.  Finally, I run the model to train:  model.fit(X_train, y_train, epochs = 50, validation_split=0.2) model.save('prediction.h5')   The problem is that the model gets over-fitted: it reaches 100% accuracy on the training data, but does not perform well on the validation set:  Epoch 95/100 12/12 [==============================] - 0s 32ms/step - loss: 1.0640e-07 -     acc: 1.0000 - val_loss: 10.7454 - val_acc: 0.3333   How to fix this issue?
What should I do when my neural network doesn't generalize well? I'm training a neural network and the training loss decreases, but the validation loss doesn't, or it decreases much less than what I would expect, based on references or experiments with very similar architectures and data. How can I fix this?    As for question        to which this question is inspired, the question is intentionally left general so that other questions about how to reduce the generalization error of a neural network down to a level which has been proved to be attainable, can be closed as duplicates of this one.  See also dedicated thread on Meta:
Let's say there is a user who has found a satisfactory answer to a common question asked on Stack Overflow (or other Stack Exchange website).  This answer may be a snippet of code, or an addon, or a framework, or something else.    Is it acceptable for this user to formulate an answer for one question, find every related question and then copy the exact answer to each of those questions?  And, if not, what should  users who finds themselves in this situation do? How to vertically center a "div" element for all browsers using CSS? What are the effects of exceptions on performance in Java? Question:  Is exception handling in Java actually slow?  Conventional wisdom, as well as a lot of Google results, says that exceptional logic shouldn't be used for normal program flow in Java.  Two reasons are usually given,   it is really slow - even an order of magnitude slower than regular code (the reasons given vary),     and    it is messy because people expect only errors to be handled in exceptional code.     This question is about #1.  As an example,  describes Java exception handling as "very slow" and relates the slowness to the creation of the exception message string - "this string is then used in creating the exception object that is thrown. This is not fast."  The article  says that "the reason for this is due to the object creation aspect of exception handling, which thereby makes throwing exceptions inherently slow".  Another reason out there is that the stack trace generation is what slows it down.  My testing (using Java 1.6.0_07, Java HotSpot 10.0, on 32 bit Linux), indicates that exception handling is no slower than regular code.  I tried running a method in a loop that executes some code.  At the end of the method, I use a boolean to indicate whether to return or throw.  This way the actual processing is the same. I tried running the methods in different orders and averaging my test times, thinking it may have been the JVM warming up.  In all my tests, the throw was at least as fast as the return, if not faster (up to 3.1% faster).  I am completely open to the possibility that my tests were wrong, but I haven't seen anything out there in the way of the code sample, test comparisons, or results in the last year or two that show exception handling in Java to actually be slow.  What leads me down this path was an API I needed to use that threw exceptions as part of normal control logic.  I wanted to correct them in their usage, but now I may not be able to.  Will I instead have to praise them on their forward thinking?  In the paper , the authors suggest that the presence of exception handlers alone, even if no exceptions are thrown, is enough to prevent the JIT compiler from optimizing the code properly, thus slowing it down.  I haven't tested this theory yet. I would like to create some kind of feature matrix with available classes in columns and implemented features as rows. Problem is, that the class names are quite long.  Rotating them by 90° simply looked quite silly, so am looking for alternate ideas.  The only idea I had so far was to write the heading text horizontally with a arrow (or similar) matching the "heading row" with a column. But frankly I have no idea how to do something like this in latex.  An minimal example of my naive approach looks like this.  \documentclass{beamer}  \usepackage{tabularx}  % I am actually using a custom theme % The space ``wasted'' is similar for PaloAlto though \usetheme{PaloAlto}  \begin{document} \frame {   \frametitle{Comparision of boost implemented graph classes}   \begin{tabularx}{\textwidth}{X c c c}     %     % Column Headers, first column is empts     %     & \rotatebox{90}{adjacency\_list}     & \rotatebox{90}{adjacency\_matrix}     & \rotatebox{90}{compressed\_sparse\_row\_graph}      \     %     % Rows, I suspect I need about 10     % Class               adjlist          adjmatrix     csrg     \hline      AdjacencyGraph        & Yes            & No          & Yes \\     Interface \#2         & Yes            & No          & Yes \\     Interface \#3         & Yes            & No          & Yes \\     Interface \#4         & Yes            & No          & Yes \\     Interface \#5         & Yes            & No          & Yes \\     Interface \#6         & Yes            & No          & Yes \\     Interface \#7         & Yes            & No          & Yes \\     Interface \#8         & Yes            & No          & Yes \\     Interface \#9         & Yes            & No          & Yes \\     % Rest omitted   \end{tabularx} } \end{document}    If its important: I am doing this for a presentation with beamer. Which topics are essential for QGIS beginners? What topics do you think are essential in an introduction for QGIS?  The target audience is those with zero or rudimentary knowledge in mapping and geospatial software. I've followed this tutorial to install Ubuntu 15.10:    After restarting my computer I got to the grub menu and chose Ubuntu. Shortly after that I got this error:  /run/lvm/lvmetad.socket: connect failed: No such file or directory WARNING: Failed to connect to lvmetad. Falling back to internal scanning.   These messages keep adding up on a black screen every second. After a while,  I get access to the initramfs ash console.  what am I doing wrong? Performing test for homogeneity of variance without samples I'd like to perform a test for homogeneity of variance in r, but I don't have samples available to test.    I have the sample mean of each group, the sample deviation of each group, and the sample size of each group(100).  Brand    ybar_i  s_i    n_i --------------------------- LowTar   9.64    .291   100 A        10.22   .478   100 B        10.77   .372   100 C        11.57   .352   100 D        13.59   .469   100   I'm thinking I should use the Levene test from Lawstat package, but that requires samples to test.  Or I think I could use the Fmax test. but that also requires samples. How bad is it to use display: none in CSS? I've heard many times that it's bad to use display: none for SEO reasons, as it could be an attempt to push in irrelevant popular keywords. A few questions:   Is that still received wisdom? Does it make a difference if you're only hiding a single word, or perhaps a single character? If you should avoid any use of it, what are the preferred techniques for hiding (in situations where you need it to become visible again on certain conditions)?   Some references I've found so far:       If you're straight-out using CSS to hide text, don't be surprised if that is called spam. I'm not saying that mouseovers or DHTML text or have-a-logo-but-also-have-text is spam; I answered that last one at a conference when I said "imagine how it would look to a visitor, a competitor, or someone checking out a spam report. If you show your company's name and it's Expo Markers instead of an Expo Markers logo, you should be fine. If the text you decide to show is 'Expo Markers cheap online discount buy online Expo Markers sale ...' then I would be more cautious, because that can look bad."   And  on the same article     We can flag text that appears to be hidden using CSS at Google. To date we have not algorithmically removed sites for doing that. We try hard to avoid throwing babies out with bathwater.   Eric Enge      The legitimate use of this technique is so prevalent that I would rarely expect search engines to penalize a site for using the display: none attribute. It’s just very difficult to implement an algorithm that could truly ferret out whether the particular use of display: none is meant to deceive the search engines or not. Beamer Bibliography Icon I want to list my references in a beamer presentation using thebibliography environment, with the nice little    book icon (\beamertemplatebookbibitems) and  article icon (\beamertemplatearticlebibitems)   in front of each source.  Is there an icon for websites too? something like \beamertemplatewebsitebibitems? If not, is there a way to implement this easily ? How quickly is the desired state of curl of the Electric Field around a point achieved where a Magnetic Field Vector has just begun changing? My question and context with explanation are given below. Thank you in advance.   Faraday’s Law () states that a time-changing magnetic field vector induces a curl of the Electric field around that point. However it does not specify how quickly the necessary spatial gradient is developed in the Electric Field to satisfy that curl. Of course, the Electric Field vectors around that point cannot just jump to the required values instantaneously in accompaniment as soon as the magnetic field starts changing at a fixed rate, since it is required that the Electric Field has a sensible rate of change to in turn induce a curl of the magnetic Field and continue the propagation of the wave. It would seem the exact rates at which the Electric and Magnetic Fields are changing is critical in the mechanism of an EM wave’s propagation. How would I then calculate the rate of change of E somewhere along the wave given I know the rate of change of the B vector that began it, and given the wave is restricted to one dimension for simplicity? There should after all be some delay in how the EM wave moves from point to point in space, seeing as it moves at the finite speed of causality (3*10^8 m/s), as shown in the first image:    Some of my ideas so far have been that perhaps the rate of change of E is determined such that it exactly cancels the growth of the time-changing E vector that gave rise to a curl of B before, which developed at such a rate to induce a curl of E, which demands this rate of change of E which I am talking about now. In this way the wave might level out the growth of the fields in its wake and spread the change further outwards, in the same way smoothening out a crease spreads it elsewhere. However I am stuck on how to show this rigorously.  I am open to any difficulty of maths, so long as the answer provides the explanation. Below are some diagrams to help visualize what I am talking about:    Faraday's Law requires that a curl of E is developed, which under these restrictions requires a spatial gradient of E to appear across that point. Since the field across that point was previously in some other state, it is required that the Electric Field vectors that are infinitesimally close to that point undergo a rate of change through time to achieve the desired state of curl. The question is, how quickly is this desired state of curl achieved? In Harry Potter and the Philosopher's Stone, Hermione is in the toilets when Harry and Ron lock in the troll. She lies and says that she went looking for the troll. Why? She could have just said that she was in there because she needed the toilet (she was allowed to be, it's a girls' bathroom) and Harry and Ron came to save her.   Is there a reason she lied? Permission for snap applications I have some trouble with the Ubuntu notepadqq package. After opening a file from my apache webroot I cannot see any content. Saving the file is also not possible. I thougt this would be a classic permission issue. Therefore, I changed the group of the webroot from root to www-data and added my user to this group. Permissions are 775, so in theory it should now be possible for me to edit files in my webroot. But nevertheless, when I open a file of this directory with notepadqq, it still only shows an empty file.  When using gksudo notepadqq I get the following error message:  No protocol specified QXcbConnection: Could not connect to display :0   I figured out that notepadqq is a so called snap application and I wonder if this is the reason for my problems.  Can someone give me a hint how I could fix this issue (without using 777 permissions)? How is Leibniz's rule for the derivative of a product related to the binomial formula? Possible Duplicate:          The  describes the expansion of the $n$th power of the sum $(a+b)$:  $$(a+b)^n = \sum_{k = 0}^n {n\choose k}a^kb^{n-k}$$  In calculus, there is a generalization of the product rule called , which describes the expansion of the $n$th derivative of the product of two functions $f$ and $g$:  $$(fg)^{(n)} = \sum_{k = 0}^n {n\choose k}f^{(k)}g^{(n-k)}$$  It's not difficult to prove using the product rule and induction, what I'm really trying understand here is why the formula is true. Is there a way to prove it using the binomial formula? How are these two formulas related? Is there a combinatorial proof for Leibniz's rule like there is for the ?   I understand this question is a bit vague, but I hope you'll agree that these two formulas look so similar it just feels like there should be some relation between the two. Of course this is not necessarily true, but I'm hoping it's not just a crazy coincidence. Confirming that my own question is a duplicate by voting to close should have immediate effect Someone voted to close my question as a duplicate. I agreed, so I cast my close vote (I have enough reputation to vote to close).  . So I expected this to happen.  But it didn't. This confused me — is this not a question I asked after all? No, it is. Maybe I need to reload the page? Nope, the question still isn't closed. Is the asker confirmation feature broken? Well, there's the “This question may already have an answer here” banner at the top. What if I click “That solved my problem!”? Ah, that does close the question.  It turns out that only clicking “That solved my problem!” instantly closes the question. Casting a close vote has a different effect, it's just an ordinary close vote.  Please make casting a close vote equivalent to clicking “That solved my problem!”. I have a model similar to below with a continuous outcome, continuous predictor and a categorical variable. the goal of this analysis is to make inferences on the group variable. The issue is that y is bounded by x in the form of y >= -x. When the model makes predictions you can see in the plot below that the predicted value is outside the possible range. See the last plot.   How can this be remedied? A custom loss function to penalize going beyond the bound and use gradient descent regression? Or is there some other way? Thank you.   x =   rep(seq(-10,10,1),each=5)  y =  rep(0,length(x) ) weights = sample( seq(1,1000,1) ,length(x)) groups = rep( letters[1:5], times =length(x)/5  )   ## the outcome y has the relationship y &gt;= -x for(i in 1:length(x)){   y[i] = sample(  seq(-x[i],-x[i]+ ifelse(x[i]&lt;0 ,                                            sample(seq(1,10,.1),1),                                           sample(seq(5,40,.1),1) ),1)                        ,1) }   dat =data.frame(x = x, y = y, weights = weights, group = groups) model = lm(dat = dat, y ~ x + group + group * x, weights = weights) dat$FIT= predict(model, dat, allow.new.levels = TRUE)  library(ggplot2) ggplot(data = dat, aes(x = x, y = y, color = group))+geom_point( aes(size = weights))+ geom_line(data = dat, aes(x = x, y = FIT, color = group))    #data for new predictions newdata = data.frame( x = seq(-100,100,1) ,                        group = rep("c", length( seq(-100,100,1) )),                       bound = -seq(-100,100,1)                       )  ## when pridicting new data the outcome y has the relationship y &gt;= -x ## but the predicted vales are below the bound. See how red line is below blue to the left of zero new_preds  = predict(model, newdata, allow.new.levels = TRUE) ggplot(data = newdata, aes(x = x, y = new_preds, color = group))+geom_line()  +   geom_line(data = newdata, aes(x = x, y = bound, color = "green")) what should I do to stop the cases environment compressing the first line of the following? \begin{equation} \alpha_{mn} = \begin{cases} \frac{16\mu^2}{mn\pi^2\left[\pi^2(m^2/a^2 + n^2/b^2) - \mu^2\right]} &amp;\mbox{for } m \mbox{ and } n \mbox{ odd;}  \\ 0 &amp;\mbox{for } m \mbox{ or } n \mbox{ even;} \end{cases} \\[1ex] \end{equation}  I'm not sure to do tex on this site, but if anyone tries it they see the first case is compressed because it's a fraction. Professionals: Would you choose GIMP over Photoshop and/or Inkscape over Illustrator? Emboss text onto curved surface I am fairly new to blender and I am trying to do something which I am assuming is fairly straight forward with no success. All I'm looking to do is emboss some text onto the curved part of a yogurt pot. Image attached below. What is the quickest and easiest way of doing this?  Thanks in advance UK visa blank refusal letter without reason for refusal Why can't I launch eclipse from a terminal ? change $PATH environment variable