What will happen if we set the same weights to two neurons in the same layer?   What will happen if weights are 0 (zero)?  Explain why.   Please, also give some references for further study.  For instance,   the following picture is with random weights,    The following with all 1s,    The following with all zeros,
121051
Does it make any sense to have two (or more) neurons in a neural network with the same weights? (intuitively it makes no sense, since all the neurons would behave the same way).   Please consider both the input and hidden layers. What if the weights are equal 0?
16634
I'm training a neural network and the training loss decreases, but the validation loss doesn't, or it decreases much less than what I would expect, based on references or experiments with very similar architectures and data. How can I fix this?    As for question        to which this question is inspired, the question is intentionally left general so that other questions about how to reduce the generalization error of a neural network down to a level which has been proved to be attainable, can be closed as duplicates of this one.  See also dedicated thread on Meta:
72926
Hello I am taking a Discrete Mathematics course and having some trouble with this question about sets:  Under what conditions is $A- B = B- A$  Diagram:   Diagram:   Maybe I'm understanding incorrectly, but how can A-B and B-A be equal if they contain elements that are not in each other?
103703
As far as I understand, one of the main claimed problems with initializing e.g. a feed-forward neural network (with several $\text{tanh}$ or $\text{ReLU}$ layers) with $W=0$ is that it doesn't break "network symmetry", meaning, backpropagation would propagate the same error through all such units (i.e. "nudging all weights in the same direction"). This is, I presume, undesirable because we would not be learning "different" calculations through different paths of the network.  However, I'm confused why that even matters in this case given the fact that if $W$ ever drops to 0, we will effectively be propagating no gradients at all through the network, since W=0 would multiply all errors from the output and prevent any learning.  Put another way, even if $W=0$ does not break network symmetry (wasting calculations and paths in the network) is it correct to say that if $W=0$ (e.g. by initialization) we are effectively killing gradients in a neural network, and thus no learning can take place?
4571
Background I am a computer programmer who is fascinated by artificial intelligence and artificial neural networks, and I am becoming more curious about how biological neural networks work. Context &amp; what I think I understand In digesting all I have been reading, I am beginning to understand that there are layers to neural networks.  A front-line layer of neurons may receive, for example, a visual stimulus such as a bright light.  That stimulus is taken in by the front-line neurons, each of which produce a weighted electro-chemical response that results in a binary decision to pass an electrical charge through its axon to the dendrites of the tens of thousands of neurons to which it is connected. This process repeats through layers channeling the electrical signals and focusing them based on their permutations until ultimately a charge is passed to a focused response mechanism such as the nerves that control shrinking of the pupils. Hopefully I got that correct. Preamble to the question ;) Assuming that I am not completely off-base with my basic understanding of how a biological neural network operates, I am beginning to grasp how an input (stimulus) results in an output (response) such as motor movement or reflexes.  That would just seem to be basic electricity of open and closed circuits. HOWEVER, what confuzzles me still is how a memory is stored.  The analogy to an electrical circuit breaks down here, for in a circuit I can't really stop the flow of electrons unless I dam up said electrons in a capacitor.  If I do that, once the electrons are released (accessed), they are gone forever whereas a memory endures. So. . . How the heck are memories constructed and stored in the human brain?  Are they stored in a specific region?  If so, where?
107168
I am a beginner to neural networks and have had trouble grasping two concepts:   How does one decide the number of middle layers a given neural network have? 1 vs. 10 or whatever. How does one decide the number of neurons in each middle layer? Is it recommended having an equal number of neurons in each middle layer or does it vary with the application?
31606
I was hoping I could receive some clarification into the the four cases:   Placing labeled balls in unlabeled boxes with repetition. Placing labeled balls in unlabeled boxes without repetition. Placing unlabeled balls in unlabeled boxes with repetition. Placing unlabeled balls in unlabeled boxes without repetition.   I just want an intuitive understanding of the four cases.  How many ways are there to arrange, say, 4 balls into 9 boxes or 9 balls into 4 boxes in each case above, and why?
12674
I'm trying to understand LSTM, using for instance   I get the overall idea, I guess. But I'm not quite sure I get the maths.  I'll set a very simple problem : I have a sequence of numbers and want to predict next number.   So x_t is of shape (1), and as I understand, h_t will be the prediction, so it should also be of shape (1). (I'm just here ignoring batch size)  Now, the equation producing h_t, using * operation should then have two operands of the same size as the result; that is, C_t and o_t should also be of shape (1).  Following on that idea, equation producing C_t forces also shape of (1) for f_t, i_t and ~C_t.  So... everything reduced to scalar real numbers in that case ?  what am I getting wrong ? because this would just not be able to learn much, would it ?
30997
I wonder if there is a clear-cut difference between the so-called zero-inflated distributions (models) and so-called hurdle-at-zero distributions (models)? The terms occur quite often in the literature and I suspect they are not the same, but would you please explain me the difference in simple terms?
32263
Can you please tell me how much is (-2) % 5? According to my Python interpreter is 3, but do you have a wise explanation for this?  I've read that in some languages the result can be machine-dependent, but I'm not sure though.
71675
Given two square matrices $A,B$ with same dimension, what conditions will lead to this result? Or what result will this condition lead to? I thought this is a quite simple question, but I can find little information about it. Thanks.
46087
Given that $X_1, X_2...$ are mutually independent random variables. For each $i$ with $1\le i \le n$   the variable $X_i$ is equal to either $0$ or $n+1$ $E(X_i)$ = $1$   also.. if $X_i$ is equal to either $0$ or $n+1$, doesn't that mean that all $X_i$ need to be $0$?  Thanks
54971
A coin is tossed three times: The probability of zero heads is 1/8 and the probability of zero tails is 1/8.  And my question is: What is the probability that all three tosses result in the same outcome?   So, if P(zero heads)= 1/8 , then that should be the same of p(all tails)?  If so, we could use the Addition Rule which is $P(A \cup B) = P(A) + P(B)$  where A and B are disjoint events, i.e. $A \cap B = \emptyset$, A is the event of tossing all heads and B is the event of tossing all tails.  I'm not sure how to continue after that... would the complement be used?
119731
I am mainly confused about my reasoning/solution  Problem: I have a bag of numbers ranging from $1,2... n$. I randomly choose a number and then I throw out all numbers less than that. Then I keep choosing numbers until I've found the largest one. What's the expected number of things I have chosen?  My Argument: I expect on average to have chosen a number of $n/2$ on each pick. Hence, everytime I choose a number, I expect to discard half of them. Hence, the $E(X) = \log_2(n)$  Note that I devised this problem as an equivalent one while trying to solve the "cluster of cars on one long road" question. If this is not equivalent, please let me know but I am confident that it is. As a result of this equivalency, I know my solution is incorrect. However, I cannot pinpoint where the issue arises.
13634
Thanks for looking. First off, I am not a biologist, just a curious layman, so I apologize in advance if this isn't a &quot;good&quot; question.  Please don't downvote me into oblivion. I read today that the human brain has about 100 billion neurons and that got me wondering: is that number pretty standard for everyone or does someone like say, Einstein, have many more neurons than this daft poster. Question Is there a correlation between total neuron count and intelligence, or does intelligence depend more on the way neurons are used--or some other factor like previous experience--rather than their total count? Can this question be answered? There are two very valid points-of-view posted as answers below, but it sounds like we simply don't have a solid answer to this question at this point in time, so I hesitate to select a &quot;correct&quot; answer.
105835
How many ways are there to stack coins on top of the other (2D stack) without any coin falling down ? Here's an example for $n=3$:  Now this is most likely just like the monotonic path of Catalan number with up and down instead of up and right but I just don't get why is that ? Well, any advice would be appreciated.
53621
I have an idea of how to get started, but not sure where to go from there.  Since I am using two 3-8 decoders to develop a 4-to-16 decoder, I want to use 4 inputs out of the two 3-8 decoders.  So I'll use all three of the first and the first of the second, and connect the last two inputs to ground, since they won't be used.  Any pointers on where to go from here are appreciated.