I am developing a LSTM network with Tensorflow. The Tensorflow  shows that there are different activations for the output of the timestep (tanh) and for the data flowing back in to the network at the next time step (sigmoid). It seems to me that we would want the same activation function for both of these, so the fact that they default to different things is surprising to me.  So why would we want to use different activations and recurrent activations?
I have read several introductory texts about LSTM online, but none of them give a rigorously mathematical explanation of what the model actually does. For instance, why does the forget gate forget any more than any of the other gates in the LSTM unit?  Which papers and/or books should I read if I want to get a deeper understanding of LSTM?
Underlined math in a Beamer presentation: felt-tip marker underlining Writing a compiler in its own language Intuitively, it would seems that a compiler for language Foo cannot itself be written in Foo. More specifically, the first compiler for language Foo cannot be written in Foo, but any subsequent compiler could be written for Foo.  But is this actually true? I have some very vague recollection of reading about a language whose first compiler was written in "itself". Is this possible, and if so how? Parallel option performing worse on cluster I'm trying out oracles parallel option on a cluster and surprisingly I am getting worse results with the parallel option. I was expecting some improvement with the parallel option but certainly not worse results. I am wondering why this is the case, and if there is anything wrong with the way I am using the parallel option on my cluster.   I am using a degree of 4 when the number of CPU's I have is 8. I have tried adding parallel to the cluster directly ALTER CLUSTER cluster PARALLEL 4 as well as at the statement on both the index /*+ PARALLEL_INDEX(clust_index, 4) */and tables /*+ PARALLEL(4) */,   Here is my trace from the parallel:    Without parallel: Introduction to string theory I am in the last year of MSc. and would like to read string theory. I have the Zwiebach Book, but along with it what other advanced book can be followed, which can be a complimentary to Zwiebach. I would like a more mathematically rigorous book or lecture notes along with Zwiebach.   Specifically, mention whether the book discusses string theory   Rigorously? Intuitively?   What's the scope of the book? Does it cover the advanced materials, e.g. Matrix string theory, F-theory, string field theory, etc. Maybe even String Phenomenology? Background: One of my co-authors faced problem of compiling the paper that I had built using TexStudio and MikteX 2.9 on his Mac Machine running TexLive.   ShareLaTeX Role: So, one solution is to use ShareLaTeX. But the first problem pops in mind is how safe is uploading unpublished article to a publicly accessible portal?  What I did: I googled security and data breach of ShareLaTex and found nothing except some words from the ShareLaTex community about their word to ensure that our data are safe; perhaps this would not be enough.  My 1st Question: To what extent can I trust ShareLaTeX and put my ready-to-publish content there?   My 2nd Question: Is there any alternate way of sharing LaTeX with others around the globe without changing the underlying platform? Sometimes I have asked a question which got no answers, but is answered in the comments to the question.  How should I handle these questions? I can't accept an answer, so it is never an accepted question.  For example this one:   (ok here the answer in the comments is from me, but nevertheless I also have others of them) Number of homomorphisms between two cyclic groups. How do I make a WinForms app go Full Screen I have a WinForms app that I am trying to make full screen (somewhat like what VS does in full screen mode).  Currently I am setting FormBorderStyle to None and WindowState to Maximized which gives me a little more space, but it doesn't cover over the taskbar if it is visible.  What do I need to do to use that space as well?  For bonus points, is there something I can do to make my MenuStrip autohide to give up that space as well? Parse JSON in JavaScript? How to prove that $4^m + 5^m$ is divisible by 9 when m is an odd number I am trying to use congruence theorems, specifically Euler's Theorem for a proof. I caught an Eevee, which of its evolutions will it evolve into? So I just caught an Eevee. Now, I need 25 candies to evolve it. The fact is that in the GameBoy color versions, that Pokémon needed a stone to evolve instead of experience. Depending of the stone he was fed with (thunder, water or fire), he was evolving in a different Pokémon.   Is the stone feature also in Pokémon Go? If not, what decide the evolution of this Pokémon?   Bonus point : If it's possible, how can I get a stone? How do I  prevent .xsession-errors from eating disk space? Is there any way to completely disable the .xsession-errors file?  I have it as a symlink to /tmp so that my laptop hard disk can (hopefully) go to sleep for once, but at least 95% of the stuff in the file (it fills up at about 500k an hour) is...  (nautilus:1618): GLib-GObject-CRITICAL **: g_value_get_object: assertion `G_VALUE_HOLDS_OBJECT (value)' failed  which is total garbage to me.  I have tried doing a symlink to /dev/null but that does not work (the resulting link is overwritten), and I also do not want the activity so that hopefully my laptop can go to sleep for once.  I am using Ubuntu 11.04, with no special add-ons to Nautilus. How do threads work in Python, and what are common Python-threading specific pitfalls? I've been trying to wrap my head around how threads work in Python, and it's hard to find good information on how they operate. I may just be missing a link or something, but it seems like the official documentation isn't very thorough on the subject, and I haven't been able to find a good write-up.  From what I can tell, only one thread can be running at once, and the active thread switches every 10 instructions or so?  Where is there a good explanation, or can you provide one? It would also be very nice to be aware of common problems that you run into while using threads with Python. Permanent PATH variable How will I make this /media/De Soft/mongodb/bin PATH variable permanent?  Everyone is saying "export PATH=$PATH:media/De\ Soft/mongodb/bin to your ~/.profile, or .bashrc, or .zshenv depending on your shell".   I don't know what is ~/.profile, or .bashrc, or .zshenv. What do they do actually?  How will I add export PATH=$PATH:my/path to my .profile/.bashrc/.zshenv?   I'm using 64 bit Ubuntu 14.04 LTS with default terminal. I have TeX Live 2013 installed via MacTeX on my Snow Leopard laptop.  I copied the following code from page 247 of version 2.10 of pgfmanual:  \documentclass[a4paper,12pt]{report} \usepackage{tikz} \usetikzlibrary{decorations}  \begin{document}  \begin{tikzpicture} \node (a) {A}; \node (b) at (2,1) {B}; \draw (a) -- (b); \draw[decorate,decoration=brace] (a) -- (b); \draw[decorate,decoration={brace,mirror},red] (a) -- (b); \draw[decorate,decoration={brace,mirror,raise=5pt},blue] (a) -- (b); \end{tikzpicture}  \end{document}   Yet somehow I get this error:  ./Untitled2.tex:11: Package pgfkeys Error: I do not know the key '/pgf/decorati on/brace' and I am going to ignore it. Perhaps you misspelled it.  See the pgfkeys package documentation for explanation. Type  H &lt;return&gt;  for immediate help.  ...                                                l.11 \draw[decorate,decoration=brace]                                       (a) -- (b); ?    Is this a bug? Ready to use leaderboard service. Is any? I'm looking for ready to use thirdparty leaderboard service where player's achivementes and scores can be placed. I'd intagrate it into my game to allow players push scores and locate themselves in the tables of tops. It would be nice whether anybody experienced share some points with me. Let $f(z)$ be a holomorphic function on a disk of radius $&gt;1$ about the origin, and suppose that $|f(z)|\leq 1$ for $z$ with $|z|=1$ and $\text{Re}(z)&gt;0$ and $|f(z)|\leq 2$ for $z$ with $|z|=1$ and $\text{Re}(z)\leq 0$.  How does one show the bound $|f(0)|\leq \sqrt{2}$? How to unplug a network cable that's stuck? How do I deliver a baby? I'm playing Fallout Shelter, and my newest bonus objective is to "Deliver 1 Baby Dweller."  I've already figured out that I can put a male vault dweller and a female vault dweller together in some residential quarters, and and eventually, if they hit it off, the lady will replace her vault suit with a yellow sweater and a decidedly pregnant figure.   Good enough.  But how long does it take her to deliver the child? Do I have to induce it through some sort of tapping ritual or placing her in the correct room? Does the pregnant dweller need to stay in the living quarters, or can I put her to work in my Diner or Water Treatment Facility? Do the SPECIAL stars of the parents matter?  Tell me everything about  and what I can do to control the process. Smooth transition between two materials