I'm working on a signal denoising problem. Because of not having enough data for training I'm considering using one of the generative models like VAE or GAN to generate data similar to real data for training. I didn't find any relevant precedents with regard to my use case for generative models. Is it because generative models don't work for my purpose or am I missing something?
121134
Can a generative adversarial network (GAN) be used for data augmentation (i.e. to generate synthetic examples that are added to a dataset)? Would it have any impact on the performance of a model trained on the augmented dataset?
27483
Background: I'm studying chapter 6 of Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville. In section 6.2.2.2 (pages 182 of 183 ) the use of sigmoid to output $P(y=1|x)$ is motivated.  To summarize some of the material they let $$z = w^Th+b$$ be an output neuron before an activation is applied where $h$ is the output of the previous hidden layer, $w$ is a vector of weights and $b$ is a scalar bias. The input vector is denoted $x$ (which $h$ is a function of) and the output value is denoted $y=\phi(z)$ where $\phi$ is the sigmoid function. The book wishes to deﬁne a probability distribution over $y$ using the value $z$. From the second paragraph of page 183:     We omit the dependence on $x$ for the moment to discuss how to deﬁne a   probability distribution over $y$ using the value $z$. The sigmoid can be motivated by constructing an unnormalized probability distribution $\tilde P(y)$, which does not sum to 1. We can then divide by an appropriate constant to obtain a valid probability distribution. If we begin with the assumption that the unnormalized log probabilities are linear in $y$ and $z$, we can exponentiate to obtain the unnormalized probabilities. We then normalize to see that this yields a Bernoulli distribution controlled by a sigmoidal transformation of z:   \begin{align} \log\tilde P(y) &amp;= yz \\ \tilde P(y) &amp;= \exp(yz) \\ P(y) &amp;= \frac{\exp(yz)}{\sum_{y'=0}^1 \exp(y'z) } \\ P(y) &amp;= \phi((2y-1)z) \end{align}   Questions: I'm confused about two things, particularly the first:   Where is the initial assumption coming from? Why is the unnormalized log probability linear in $y$ and $z$? Can someone give me some inituition on how the authors started with $\log\tilde P(y) = yz$? How does the last line follow?
88821
In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, mirroring, adjusting contrast, etc. it is possible to generate additional images from the original ones.  In the case of images, the data augmentation is relatively straightforward. However, suppose (for example) that one has a training set of $100$ samples and few hundred continuous variables that represent different things. The data augmentation does not anymore seem so intuitive. What could be done in such case?
52356
Suppose I have a set of sample data from an unknown or complex distribution, and I want to perform some inference on a statistic $T$ of the data. My default inclination is to just generate a bunch of bootstrap samples with replacement, and calculate my statistic $T$ on each bootstrap sample to create an estimated distribution for $T$.  What are examples where this is a bad idea?  For example, one case where naively performing this bootstrap would fail is if I'm trying to use the bootstrap on time series data (say, to test whether I have significant autocorrelation). The naive bootstrap described above (generating the $i$th datapoint of the nth bootstrap sample series by sampling with replacement from my original series) would (I think) be ill-advised, since it ignores the structure in my original time series, and so we get fancier bootstrap techniques like the block bootstrap.  To put it another way, what is there to the bootstrap besides "sampling with replacement"?
117554
What are the advantages, why would one use multiple LSTMs, stacked one side-by-side, in a deep-network? I am using a LSTM to represent a sequence of inputs as a single input. So once I have that single representation— why would I pass it through again?  I am asking this because I saw this in a natural-language generation program.
74573
My understanding is that GPUs are more efficient for running neural nets, but someone recently suggested to me that GPUs are only needed for the training phase. Once trained, it's actually more efficient to run them on CPUs.  Is this true?
111012
A neural net learns features of a data set as a means of achieving some goal.  When it is done, we may want to know what the neural net learned.  What were the features and why did it care about those.  Can someone give some references on the body of work that concerns this problem?
119537
I used to work with Monte-Carlo simulations for a while. In my case, I generated random data for a variety of input parameters according to uniform distributions (with non-negative support), say for example two variables $a$ and $b$ where data is generated: $a_1, \dots, a_n \leftarrow U(1,2)$ and $b_1, \dots, b_n \leftarrow U(4,5)$.   Eventually, I evaluated a deterministic model $M$ for all generated pairs $(a_i,b_i)$ and calculate some statistics, like sample average of $\frac{M(a_1,b_1) + \dots + M(a_n,b_n)}{n}$.  Here is my question: What is the purpose of drawing random samples from the uniform distribution, rather than deterministically select $n$ equally distributed point from the interval $[1,2]$ and $[4,5]$ respectively and simply evaluate $M$ with those points? So, what would be the purpose of doing this stochastically rather than deterministically?
13641
I am a grad student developing an interest for statistics. I like the material over-all, but I sometimes have a hard time thinking about applications to real life. Specifically, my question is about commonly used statistical distributions (normal - beta- gamma etc.). I guess for some cases I get the particular properties that make the distribution quite nice - memoryless property of exponential for example. But for many other cases, I don't have an intuition about both the importance and application areas of the common distributions that we see in textbooks.   There are probably a lot of good sources addressing my concerns, I would be glad if you could share those. I would be a lot more motivated into the material if I could associate it with real-life examples.
86995
In my machine learning class these two methods were discussed and mentioned that both should be used. I have a couple questions about this:  1) Can I mix and match these two approaches? e.g. Feature Scale x1 and Mean Normalize x2?  2) How do you determine which of these options to apply? It seems that either could accomplish the task of increasing your convergence rates... I suppose you just need to know your data set to understand which will reliably reduce your values while leaving as few outliers as possible?
56025
Are there any speedy alternatives to the EM algorithm for learning models with latent variables (especially pLSA)? I'm okay with sacrificing precision in favor of speed.
45073
Normally in principal component analysis (PCA) the first few PCs are used and the low variance PCs are dropped, as they do not explain much of the variation in the data.  However, are there examples where the low variation PCs are useful (i.e. have use in the context of the data, have an intuitive explanation, etc.) and should not be thrown away?
23836
I am thinking about the  in a variational auto-encoder. I know that it can be used with normal distribution. Can the re-parameterization trick be applied to other distributions like Bernoulli? What are the limitations of its application?
120141
I read many threads discussing why 2D convolutional layer is typically used for RGB images in neural network. I read that it is possible to use 3D conv layer. What I do not understand is the math behind it. Say your image is 300 by 300, and the kernel_size = (3, 3) and filter = 16 for the Conv2D layer. Input_shape would be (300, 300, 3) because there are 3 channels(RGB).  Since the kernel is 2D, the convolution can only be done at 1 channel at a time. Is that correct? Are the same kernel applied/convolved for the 3 channels? If so there should be 3 output but the dimension of the output would be (298, 298, 16). Is it averaged over the 3 channels?
62878
I tried an artificial neural network (ANN) model. Using same data set, it gives a different answer every time I run it in MATLAB. Does anyone why this happens and can suggested the best way to analyze the result?
63510
Why are bias nodes used in neural networks?  How many you should use?  In which layers you should use them: all hidden layers and the output layer?
108874
According to the , we have the scenario shown below, but how can ABC generate simulation datasets from samples of $\theta$ without knowing or evaluating the likelihood function?  For example, how would this method work for a normal $N(\mu, \sigma)$ distribution? Assuming that the parameter to estimate is $\mu$ and that $\sigma$ is known, how would it generate the simulations without evaluating $N(\mu, \sigma)$?