I have the following problem that I would like to solve, but I don't know where to look. I have a Machine A that produces candles. According to the pre-setting of the machine, the color, the height, and the width of the candle that is being produced, the machine builds a certain number of candles per minute. E.g., a candle without color, and with a width of 2cm and a height of 10cm, and with a single candle wick, and with the pre-setting mode 1 of the machine, the machine produces 10 candles per minute. If we have a red candle, with a width of 5cm and a height of 15cm, and with a triple candle wich, the machine produces 5 candles per minute. The goal of my model is to find out what is the best configuration setting that it is possible to have for a specific candle type so that we can increase the number of candles to be produced per minute. I created a Neural Network model that takes several features as input, and the dependent variable is the number of candles to be produced per minute. The goal of this model is to predict the dependent variable based on machine pre-settings and candle characteristics. I also use Bayesian optimization to obtain the best hyperparameters of the model. Now, I want to find if there is any other pre-setting of the machine so that we can increase the number of candles to be produced. What can I do to find this pre-setting? The only way that I found to do is the following:  Use a brute-force mechanism that tries different configuration settings of the machine in the already trained model, and keep the ones that predicted a higher number of candles to be produced.  I don't know if Bayesian optimization could also help me, and what should I do. Can someone give me some directions so that I could start to research it?
123869
I am very new to design of experiments concept. I would like to design an experiment to understand the relationship between input parameters like amount of various ingredients, cooking temperature, time of cooking, etc. vs. the quality of a cake to be baked.  Assuming that all input parameters are continuous and range bounded, and the quality of the cake to be assessed by an expert from a scale of 1-5 stars, is the best sampling strategy for input parameters uniform over their ranges?  My understanding is that the uniform sampling helps us collect information regarding the entire function space. However in this experiment I am interested in figuring out the best inputs in order to achieve 5-star quality cakes. Therefore is there a better sampling strategy than uniform?
21275
I'm wondering what the value is in taking a continuous predictor variable and breaking it up (e.g., into quintiles), before using it in a model.   It seems to me that by binning the variable we lose information.   Is this just so we can model non-linear effects?  If we kept the variable continuous and it wasn't really a straight linear relationship would we need to come up with some kind of curve to best fit the data?
85834
Appreciate your wisdom on this,  My understanding is most of the tutorials recommend normalizing / scaling the data prior feeding the tensorflow models. Doesn't normalization require that data conforms to the normal parametric distribution?  What good is a non-linear model if scaling / normalizing is a pre-requisite prior using the non-linear model such as tensorflow?   So back to the question, should I always normalize / scale my data prior feeding my tensorflow models?  thanks
85814
When training a neural network, what difference does it make to set:   batch size to $a$ and number of iterations to $b$ vs. batch size to $c$ and number of iterations to $d$   where $ ab = cd $?  To put it otherwise, assuming that we train the neural network with the same amount of training examples, how to set the optimal batch size and number of iterations? (where batch size * number of iterations = number of training examples shown to the neural network, with the same training example being potentially shown several times)  I am aware that the higher the batch size, the more memory space one needs, and it often makes computations faster. But in terms of performance of the trained network, what difference does it make?
71957
I am in the market for a new system (probably a laptop) that would be be used primarily for Bayesian/MCMC analyses.  If I had unlimited funds I would obviously buy very high end hardware and be done with it.  Unfortunately, this is the real world and I have to make my budget stretch.  I am looking for advice about the type of system I should get particularly in terms of processor(s) and amount of RAM.  Please note that I am not looking for specific recommendations regarding RAM and processor.  I am looking for general guidance about how to financially prioritize the various aspects of the computational hardware.  I am fine with a PC or a Mac.  If it is relevant, most of my computations would involve running R and/or C.
89536
Having studied parametric Bayesian statistics during the two last years, I plan to begin to self-study non parametric Bayesian model during this summer and look for recommendations. I would like the book(s) to cover both the theoritical aspects and the practical ones (with implementation, examples of models ...) Thanks for your recommendations.
104728
I'm a software engineer working on machine learning. From my understanding,  (such as OLS) and   (such as logistic regression and SVM) make a prediction based on an inner product between trained coefficients $\vec{w}$  and feature variables $\vec{x}$:  $$ \hat{y} = f(\vec{w} \cdot \vec{x}) = f(\sum_{i} w_i x_i)  $$  My question is: After the model has been trained (that is, after the coefficients $w_i$ have be computed), is it the case that the coefficients will be larger for feature variables that are more important for the model to predict more accurately?  In other words, I am asking whether the relative magnitudes of the coefficients can be used for  by just ordering the variables by coefficient value and then selecting the features with the highest coefficients? If this approach is valid, then why is it not mentioned for feature selection (along with wrapper and filter methods, etc.).  The reason I ask this is because I came across a discussion on . There is a blurb that says:     Built-in feature selection is frequently mentioned as a useful   property of the L1-norm, which the L2-norm does not. This is actually   a result of the L1-norm, which tends to produces sparse coefficients   (explained below). Suppose the model have 100 coefficients but only 10   of them have non-zero coefficients, this is effectively saying that   "the other 90 predictors are useless in predicting the target values".   Reading between the lines, I would guess that if a coefficient is close to 0, then the feature variable with that coefficient must have little predictive power.  EDIT: I am also applying z-scaling to my numeric variables.
17688
I am beginning to learn how to use scikit-learn and I have a hard time choosing the right model.  Here is my dataset: I have 100 persons. Each person was measured three times: baseline, first event and second event. Each measurement had 100 different markers per person that range from 0.1 to 1000. Additionally I have outcome measurements of each event: outcome can be 0, 1 or 2. My task is to find just a few markers (let’s say 10) that can predict outcome with a good accuracy. If I am right it should be: Supervised learning/Classification problem. What model would be the best?  Thanks for your help!
97392
I am currently trying to recreate the laser line described in    I have selected the Cycles Renderer, enabled nodes on my light (spot and point tested). I can modify the strength and color using various node outputs so the nodes seem enabled. I have tested the other types of laser schemes presented in the link above but none produce output close to what is expected.  Is it the light source which I should be modifying the nodes for?  Something that feels quite strange to me is the use of the value of the cross product which I am uncertain how it would be defined. (area of the parallelogram between the vectors?)   Have I missed something fundamental about how nodes work and if so a source to start reading and learning about them would be appreciated.  Thanks in advance.
25149
This example purports to show reflection of light rays from a spherical mirror. It looks good, until you try to draw a ray from the tip of the candle flame, then through the focal point, $F$, and then emerging parallel to the optic axis $(CF)$.  Go ahead, draw it on your computer monitor screen, or print this page and draw it on the paper. Whoops! This ray should then pass through the image of the tip of the candle flame, but it doesn't!  Here's what you get when you draw the third light ray (in red) from the candle flame. It comes nowhere near the image of the candle flame. As soon as you draw some of the "missing" rays, you discover that the diagram becomes inconsistent.    According to me, the diagram seems to be perfect. From where did this anomaly come from? Please help.
108874
According to the , we have the scenario shown below, but how can ABC generate simulation datasets from samples of $\theta$ without knowing or evaluating the likelihood function?  For example, how would this method work for a normal $N(\mu, \sigma)$ distribution? Assuming that the parameter to estimate is $\mu$ and that $\sigma$ is known, how would it generate the simulations without evaluating $N(\mu, \sigma)$?
12674
I'm trying to understand LSTM, using for instance   I get the overall idea, I guess. But I'm not quite sure I get the maths.  I'll set a very simple problem : I have a sequence of numbers and want to predict next number.   So x_t is of shape (1), and as I understand, h_t will be the prediction, so it should also be of shape (1). (I'm just here ignoring batch size)  Now, the equation producing h_t, using * operation should then have two operands of the same size as the result; that is, C_t and o_t should also be of shape (1).  Following on that idea, equation producing C_t forces also shape of (1) for f_t, i_t and ~C_t.  So... everything reduced to scalar real numbers in that case ?  what am I getting wrong ? because this would just not be able to learn much, would it ?
53185
Assume that you found the optimal hyperparameters of a neural network (e.g. a multi layer feed forward NN) with k-fold cross validation in a grid search. Lets assume you have varied the number of epochs and got the optimal epoch number as a result.  Now you want to train the network for the optimal number of epochs on the whole data used for CV. In my opinion following problem arises: When using a constant mini batch size, number of iterations varies for datasets of different size - so your optimal epoch number found for the smaller training split in the CV can not be applied to training with the whole CV data. Am I missing something here? Right at the moment I can not find a proper solution for finding the optimal training epochs for my NN. I searched the net for some time now, but could not find an answer.  P.S. I only described the problem after the inner loop of nested CV, but the same problem arises after the outer loop (training the model on ALL data, not only on CV data for putting the model to production).
112911
Context: I'm trying to understand the use of variational autoencoders as generators. My understanding:   During training, for an input point $x_i$ we want to learn latent $\mu_i$ and $\sigma_i$ and then sample $z_i \sim N(\mu_i, \sigma_i)$ and feed it to the decoder to get a reconstruction $\hat{x}_i = \text{decode}(z_i)$. But we can't do back propagation with sampling operator, so instead we reparametrize and use $z_i = \mu_i + \sigma_i \epsilon$ where $\epsilon \sim N(0, 1)$. Our reconstruction becomes $\hat{x}_i = \text{decode}(\mu_i + \sigma_i \epsilon)$.   However when we're done with training and ready to use it as generator, we sample $z \sim N(0, 1)$ and feed it to decoder: $x_{sample} = \text{decode}(z)$ .  The part that confuse me is that during training, the decode operation was done using $\mu_i + \sigma_i \epsilon$ which to my understanding this is using $N(\mu_i, \sigma_i)$ with different $\mu_i$ and $\sigma_i$ for each training example. However during the generation time, the decode operation is done (effectively) on $\epsilon$ alone from $N(0, 1)$. Why are we setting $\mu = 0$ and $\sigma = 1$ during generation (i.e. using $z = 0 + 1 \cdot \epsilon$)?
100554
Is this telling the model that there are two dimensions (i.e. it’s a matrix) but we don’t yet know the size of that particular dimension? If so, how can the model be compiled? Doesn’t the size of each dimension affect the number of nodes in middle layers?
68626
I have a model that has two preconditions, in which I need to figure out if there is a way to prioritize one precondition to 'run' prior to the second. Is anyone aware of any built-in ability to do so with Model Builder or that can suggest a work around?  For instance, if I try to 'link' one precondition to another, I get the error "Can only connect a variable to a process", this is because my 'output' has a precondition that a python script needs to be run against it first, and then that output has the second precondition executed (but only once the original output has been altered by the Python script).
118642
I'm looking to generate a set of 5 random variables and enforce a dependence structure between them and onto a dependent variable $Y$.  I understand how to generate correlated random variables for multivariate normal, but not when mixing different types.  Below is a little more than I need, but I'm hoping someone can give me a general way of solving this problem...   $X_1$ and $X_2$ need to be highly correlated Bernoulli variables.  $X_3$ needs to take one of 5 categorical values, call them "A"..."E".  $X_4$ needs to be normal, and negatively correlated with $X_1$, $X_2$.  $X_5$ needs to approximate test scores from $0$ to $100$ with a high skew, so gamma probably.  $X_5$ needs to be positively correlated with $X_1$, $X_2$, $X_4$.   Each of these variables must impact a "success/occurrence" Bernoulli distributed variable $Y$.  How would I begin?  I would like to enforce correlation both between the values of $X$, and also between each $X$ and $Y$.  (The categorical correlations seem particularly confusing to me.)