texts: RandomForestClassifier OOB scoring method

texts:
Does the random forest implementation in scikit-learn use mean accuracy as its scoring method to estimate generalization error with out-of-bag samples? This is not mentioned in the documentation, but the score() method reports the mean accuracy.
I have a highly unbalanced dataset, and I am using AUC of ROC as my scoring metric in the grid search. Is there a way to tell the classifier to use the same scoring method on the OOB samples as well?


texts: Can Gaussian Process be fit incrementally?

texts:
I am using Gaussian Process Regressor to fit data for a Bayesian Optimiser. This is a relevant part of my Python code.

from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import (RBF, Matern, RationalQuadratic,
                                          ExpSineSquared, DotProduct,
                                          ConstantKernel)
kernel = Matern(length_scale=lenSc,nu=2.5)
gp = GaussianProcessRegressor(kernel=kernel)
l2=10
Xi=np.concatenate((nprand.uniform(10.0,30.0,[l2,6]),nprand.uniform(-0.3,0.3,[l2,42]),nprand.uniform(0.0,30.0,[l2,1]),nprand.uniform(-30.0,30.0,[l2,1]),nprand.uniform(0.0,0.3,[l2,6]),nprand.uniform(5.0,20.0,[l2,6])),axis=1)
yi=map(lambda x: f1(x), Xi)
gp.fit(Xi, yi)
while(1):
   #Some code to get the new values- cc and nv
   Xi=numpy.row_stack((Xi,cc))
   yi=np.concatenate((yi,[nv]))
   gp.fit(Xi, yi)

Each iteration of the Bayesian Optimiser, I add a new element to $Xi$ and $yi$ and fit $gp$, the Gaussian Process, again. As the size of $Xi$ and $yi$ reach over 200, the time taken to fit the data becomes noticeable, this decreases the overall efficiency of my Bayesian Optimiser.

The Bayesian Optimiser has to make around 1000 iterations, each iteration needs to fit the data all over again. So, I was wondering if there's a way to incrementally fit the data? If that's not already being done by sklearn's GaussianProcessRegressor.

Thank you!


texts: Image data augmentation with this function

texts:
Following function is from http://scikit-learn.org/stable/auto_examples/neural_networks/plot_rbm_logistic_classification.html#sphx-glr-auto-examples-neural-networks-plot-rbm-logistic-classification-py where it is used to augment digits data:

Its documentation says:

    This produces a dataset 5 times
    bigger than the original one,
    by moving the 8x8 images in X
    around by 1px to left, right,
    down, up

def nudge_dataset(X, Y):
    direction_vectors = [
        [[0, 1, 0],
         [0, 0, 0],
         [0, 0, 0]],

        [[0, 0, 0],
         [1, 0, 0],
         [0, 0, 0]],

        [[0, 0, 0],
         [0, 0, 1],
         [0, 0, 0]],

        [[0, 0, 0],
         [0, 0, 0],
         [0, 1, 0]]]

    shift = lambda x, w: convolve(x.reshape((8, 8)), mode='constant',
                                  weights=w).ravel()
    X = np.concatenate([X] +
                       [np.apply_along_axis(shift, 1, X, vector)
                        for vector in direction_vectors])
    Y = np.concatenate([Y for _ in range(5)], axis=0)
    return X, Y

Can this function be used for any kind of image dataset?


texts: Arguments in python fast.ai function that are not in the function definition?

texts:
I have been coming across function calls that use arguments that are not in the function definition. I would like to know how that works (i.e. how the compiler interprets this).

For example, this function call:

 interp.plot_confusion_matrix(figsize=(12,12), dpi=60)

uses the variables "figsize" and "dpi", but neither of them turn up in the definition of plot_confusion_matrix: help(interp.plot_confusion_matrix) gives:

plot_confusion_matrix(normalize: bool = False, title: str = 'Confusion matrix',
cmap: Any = 'Blues', norm_dec: int = 2, slice_size: int = None, **kwargs) -> None
method of fastai.vision.learner.ClassificationInterpretation instance
    Plot the confusion matrix, with `title` and using `cmap`.

Why is it not an error to use "figsize" and "dpi" in the function call if they are not arguments of the function? How does this work in python?


texts:
How can I calculate the proximity between two strings

Ex: I need to find all the names in unstructured text and also need to look if there is any given string (i.e 'title') nearby

Input:

  'Naomi Osaka defeated her hero Serena Williams, 6-2. 6-4, to win the
  U.S. Open on Saturday, the first major title of the 20-year-old’s
  career. But this was not the fairy-tale moment Osaka had worked
  toward.

  Williams, attempting to tie the record with her 24th Grand Slam title,
  was levied a game penalty in the second set during a shocking display
  of anger directed at the match’s umpire'

output :
                           title -proximity (No of words far)
Naomi Osaka           {first major title: 14, Grand Slam title: 42}
Serena Williams       {first major title: 10, Grand Slam title :38}

I have the list of names so I am able to look for the exact name using regex.


texts:
This phrase is from the "Rethinking the Inception Architecture for Computer Vision" paper. it says :

  Higher dimensional representations are easier to process locally within a network. Increasing the activations per tile in a
  convolutional network allows for more disentangled features. The
  resulting networks will train faster.

What does the phrase

  Higher dimensional representations are easier to process locally within a network"

mean?

Does it simply mean, a higher representation is achieved more easily by having more feature-maps in a layer? again indicating, having lots of feature-maps per layer, (more width) is more beneficial?
If so, does the next part indicate the same thing where it says :

  Increasing the activations per tile in a convolutional network allows
  for more disentangled features.

what does disentagled features mean in that regard?


texts: Is there a particular order in which to do feature selection and sampling?

texts:
I want to use feature selection and observation subsampling on my data, for several reasons:

feature selection for the usual motivations (reduce noise, decrease running time, etc.)
observation subsampling because I have strongly imbalanced data, and I want not to introduce bias towards the most prevalent class in downstream classifiers

My question is: is there a specific order in which I should do feature and observation selection? E.g. first feature selection then subsampling?


texts: How to compare the complexity of different RNN cells?

texts:
I want to compare three different types of RNNs to decide which architecture can handle my data best.

To do that, I want them to have the same complexity. Can I simply define the complexity by the number of trainable parameters? If not, why?

Suppose I can compute the network's complexity like this, how do I correctly compute the complexity of a vanilla RNN cell, a GRU cell and an LSTM cell?


texts: I want my index which is mixed up (i just sorted out ratings from highest to lowest) to be in ascending (sorted) order corresponding to my sorted ratings. How can I do that?


texts: How to use a deep learning algorithm to cluster image *styles* in an unlabeled data set?

texts:
I have a hard problem, and I'd be interested in hearing people's thoughts. I have a set of images depicting a large variety of phenomena, but only a few styles. The images are already labeled by phenomenon, but not by style. The distinction between image styles is largely based on what technology was used to produce the images. We want to use a clustering algorithm to separate the different styles.

For an analogous problem, imagine that you had a very large dataset of paintings, but with many kinds of different subjects and only a few styles. Further imagine that you don't care what the subject of the painting is, only about its stylistic features (e.g. watercolor vs oil vs acrylic, realistic vs photorealistic vs impressionist vs expressionist). How could you try to go about "telling" a deep learning algorithm to emphasize these stylistic features over what the subject of the painting is, without using any labels?


texts: Why is the correlation coefficient of a constant function with function input is not defined?

texts: Mathematically, in case of a constant function (f(x) = c) the correlation coefficient of a constant function with function input is not defined. Neverthless, from the function plot, we see that there is no correlation between the function input and output, so it must be returning 0.


texts: Classification: how to handle reviews/long english words in feature set with all other numerical features

texts:
I am currently working on an use case where feature set contains numeric values such as amount, as well as a review feature which contains long winded english text.
the english text will very well differ between train and test data.
eg 'i have seen and its good' , 'nto ok','timepass',etc

how do i combine the text feature set with numerical data and feed it to a machine learning model?

i will nt be able to use encoding , these text variables are not categorical values . they are varying .

import pandas as panda
from sklearn.feature_extraction.text import TfidfVectorizer

words = ['i hv paid','i dont like','its good','yum yum']

a = panda.DataFrame({'amount':[10,20,30,40],'word':words})

tf = TfidfVectorizer()

csr = tf.fit_transform(words)

#how do i now use my csr to feed both amount and word to my machine learning model


texts:
Could anyone help to derive equation (15) dL/dx in L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks ?

I found that the term inside the rectangle for the expression (12) is equal to zero.


texts:
Have you guys tried to compare the performance of TF-IDF features* with a shallow neural network classifier vs a deep neural network models like an RNN that has an embedding layer with word embedding as weights next to the input layer? I tried this on a couple of tweet datasets and got surprising results: f1 score of~65% for the TF-IDF vs ~45% for the RNN. I tried the setup embedding layer + shallow fully connected layer vs TF-IDF + fully connected layer but got almost same result difference. Can you guys give some opinion on how TF-IDF features can outperform the embedding layer of a deep NN? Is this case common? Thanks!

I've used unigrams and bigrams to produce the TF-IDF features


texts: what is darknet and why is it needed for YOLO object detection?

texts:
what is darknet and why is it needed for YOLO object detection ? I read that its a neural network written in C , but why is it needed for YOLO object detection  when we  have lot of machine learning framework,api like tensorflow,keras,pytorch .

Im trying to train yolo from git code and i could see they are using tensorflow/keras as well but not sure why darkenet is used initially for traning yolo  .

darknet/yolo algorithm


texts: How to calculate prediction error in a LSTM keras

texts:
I have an LSTM which I have constructed and run in keras using python.  I use this model to predict $n$ points into the future for a time series forecasting problem.

When I use a method such as ARIMA to make the forecast I am able to generate prediction errors for my predictions as the model is fit by minimising the MLE using AIC for example.

Is there a way that is currently supported in keras for me to generate prediction errors for my regression predictions?  If there isn't, is there a way that I can calculate it myself?


texts: Tensorflow F-RCNN first stage input and output strides

texts:
Trying to optimize performances on Tensorflow's faster_rcnn_resnet50 (from the model zoo), I'm currently working on understanding the full .config file they provide, and I'm having a hard time with the strides.

Here's the relevant part of the file, with bolded parts being things I'm unsure about:

image_resizer {
  keep_aspect_ratio_resizer {
    min_dimension: 1134
    max_dimension: 2016
  }
}
feature_extractor {
  type: "faster_rcnn_resnet50"
  first_stage_features_stride: 16
}
first_stage_anchor_generator {
  grid_anchor_generator {
    height: 28
    width: 31
    height_stride: 14
    width_stride: 15
    scales: 0.242
    scales: 0.621
    scales: 1.0
    scales: 1.739
    scales: 2.478
    aspect_ratios: 0.386
    aspect_ratios: 0.693
    aspect_ratios: 1.0
    aspect_ratios: 1.464
    aspect_ratios: 1.929
  }
}

What I would like to know is if I should be computing the feature map stride according to the anchors' dimensions and strides or if they're completely unrelated ? Also is there any restriction on the strides themselves, for example keeping them in powers of 2 like the original f-rcnn ?


texts: Understanding pseudocode for co-democratic learning

texts:
I am reading this blog from Sebastian Ruder blog-link, researcher scientist at Deepmind, and
having problems understanding this pseudocode for Democratic Co-learning. Can somebody help me understand what is going on; specifically, I'm just trying to understand line 1-6.

Description of Democratic Co-learning from blog:

Democratic Co-learning Rather than treating different feature sets as views, democratic co-learning (Zhou and Goldman, 2004) [18] employs models with different inductive biases. These can be different network architectures in the case of neural networks or completely different learning algorithms. Democratic co-learning first trains each model separately on the complete labelled data L. The models then make predictions on the unlabelled data U. If a majority of models confidently agree on the label of an example, the example is added to the labelled dataset. Confidence is measured in the original formulation by measuring if the sum of the mean confidence intervals w of the models, which agreed on the label is larger than the sum of the models that disagreed. This process is repeated until no more examples are added. The final prediction is made with a majority vote weighted with the confidence intervals of the models. The full algorithm can be seen below. M is the set of all models that predict the same label j for an example x.

I guess line 1-3 is the training of several models on labeled data. Is i each sample in the labeled dataset? Is mi a trained model using all the samples from the labeled dataset?
I guess that line 4-6 is going through each sample in the unlabeled dataset but what's j? Is it iterating through each label?

This line I'm especially confused about is M <- {i | pi(x) = j}. Is i being assigned the output of the model from the unlabeled sample and then being assigned to M and what is M?


texts: Why categorical cross entropy loss is not correlated with NLP scores?

texts: I'm training a deep network for image captioning which is consist of one CNN and three GRUs. During training epoch by epoch model loss (categorical cross entropy) decreases but when I'm measuring bleu,METEOR,ROUGE,CIDEr and SPICE scores,I get best ones in the first epoch that has worst loss. I don't get why this is happening? And if categorical cross entropy is not a suitable loss function for autoencoder then what should I use instead?


texts: Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?

texts:
Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the prediction is their average: $\frac1nT_1(\underline{x}_i)+...+\frac1nT_n(\underline{x}_i)$

In the book:

https://www.amazon.com/Ensemble-Methods-Data-Mining-Predictions/dp/1608452840

the authors have suggested that post-processing Adaboost with linear regression, they have witnessed improved performance across the board. Namely, they suggest doing linear regression with the new training data:

$((T_1(\underline{x}_1),...,T_n(\underline{x}_1),y_1),...,((T_1(\underline{x}_N),...,T_n(\underline{x}_N),y_N)$

and then use the resulting coefficients rather than the coefficients $1/n$ for the different trees.

This book was authored way back in 2010, and XGBoost, so far as I can tell, does allow this post-processing. So I was wondering what's going on:

Is it that this method was largely found to be unhelpful?
Is it that this method is not well known enough yet?
Or -- is this method used all the time and I just don't know it?


texts: Does Convolution kernel size affect number of channels?

texts:
I am going through Dilated Residual Network blog post. In this, Under 2.Multi-scale Context aggregation heading, author mentioned this.

  The last one is the 1×1 convolutions for mapping the number of
  channels to be the same as the input one. Therefore, the input and the
  output has the same number of channels. And it can be inserted to
  different kinds of convolutional neural networks.

I thought, we decide number of channels in the next layer and kernels will be initialized randomly. These kernels shape is decided by us which are 1x1 or 3x3 etc., So, what did author mean when he said, 1x1 convolutions for mapping the number of channels to be the same as the input one.When, Even if its 2x2 convolutional kernel, number of channels are not changed.


texts: Recommendation Engine - Content based and Collaborative recommendation?

texts:
I am building a recommendation system for hotel accommodation. I scraped data from online booking portal and now my data has Name of the hotel, review, description and location.

I built a simple content-based recommendation. I combined the hotel name, location and description. Then, used NLP and TF-IDF to convert into vectors. I used cosine similarity for recommending the hotel name.

I have taken a hotel name as an input and model recommends the hotels. I have the following questions,

1) If I wanted to find hotel recommendations based on keywords instead of hotel names, do you have any suggestions on how I can do it? For example, Instead of giving hotel name, can I give the location name or typing IBIS (IBIS Singapore - > is a complete hotel name)

2) Second, I don't have a hotel ID, how to do Collaborative recommendation system with my present dataset?

Please advise


texts:
I have 600 customers per week. How many do I need to survey to get a good sample to work with?

I'm very new to stats.


texts: Accuracy on Validation and Test set, Overfit?

texts:
Just a quick question, I am building a ML model right now however I am receiving very similar (72.2 and 72.4 for example)% for both Accuracy and F1-Score on my Validation Dataset and my unseen Test Set respectively. This is occuring on most of the baseline models I have produced for my problem right now.
Is this showing that my model is completely overfitting or just acting completely random and getting lucky.
Thanks


texts:
I have been reading about Generative Adversarial Networks (GANs) and was wondering if it would make sense to train a generator function only to use it for creating more training data.

In a scenario where I don't have enough training data to build a robust classifier, can I use this limited data to train a generator that'll produce samples good enough to improve the accuracy of my discriminator (classifier)?


texts: Designing Custom Kernel from my Mathematical model

texts:
I derived a mathematical model for a porous system and the final function looks like this ,

after going through the Mercers Theorem and its condition for a kernel I would love to write an SVM kernel with this equation on python and use it for predictions, please can someone help me with resources that can assist me in going about this or the equations python code as a kernel.


texts:
I'm trying to create an analytical model for performance of a greedy scheduler in a particular domain.

I've created a preliminary model, but I think it's very confusing to others and was wondering if someone could provide feedback.

My problem domain follows. A greedy scheduler has the capacity to schedule c tasks and will select the c highest rewarding tasks. Tasks arrive dynamically online from an arrival pattern, A--that is for every time unit within a time horizon X tasks will arrive, where X is an arbitrary random variable. Task details are all drawn from an arbitrary predefined distribution. Task details include, reward, maximum waiting time before being serviced (i.e if a task is not serviced after m time units, the task will fail), and duration. I'll label the respective distributions as R for reward, M for maximum waiting time, and D for duration.

I know this problem is very large, so I would like to focus on a much smaller component of this larger problem.

Consider a greedy agent with capacity c and no tasks currently scheduled with a task mix of T, where the size of T is larger than c and T's task's rewards are representative of the R reward distribution.

My question is what should be the expected reward (or even better the distribution of rewards) for the c selected tasks from T. This is similar to a k-order statistics problem; however, I've been unable to format this problem into a k-order statistics problem due to my basic/limiting understanding of statistics. (I study computer science and only know basic statistical analysis).

My approach follows; however, I feel it could be improved. In my approach, I try to calculate the lowest reward of the highest rewarding tasks:

If the scheduler can choose c tasks from T and c < |T|, the scheduler will choose the highest rewarding c tasks. Since T's task's rewards are representative of R, The expected reward of a single task will be the expected reward of the upper c / |T| portion of the R distribution.

To calculate the expected value of a reward distribution, one takes the

integral from x= -infinity to x=+infinity of P(R=x)*x

Similarly, to calculate the expected value of a distribution of the highest c rewards, we take the

integral from x = Y to x = +infinity of P(R=x)*x
 where y is the lowest of the highest rewarding task's rewards.

To calculate the upper c/ |T| portion of the R distribution, I let the lowest reward of the highest rewarding task be Y and thus c / |T| = 1 - CDF(Y), where CDF is the cumulative distribution function applied to the reward distribution.

c / |T| = 1 - CDF(Y) states that:

The proportion of schedulable tasks to total tasks is equal to the proportion of the upper half highest rewarding tasks of the reward distribution.

By solving for Y, we get that

Y = inverse_CDF(1 - c/|T|)

We can then plug the highest rewarding task's reward into our expected value equation earlier and then the expected value of the highest c rewarding tasks are then the integral from x = Y to x = +infinity of P(R=x)*x.

Any feedback on how I can improve clarity or improve my analytical equation would be appreciated or if there is a more standard/correct way to describe what I'm doing any suggestions would be appreciated. Also, I'm unsure if this community is the correct community to post this question. If it is not, please provide suggestions as to where I should post this question. I've looked at similar work, and they typically use omniscient scheduling and I'm not too understanding of their syntax/approach.


texts: Visualization of SeqSelfAttention layer from Keras

texts:
Can someone help me to find, explain and visualize the SeqSelfAttention layer from Keras. I found a lot of flowchart that use the figures from the "attention is all you need paper" where Adresses, Keys and Values are used. But never found any figure that related to the formulas below by the OpenTag paper

Can anyone help me on flowchart of clear figure that explains the working mechanism of a simple implemention of the SeqSelfAttention layer with the formulas?


texts: Ship detection on high resolution unlabeled image dataset

texts:
I have been given a dataset of 4000 high resolution photos of  different size of ships entering a port, that need to be clustered in order to execute the following tasks
1)Ship detection
2)Count the number of ships in image
3)Find if the ship is parked or not
I am new to object detection and image recognition so i have the problem that the dataset is not labeled so i have to labeled to accomplish the above tasks.I am asking for guidance from people that faced similar problems


texts: Seeking advice on database architecture -- given my problem, what tools should I learn?

texts:
I'm a fairly experienced R user, but until now I haven't had a good reason to learn to use databases.  Now I have a problem where I am dealing with model output that I need to save to disk, and then query for another process.  If the data were smaller, I'd store everything in a list, with hierarchical elements.  For example, if my object is called output.OLS:

1> summary(output.OLS)
        Length Class  Mode
SEP0307 3      -none- list
SEP0308 3      -none- list
SEP0306 3      -none- list
SEN0308 3      -none- list
<<snip -- hundreds of sub-lists>>
SEN0307 3      -none- list

the second level of hierarchy:

1> summary(output.OLS[[1]])
      Length Class  Mode
c1  11     -none- list
c2  11     -none- list
c2  11     -none- list

the third:

1> summary(output.OLS[[1]][[1]])
         Length Class      Mode
Baseline 1008   data.frame list
CanSEM45 1008   data.frame list
CanSEM85 1008   data.frame list
CCSM45   1008   data.frame list
CCSM85   1008   data.frame list
GISS45   1008   data.frame list
GISS85   1008   data.frame list
HadGEM45 1008   data.frame list
HadGEM85 1008   data.frame list
MIROC45  1008   data.frame list
MIROC85  1008   data.frame list

and we can see that now we're looking at data.frames

1> dim(output.OLS[[1]][[1]][[1]])
[1]   33 1008
1> colnames(output.OLS[[1]][[1]][[1]])[1:10]
 [1] "scen"     "year"     "region"     "size"    "amount"     "mean"
 [7] "semean"   "varres"   "result.1" "result.2"
<<snip>>
[1006] "result.998"  "result.999"  "result.1000"

There are four such output objects, for a total of about 6GB.  They don't all have the columns mean, semean and varres

This list is about 1.5 gigs, which is unwieldy given what I want to do with it.  It'd be much better to read subsets of it from a database.  I can think of two potential architectures:

Architecture 1: Single, huge table
The variable names would be region (the outermost hierarchy), cx (i.e.: c1, c2, c3), model (Baseline, CanSEM, etc), and then all of the results: result.1, result.2, ..., result.1000

This would be most convenient.  The problem is that the table would be both deep and wide.  Too wide for SQLLite, which is the simplest database, as far as I can tell.

Architecture 2:: one keyvalue table and many data tables.  The keyvalue table would have the following columns:
cx, scenario, year, place, keyvalue = coerce_to_integer(unique(scenario, year, place, cx))

The data tables would look like this
keyvalue, result_number, result

This option uses hierarchy and is actually a relational database (relational via the keyvalues, right)?

But it has the drawback that it nearly triples the size of my data:  for each datum, I need to store a keyvalue and an index (result_number).  This seems like it wouldn't make sense.

Are there other options
For example, are there databases similar to R's lists that can be stored in memory, and read from memory?  Is that sort of like what NoSQL databases are?

So, my question

What database architectures would make sense for this problem?
Where should I start learning about these sorts of databases?
How does R interface with them?  Can I create the database from R?  If so, how?


texts: Insights betwwen two columns/variables in Dataframe

texts:
I have data in two columns one is range of old credit score (Input score range) and new credit score (cvsc100). How do i find insights from both of them ? where the old is range of values and other column is not(CVSC100)

I know how to calculate Pearson Correlation in Python of Dataframe of two column . but that should not be sufficient i believe. How should i proceed can you please advise


texts: What loss function avoids overconfidence?

texts:
In the case of a neural net with a relatively small training data set, doing simple classification with categorical cross entropy (log loss), it is very easy for the results of the network to be "overconfident" because that improves log loss on the training data.

What I mean by overconfident (this is partly intuition and partly something that can be derived from Bayesian stats): given 1000 training examples in 2 equal classes, it is unreasonable for any classifier to give a probability of a new input as being in one class or the other which is substantially higher than (1-1/1000), even if the classifier is perfect on the training data.  That is because if the training set was increased by just one more example, the 1001st example could break whatever rule seems to have been followed in the data set up to that point, and that would force all the predicted values/probability estimates to be updated accordingly.

However, in practice trained neural nets with log loss function will output values which are arbitrarily close to 0 or 1 given a long enough training run, because that continues improving their performance on the training data way past the point that it makes sense to do so based on the size of the training data.

The question: how can the usual log loss function be modified so that making predictions extremely close to 0 or 1 does not hugely improve log loss compared to predictions which are only somewhat close?


texts:
I have a dataset with monthly revenue per customer. I want to build a model that can try to predict if the customer will exceed $10,000 3 months out (yes/no).

While this seems like a traditional ML problem I have an important questions

Should I build my dataset with one row per customer id and let the label be the revenue 3 months out
Should I instead have one row per month per customer and let the label by the revenue 3 months out

Thanks


texts: Calculating errors of convolutional filters with differing number of channels in input and output

texts:
I am trying to implement a convolutional neural network with the following architecture

Input image is (1X28X28). This just has one channel
3 Convolutional Filters, each with the dimensions (1X5X5) with a stride = 1. This is to convolve the image and to represent it in three channels. The resulting output will be (3X24X24)
Max Pool Layer with (2X2) dimension and stride =2. This results in output (3X12X12)
This is then followed by a fully connected layer which has (12X12X3=432) nodes.
A fully connected layer with 80 nodes
An output layer with 10 nodes.

I am able to understand how the errors of each fully connected layer and the errors of their weights are calculated. The errors in the first fully connected layer, are then projected back to the first convolutional output layer, in positions where max values were retrieved via max pooling.
This would correspond to the error of the convolutional output. This would have the dimension (3X24X24). To find the error of the convolutional filters, we need to do convolution between the input (1X28X28)(after rotating each channel by 180 degrees), and the error of the the convolutional output(3X24X24).
Given that the channel dimensions do not match here, should we do zero padding for the input so that dimension of the input becomes (3X28X28), thus resulting in the convolutional output of (3X5X5)?
Its 3X5X5 because (28-24)+1 = 5


texts: What is Extreme Learning Machine? Why tuning of weights is not required?

texts:
The wikipedia says:

  Extreme learning machines are feedforward neural network for classification, regression,
      clustering, sparse approximation, compression and feature learning with a single layer or
      multi layers of hidden nodes, where the parameters of hidden nodes (not just the weights
      connecting inputs to hidden nodes) need not be tuned.

How is it possible to assign values to links without actually seeing the Training set?

Till now from my research to understand this, I've made a lead, but not satisfactory because I can't anything from this below paragraph.

  For any nonconstant piecewise continuous function
  that is used as the activation function, if the parameters of
  the hidden neurons are tuned, then the function can make
  the SLFNs approximate any target continuous function f (x).
  Then, according to any continuous distribution probability, the
  function sequence {h subscript i (x)} where i goes from 1 to L   can be randomly generated, and
  it has the universal approximation capability, which means that
  $$\lim_{L→ \infty} ||L
i=1 βi hi (x) − f (x)|| = 0$$
  holds with probability
  of one with appropriate output weights $β$.

SOURCE: A Parallel Multiclassification Algorithm for Big
Data Using an Extreme Learning Machine, IEEE transactions on neural networks 2017

Can anyone prove/explain this with an example please?

Also any extra knowledge on extreme learning machine is very welcomed.


texts: Taxonomy of recommender system methodologies

texts:
There's tons of material online but yet I can't reconcile the different definitions for recommender system methodologies / strategies.

I think we can identify several axes:

memory vs model based; Model based models do not need to store historical data such as user ratings or proxies (implicit data) such as purchase behavior, views, etc.
product based vs collaborative filtering, aka user-based filtering. Collaborative filtering relies on user preferences data (implicit or explicit - explicit when users explicitly provide ratings, likes, stars). Product based rely uniquely on product features.

collaborative filtering comprises item based, which is totally based on user-item rankings, and content-based recommendation, which exploits product features and user preferences (although other sources put content based filtering outside the family of collaborative filtering)

Is this correct? Anything (big) missing?


texts: Pyspark.read from MariaDB fetches duplicate header rows instead of data

texts:
It's a very strange issue- instead of fetching data like expected, it returns every row as a duplicate of the header. Every cell is a literal string of the column name.
If I don't use the query like below and instead just put the table, it errors out trying to cast the literal column name as its datatype, e.g. it will try to cast literally "dob" as a date. I'm posting because I don't see anything about similar issues elsewhere online.I've tried multiple versions of the JDBC driver, so I'm not sure where to go from here.
Code to reproduce (network and user info removed ofc):
##import required libraries
import pyspark

##create spark session
spark = pyspark.sql.SparkSession \
   .builder \
   .appName("Python Spark SQL basic example") \
   .config('spark.driver.extraClassPath', "/Users/username/Downloads/mariadb-java-client-2.7.3.jar") \
   .getOrCreate()

##read table from db using spark jdbc
patients_df = spark.read \
   .format("jdbc") \
   .option("url", "jdbc:mariadb:my_server:3306/SYNTH_EHR_NORM") \
   .option("dbtable", "(select fname, lname from SYNTH_EHR_NORM.PATIENTS limit 20) tmp") \
   .option("user", "username") \
   .option("password", "not_my_real_password") \
   .option("driver", "org.mariadb.jdbc.Driver") \
   .load()

##print the users dataframe
print(patients_df.collect())

which returns:
[Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname'), Row(fname='fname', lname='lname')]


texts:
I came across an earlier post that was resolved and had a follow up to it but I couldn't comment because my reputation is under 50. Essentially I am interested in calculating the denominator in Naive Bayes.
Now I understand that the features in Naive Bayes are assumed to be independent so could we calculate $p(x) = p(x_{1})p(x_{2})...p(x_{n})$ or would we have to use this formula $$p(\mathbf{x}) = \sum_k p(C_k) \ p(\mathbf{x} \mid C_k)$$ with the conditional independence assumption that$$ p(\mathbf{x} \mid C_k) = \Pi_{i} \, p(x_i \mid C_k) $$
My question is would both ways of calculating give the same p(x)?
Link to the original question : https://datascience.stackexchange.com/posts/69699/edi
Edit** : Sorry I believe the features have  conditional independence, rather than complete independence. Therefore it is incorrect to use $p(x) = p(x_{1})p(x_{2})...p(x_{n})$?
Lastly, I understand we don't actually need the denominator to find our probabilities but am asking out of curiosity.


texts: Small project ideas for Machine Learning

texts:
I need some serious help. I am supposed to implement a project (Non-Existing as of now) for my Machine Learning course. I have no basics in AI or Data mining or Machine learning. I have been searching for a while and unable to find something that i can finish implementing in 3-4 weeks time. It carries a huge chunk of my final marks and no matter how much i try i am unable to understand how it works!
Can the machine learning masters please help me out with this. I need a project suggestion to start with. And i want to know how to proceed after gathering the data set. I am totally blank and running out of time for my graduation :(

Appreciate your suggestions! Thanks in advance.


texts: How to use live data to improve a existing model?

texts:
I am using logistic regression to train a model to predict 'click/non-click' using ['browser info', 'publisher info', , 'location', 'time', 'day'].

I wanted to know the ways in which I can use the new live data to improve the improve the already trained model.

Does a solution exist which takes into account - change in feature set?


texts: What is a good method for detecting local minims and maxims?

texts:
I'm using kernel density estimation in order to compute
probability density function for item occurrence.
Using this output, i want to find all the local minims and maxims.
I'm interested in different methods for local-min/max extraction.


texts: Please let me know if I am on the right track to being an NLP Expert

texts:
I am a college student(rising senior) and became interested in Natural Language Processing last semester. I decided to focus on studying this area this summer and become skilled in this area. I wanted to get some advice for studying this particular subject.

Right now, I am taking Andrew Ng's Machine Learning course on Coursera to get a sense of how Machine Learning works. After finishing this course, I am planning to take Standford's CS224n NLP course on Youtube and do its class activities. I am assuming AWS and Tenserflow is also important since they are included as topics of CS224n.

I want to know if my summer plan sounds reasonable. If not, could you please give me an advice of how to make a better plan. If this sounds reasonable, it would be great if you can add more or specify which part is particularly important in these areas.


texts: Merging multiple data frames row-wise in PySpark

texts:
I have 10 data frames pyspark.sql.dataframe.DataFrame, obtained from randomSplit as (td1, td2, td3, td4, td5, td6, td7, td8, td9, td10) = td.randomSplit([.1, .1, .1, .1, .1, .1, .1, .1, .1, .1], seed = 100) Now I want to join 9 td's into a single data frame, how should I do that?

I have already tried with unionAll, but this function accepts only two arguments.

td1_2 = td1.unionAll(td2)
# this is working fine

td1_2_3 = td1.unionAll(td2, td3)
# error TypeError: unionAll() takes exactly 2 arguments (3 given)

Is there any way to combine more than two data frames row-wise?

The purpose of doing this is that I am doing 10-fold Cross Validation manually without using PySpark CrossValidator method, So taking 9 into training and 1 into test data and then I will repeat it for other combinations.


texts: What is the difference between "offline trained model" and "pretrained model"?

texts: I am confused that both are same or not, and then how can I differentiate with the online training model.


texts: I want to detect similar meaning in sentence "in my final year web based system"

texts:
When students submit a topic for his/her final year research, the system should be able to detect that this topic was carried on by some student or already exists.

How can I be able to do it? If its a model, how can embed python in php database?


texts: Mean-variance mapping optimization (MVMO) in R

texts: Someone know tell me if there is any package in R about Mean-variance mapping optimization (MVMO) algorithm? I already researched, but don't find anything about this.


texts:
Is there a way to find the number of clusters in mapper.map?
It is a module in kmapper.KeplerMapper.
When I plot the graph generated by mapper.map with matplotlib, I can see roughly the number of clusters. When I visualize it in jupyter notebook via HTML, I can see the cluster and the details.
However, I was unable to find out the overall number of distinct clusters.


texts: Class Imbalance and Cost-Sensitive Learning XGBoost

texts:
I'm fairly new to data science and machine learning and have been trying to read a bit more on methods like boosting for one of the projects I am working on. The investigator on this project is interested in utilizing xgboost for this project, but there are a couple of main issues that we anticipate once we get the data.

There will be class imbalance, as in way fewer 1's than 0's. I've seen that people handle this by using the scale_pos_weight parameter in xgboost, but from my understanding, xgboost allows for custom objective functions. I was wondering how people come up with custom objective functions and the basis behind their chosen objective function. I'm interested in a customized objective function that penalizes false negatives more and was wondering if there's anything out there on the web that's recommended. We are dealing with a classification problem, so by default we would be using a log-loss function. I am hoping there is some sort of custom log-loss function out there that is commonly used.

I apologize if I am using the wrong terminology, as I'm still very new to this. Thanks so much!


texts: Importance Sampling in Off-policy n-step Sarsa

texts:
In Chapter 7.3 of Reinforcement Learning: An Introduction by Sutton and Barto, the off-policy pseudocode has the following update equation for $Q$:

Compute importance sampling ratio:
$$ \rho \leftarrow \prod^{\min(\tau+n-1, T-1)}_{i=\tau+1} \frac{\pi(A_i \mid S_i)}{b(A_i \mid S_i)} $$
Compute truncated return
$$ G \leftarrow \sum^{\min(\tau+n, T)}_{i = \tau+1} \gamma^{i-\tau-1} R_i $$
Compute discounted estimate:

$$ \text{If } \tau+n < T, G \leftarrow G + \gamma^n Q(S_{\tau+n}, A_{\tau+n}) $$

Update $Q$

$$ Q(S_\tau, A_\tau) \leftarrow Q(S_\tau, A_\tau) + \alpha \rho [G - Q(S_\tau, A_\tau)] $$

($\tau$ is the time whose estimate is being updated, $t$ is current timestep, $n$ is $n$-step return, and $T$ is termination timestep)

I believe that the importance sampling ratio $\rho$ should only be multiplied to $G$ and not to $Q$, since $Q$ is for target policy $\pi$ and $G$ is from behavior policy $b$. In other words, this is how I see this equation:

$$ Q_\pi \leftarrow Q_\pi + \alpha \rho_{b \to \pi} [G_b - Q_\pi]$$

Am I correct in thinking that this is a typo, or did I miss something?

Thank you for your help!


texts: What to do after selecting a model with cross-validation?

texts:
I have been building a neural network for classification.

To select my best model. I have been using 10-Fold cross validation. and selected the network that gives the highest mean accuracy.

Now that I have selected the best model, I want to use all the data I have to train this model because the amount of data I have is limited (I will merge training, dev and test data).

My issue is that, when training with all the data, I don't know when to stop training. Training loss is not an indicator for sure. Usually, I have a development set that I use to monitor training. When the training loss does not improve anymore, I stop training.

Any suggestions on how to supervise a model with only training data? In other words, how to tell when the network needs to stop?


texts: Reducing the training time of an RL agent

texts:
I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my agent to train for some 1000000(to achieve convergence) episodes, its becoming computationally infeasible, Is there a way anyone is aware to speed up my training process, like using parallel threading or using cuda. Or is it something because of the algorithm?

my episode here basically is of a one day long with actions taken at every 15 mins. I am using FMIs for the simulation and to finish this 1 episode it takes 10 mins. I can reduce my episode time to few seconds by using reduced order models/equations. This decreases the episode length(time). But my question is are there ways to speed up the training process. Or there better algorithms than DQN which donot reqire this much of training.

Thank you.


texts:
I'm copying an example directly out of a book I am working through, and I currently getting this error:

InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_10' with dtype float and shape [?,2]
 [[{{node Placeholder_10}} = Placeholder[dtype=DT_FLOAT, shape=[?,2], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

My tensors and placeholders are declared like so:

XOR_X = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]]
XOR_Y = [[0.0], [1.0], [1.0], [0.0]]

num_input = 2
num_classes = 1

x_ = tf.placeholder("float", shape=[None, num_input], name='X')
y_ = tf.placeholder("float", shape=[None, num_classes], name='Y')

And the line producing the error is here:

_, summary = sess.run([train_step, merged_summary_op], feed_dict={x_ : XOR_X, y_: XOR_Y})

What exactly is the issue here?

for context, I attach the entire implementation here:

import tensorflow as tf

XOR_X = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]]
XOR_Y = [[0], [1], [1], [0]]
# XOR_Y = [0.0, 1.0, 1.0, 0.0]

num_input = 2
num_classes = 1

x_ = tf.placeholder("float", shape=[None, num_input], name='X')
y_ = tf.placeholder("float", shape=[None, num_classes], name='Y')

#Model structure
H1 = tf.layers.dense(inputs=x_, units=4, activation=tf.nn.sigmoid)
H2 = tf.layers.dense(inputs=H1, units=8, activation=tf.nn.sigmoid)
H_OUT = tf.layers.dense(inputs=H2, units=num_classes, activation=tf.nn.sigmoid)

#Define cost function
with tf.name_scope("cost") as scope:
    cost = tf.losses.log_loss( labels=y_, predictions=H_OUT)
    # Add loss to tensorboard
    tf.summary.scalar("log_loss", cost)

with tf.name_scope("train") as scope:
    train_step = tf.train.GradientDescentOptimizer(0.1).minimize(cost)

merged_summary_op = tf.summary.merge_all()

# Initialize variables(weights) and session
init = tf.global_variables_initializer()
sess = tf.Session()

# Configure summary to output at given directory
writer = tf.summary.FileWriter("./logs/xor_logs", sess.graph)
sess.run(init)

# Train loop
for step in range(10000):
    # Run train_step and merge summary op
    _, summary = sess.run([train_step, merged_summary_op], feed_dict={x_ : XOR_X, y_: XOR_Y})
    if step % 1000 == 0:
        print("Step/Epoch: {}, Loss: {}".format(step, sess.run(cost, feed_dict={x_ : XOR_X, y_: XOR_Y})))
        writer.add_summary(summary, step)


texts:
The target is a probability between N classes, I don't want it to predict the class with the highest probability but the 'actual' probability per class.

For example:

|    | Class 1 | Class 2 | Class 3 |
------------------------------------
|  1 |     0.9 |    0.05 |    0.05 |
|  2 |     0.2 |     0.8 |       0 |
|  3 |     0.3 |     0.3 |     0.4 |
|  4 |     0.7 |       0 |     0.3 |
------------------------------------
|  + |     2.1 |    1.15 |    0.75 | <- correct this imbalance?
| >0 |       4 |       3 |       3 | <- or this one?

Some classes have 'more' samples in the sense that the sum of probabilities is higher than other classes. Do I have to balance this out with weights in the loss function? Or do I only correct for the imbalance in >0 as normally?


texts: Model selection for multiple groups to maximize overall f1 score

texts:
Model 1
Model 2
..
Model m

Group 1
tp11/act1/pred11
..
..

Group 2
tp21/act2/pred21
..
..

..
..
..
..
..

Group n
tpn1/actn/predn1

..
tpnm/actn/prednm

I have a multiple groups model selection for classification problem, and my target is to maximize the overall f1 score.
Let's say that I divide the data into N groups, and I have developed M models to predict. The question is, I want to find the optimal combination which maximizes the overall f1 score, such as group 1-> Model 3, group 2-> Model1...
The definition of the overall f1 score = 2*TP/(ACT+ PRED) where TP= the total sum of true positives of all groups, ACT= the total sum of actual labels in all groups, and PRED= the total sum of the predicted number of all groups.
The information (above table) I have is true positives, actual labels, and the predicted number of each group with each model.(e.g. tpij= the number of true positive in group i with model j, acti= the number of actual labels in group i, predij = the number of predicted in group i with model j)
For example, if I have three groups with three methods and the combination is "G1->M2, G2->M3, G3->M1", then the TP=tp12 +tp23+tp31; ACT= act1 + act2 + act3; PRED=pred12 + pred23 + pred31; overall f1 = 2*TP/(ACT+PRED).
To find the optimal combination by brute force, I need to calculate m^n times, and this is impossible to run out when the number of the models and groups is enormous. If I choose greedy optimization, the first step I would find the best model for all the data. After that, in each step, I find the best model in each group that can maximize the overall f1 score and other groups remain using the previous methods till I loop over all the groups; then the calculation times drop to m*n. But I am sure that this is not the optimal solution. I have also thought of the Viterbi algorithm by treating f1 as a distance. Still, in each step, the numerator and denominator are changing, so I cannot simply treat the maximum f1 score path as a regular shortest path.
Is there any method to find the optimal combination without a run through m^n times, and even if there isn't, could you please provide any proof. I can't find any combination better than the combination found by the greedy method. Is this already the optimal solution for this question? Is there any key words to search for similar problems (discrete optimization?).


texts: How to speed up optimization using Differential Evolution?

texts:
My application is high frequency trading.  My data are time series of the bid and ask prices of a stock recorded on every tick (change in price). For each data point I also have a certain indicators that predict the future movement of the price. The indicators have different horizons of the predictions, some being optimal at few second intervals and others few minutes. I need to assign these predictors weights and based on whether the linear combination crosses a threshold, the decision will be taken to buy of sell the stock. So far I have tried the Differential Evolution (DE) method to figure out the weights. I use a black box model with the weights vector $w_i$ and threshold as inputs. For each data point I have a vector of indicators $\alpha _i$.
$$
total\_alpha = \sum\alpha _i*w_i
$$
If
$$
total\_alpha > threshold, BUY
$$
Else If
$$
total\_alpha < -threshold, SELL
$$
The output of the model is the sum of difference between each between the price of each consecutive buy and sell. This output is being optimised by the DE algorithm.

I am having trouble with the computational aspects. My data is very large (~$7 \times 10^8$ rows by 20 columns), and thus the execution time for the DE algorithm is unacceptable.

My question: Is there a better and a faster way to solve this problem?


texts: Predicting complete time series for given parameter sets

texts:
I am searching for an approach that is able to predict a complete time series for a given parameter set.
Imagine a robotic arm which has a starting position and a target position. There is  a sensor mounted to the top of the arm. For random start and target positions I can record the sensor values as a time series. Now, I would like to predict the sensor values for a previously unknown start and target position.
Currently, the only approaches I can find try to forecast the next values for a given time series. Other approaches aim for generating new sequences for a given set of sequences, which is close to what I would like, but without the possibility to connect the sequences to parameter sets.
Does anyone have an idea?


texts: Using Logical Functions in Tableau Prep for Address string editing

texts:
I have a data set with A LOT of inconsistent addresses. For example, in some cases you have STREET versus ST and "Fourth Ave" versus "4th Ave." I have been using Tableau Prep to clean these addresses into the same format as another data set to join. I would like to use some sort of calculated, logical function to clean out many of these inconsistencies throughout, such as CASE. However, I run into the problem of it not wanting to replace just the one string word case, but predictably returns a null (see block on my initial attempt).
CASE[simple address]
    WHEN 'AVENUE' THEN 'AVE'
    WHEN 'STREET' THEN 'ST'
    ELSE ''
END

Does anyone have any suggestions on how to better phrase this logical function to replace all the instances of inconsistencies? I was hoping to do it in one field instead of splitting each address element into new columns (ie house_number | street_name | street_direction) and rerunning calculated fields for each. Thanks in advance for any resources/help!


texts: How to use an existing model as in input into a new model

texts:
We have a click-model which is currently being used for search ranking in production, and I want to create a new model which takes the old model click probability as one input and adds some other variables in too. The problem is that training data will positionally biased by the fact that the probability of a click is correlated to the old model's prediction.

My plan is to introduce a penalty factor on the original model's prediction to ensure that it doesn't dominate the new model (eyeballing the results to decide on an appropriate penalty factor). Is this approach valid or would there be a better way to approach this?

Note that I don't want to rebuild the old model with the new variables because

The existing model takes a long time (days) to build
The new model and old model will be deployed separately, i.e. the old model will be scored offline/batch whereas the new model will be scored real-time


texts: How to understand conv layer to another same conv layer in VGG16?

texts:
VGG16 struct is:

Img->
Conv1(3)->Conv1(3)->Pool1(2) ==>
Conv2(3)->Conv2(3)->Pool2(2) ==>
Conv3(3)->Conv3(3)->Conv3(3)->Pool3(2) ==>
Conv4(3)->Conv4(3)->Conv4(3)->Pool4(2) ==>
Conv5(3)->Conv5(3)->Conv5(3) ====> FC

The flow :
http://ethereon.github.io/netscope/#/gist/dc5003de6943ea5a6b8b

In keras code https://github.com/keras-team/keras/blob/master/keras/applications/vgg16.py

# Block 1
x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1')(img_input)
x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x)

....

It looks like just pass Conv1_1 result to Conv1_2, and the rest too.

I can understand this:

Img->
Conv1(3)->Pool1(2) ==>
Conv2(3)->Pool2(2) ==>
Conv3(3)->Pool3(2) ==>
Conv4(3)->Pool4(2) ==>
Conv5(3) ====> FC

But don't understand why Conv1_1 can connect to a same layer - Conv1_2 :

For example:

Img: 224x224x3
kernel:3x3
depth:3 (channel)
Conv1_1: use 64 kenels. So there are 64 3x3x3 kernel used to scan 224x224x3(with padding 1) ,  then get Conv1_1_feature_map 224x224x64 .

Here, does it mean:

Conv1_2: use 64 3x3x64 kernel to scan Conv1_1_feature_map 224x224x64 , then get Conv1_2_feature_map 224x224x64 ??

If it is, could you explain what is the benifit ? I can't understand the meaning of this flow clearly. It is not clear that why Conv1(3)->Conv1(3)->Pool1(2)  is better than Conv1(3)->Pool1 . On my feeling, another same size Conv looks like just make previous layer output more blurred/ambiguous, doesn't like Pooling layer concentrate previous Layer's output feature.


texts: Extract Pattern using Short Text Processing

texts:
We are facing issue in our project. We have a data set of around 25000 rows, we have a column name title, it contains text data and we have a score column in the data set.We want to use Machine Learning techniques to know what are the factors that make the label High, by factor i mean what are the key words etc.., as use can see we have a short text so what is the most proper technique to extract the maximum knowledge related to the label from this short text,I did some pre processing on the text by cleaning it then extracting ngram features using tf-idf weighting function, then i tried using decision trees algorithm for classification of text
For example:

   Score Label                       Tactic Title
1.  High                        Opportunity Movement.
2.  Low                         Partner Launch.
3.  High                        Implement Mix panel stories.
4.  Low                         Improve app performance and reduce multiple API Calls.
5.  High                        Review Comments.
6.  Low                         Support SimpleStarta Team Goal.


texts: Regime classification pre model training?

texts:
i don't have a formal background in this so please bear with me.
This is what my dataset looks like :

I'm interested in modeling the first variable using the rest as explanatory.
A simple OLS yields a more or less satisfying model but inuitively, i know that there are k (between 2 and 4) regimes where the correlation between my Y and certain variables switch completely making the one full OLS across the dataset sort of irrelevant as we sort of smooth all of these regimes by doing this.
I attempted differentiating the regimes using correlation similarity with a k-means algorithm in R, the result is the coloring in the pictures.
i was hoping that dividing the dataset this way would reveal more "straight lines" where the OLS would be more "fitting".
I wonder if i'm going in the right way, would be grateful if anyone could point to better ways at achieving the fundamental problem : Modeling Y~X while taking into account that there are different regimes which require a likely different coefficient or (model altogether ?)
EDIT : I also tried MARS models but the result is not satisfying at all from a "physics" sense of how these variables are supposed to actually work together + changing the training set slightly yields changes the coefficients too much.


texts: How do I compare coefficients from my glm when I have more than one factor variable in my formula?

texts:
I am trying to model a binary outcome in R that has many independent variables. 5 of the Ivs are factors with more than two levels. When I try to remove the intercept it only does it for one of the independent variables. I would like to see if any of the coefficients for the levels are significantly different from any of the others within that variable(and it would be nice to see if any of the levels are significantly different from the average). When I do a summary of the glm it only shows the values of the coefficient as it relates to the first level(Except for the first IV).
To recreate it I have the R code for something similar using mtcars:

md <- datasets::mtcars
md <- mutate(md, ncyl = ifelse(cyl==6,"Six Cyl", cyl))
md <- mutate(md, ncyl = ifelse(cyl==4,"Four Cyl",ncyl))
md <- mutate(md, ncyl = ifelse(cyl==8,"Eight Cyl",ncyl))
md <- mutate(md, wgear = ifelse(gear==3,"Three G",gear))
md <- mutate(md, wgear = ifelse(wgear==4,"Four G",wgear))
md <- mutate(md, wgear = ifelse(wgear==5,"Five G",wgear))
md <- mutate(md, wcarb = ifelse(carb==1,"One Carb",carb))
md <- mutate(md, wcarb = ifelse(carb==2,"Two Carb",wcarb))
md <- mutate(md, wcarb = ifelse(carb==3,"Three Carb",wcarb))
md <- mutate(md, wcarb = ifelse(carb==4,"Four Carb",wcarb))
md <- mutate(md, wcarb = ifelse(carb==6,"Six Carb",wcarb))
md <- mutate(md, wcarb = ifelse(carb==8,"Eight Carb",wcarb))
md <- md[,-c(2,10,11)]
model <- glm(mpg~., data = md)
summary(model)
modelint <- glm(mpg~.-1, data = md)
summary(modelint)

So for summary(modelint) I can see the values for the coefficients of all 3 possible values ncyl, and with confint(modelint) I can at least see confidence intervals for all of the possible values. But wgear and wcarb are just compared to their first level alphabetically and the t values are just if it is different from the first level. I know if I just did mpg~wgear-1 I could see their comparisons, but I think this does not take into account all of the other variables. Ideally I would like to see how 1 carb compares to two carb, three, four,...  and 1 carb vs not 1 carb while taking into account the other variables. Is there a better function to use than glm? Any advice would be greatly appreciated!


texts: How to calculate accuracy on keras model with multiple outputs?

texts: I have a keras model that takes in an image with (up to) 5 MNIST digits and outputs a length and then (up to) 5 digits. I see that model.evaluate() reports accuracies for each of the outputs but how do I determine how good the model is at predicting the numbers? Do I need to write that myself?


texts:
My girlfriend has recently been struggling with finding a new job, so I thought I'd make a website to help her out. The basic idea is that she'll be shown a list of jobs, rate her interest, and then a hopefully interesting job will be emailed to her every day. So far I have the frontend worked out, but I'm struggling with the engine. I've looked into different kinds of recommendation engines, and it seems like a content based engine is the way to go since she will probably be the only user.
With that said, feature selection is problematic because job categorizations kind of unrealistic. For instance, I could have job categories like "administrative" or "analytical", but those would be hard to define, especially if I was analyzing the job description for certain keywords.

Sorry about the wall of text. My basic question is how should I go about selecting features for this job recommendation engine?

Thanks for the help.


texts:
I am studying about evaluation of both recommendation systems and machine learning algorithms in recent times, trying to define a scope for my masters research. After some reading time I'm starting to understand several concepts, but one thing was not clear to me:

Do recommendation systems necessarily use machine learning algorithms?

I mean, I know these two can be used combined, but in most of the papers I read about recommender systems evaluation, they do not even mention anything about Machine Learning.

Also, if you can suggest some papers that I can read, I would be very grateful


texts: Why is the GoogLeNet retrained model size less compared to others?

texts:
I have some questions if someone can answer me or guide me articles to understand them. I investigated different pre-trained model i.e. AlexNet, VGG, GoogLeNet, InceptionV3 and ResNet. I have retrained these models(In MATLAB2018b using single CPU) on my disease dataset and the retrained models are of following sizes:
AlexNet: 207.266MB
VGG: 407.981MB
GoogLeNet:22MB
Inception V3: 79.46MB
ResNet: 155MB
**Q1) Among all the size of the GoogLeNet and InceptionV3 is less among all? What can be the be the possible reasons? Usage of inception models or usage of 1x1 convolution filter?
Q2) Why the size of AlexNet is more than GoogLeNet and ResNet? Is it because of computation of gradient at each layer?**
The other question is related to understanding the training progress provided by MATLAB? Why the model struggle in the initial epoch?
What are the possible options?

I will be really very thankful if anybody can help me in understanding any of these questions. Thanks in advance.


texts:
I'm working on a dataset of computer game replays. Each replay file describes the game of two players as a timeseries of game commands, where each command has a type and coordinates, like so: (attack, 23.4, 75.1) (simplified description).
I'd like to evaluate games based on the spatial information in the coordinates. For example, which player had broader command of the game map (e.g., area of the convex hull of their commands at different time points), which player had higher command density overall or in specific areas, etc.
I need something that summarises effectly across time and space. For example, if a player gave commands on all four corners of the map in one minute, then stayed in their little corner for the rest of the game, that's not as "powerful" as a player that effectively commanded the entire map area throughout the game.
I feel there must be a whole domain of pre-existing algorithms and metrics that should help me, but I'm not sure which keywords to search and read up on.
Does this problem sound like an obvious "oh, this is clearly a foo domain problem"? What is foo?


texts: Input shape for simpler time series in LSTM+CNN

texts:
Following is code from https://machinelearningmastery.com/how-to-develop-rnn-models-for-human-activity-recognition-time-series-classification/ which uses LSTM and CNN with TimeDistributed for human activity timeseries:

def evaluate_model(trainX, trainy, testX, testy):
    # define model
    verbose, epochs, batch_size = 0, 25, 64
    n_timesteps, n_features, n_outputs = trainX.shape[1], trainX.shape[2], trainy.shape[1]
    # reshape data into time steps of sub-sequences
    n_steps, n_length = 4, 32
    trainX = trainX.reshape((trainX.shape[0], n_steps, n_length, n_features))
    testX = testX.reshape((testX.shape[0], n_steps, n_length, n_features))
    # define model
    model = Sequential()
    model.add(TimeDistributed(Conv1D(filters=64, kernel_size=3, activation='relu'), input_shape=(None,n_length,n_features)))
    model.add(TimeDistributed(Conv1D(filters=64, kernel_size=3, activation='relu')))
    model.add(TimeDistributed(Dropout(0.5)))
    model.add(TimeDistributed(MaxPooling1D(pool_size=2)))
    model.add(TimeDistributed(Flatten()))
    model.add(LSTM(100))
    model.add(Dropout(0.5))
    model.add(Dense(100, activation='relu'))
    model.add(Dense(n_outputs, activation='softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    # fit network
    model.fit(trainX, trainy, epochs=epochs, batch_size=batch_size, verbose=verbose)
    # evaluate model
    _, accuracy = model.evaluate(testX, testy, batch_size=batch_size, verbose=0)
    return accuracy

However, I want to modify above code for a simple time series data (1000 rows and 100 columns), where each row is a time series of 100 values (shown as columns). The output is a class and there are 7 types of classes (hence n_output=7). It is a "sequence classification" problem.

What change should I make in above function for this? Do I need n_steps and n_length (since my data is simpler than human activity recognition data)? For simple CNN1D, I have to reshape as follows:

trainX = trainX.reshape(nrows, ncols, 1)
# then use (ncols, 1) as input shape

How should I reshape here?

I tried by removing n_steps and n_length and putting input_shape as (100, 1) (after reshaping data to (1000,100,1)) but it did not work.


texts: Plotting facetgrid in subplot for IRIS dataset

texts:
I'm trying to plot 6 selected pair subplots with the combination of facetgrid of seaborn and scatter plot from matplotlib.
I'm getting plot, but subplots remains empty whereas facetgrid gets plotted in a new figure.

Below is my code-

pairs = [["SepalLengthCm", "SepalWidthCm"], ["PetalLengthCm", "PetalWidthCm"], ["SepalLengthCm", "PetalWidthCm"], ["PetalLengthCm", "SepalWidthCm"], ["PetalLengthCm", "SepalLengthCm"], ["PetalWidthCm", "SepalWidthCm"]]
plt.figure(figsize=(15,8))
i = 1
for each in pairs:
    plt.subplot(2,3,i)
    sns.set_style("whitegrid")
    fg = sns.FacetGrid(iris, hue="Species", size=4)
    fg.map(plt.scatter,each[0], each[1]).add_legend();
    plt.show()
    i += 1

Sample Output:

Can someone help out with it?


texts: Service Request classification, questionnaire filling and call logging

texts:
I am very new to machine learning. I just went through some of the tutorials in Azure and completed one practice workflow(car price prediction). I hope I can ask basic questions here.

Scenario : We get service request from our customers via email. This has fields like customer name, user name, email id, Equipment affected, type of call and Issue experienced(this is a free text area).

The employee reads this email, mainly the issue experienced. Based on the issue experienced section, s/he takes the appropriate actions. We will have 4-6 fields(type of request , a few questionnaire etc). The issue experienced is a free text area where customer can write anything about the issue.

Does this qualify as a AI model if we have last 2-3 years data. If yes, Is multi class classification the solution? If not, which ML algorithm needs to be used here. Can I rely on Azure for this or do we need to build a new model/algorithm for this?

Sorry if it is a too basic question


texts: How many features to sample using Random Forests

texts:
The Wikipedia page which quotes "The Elements of Statistical Learning" says:

  Typically, for a classification problem with $p$ features, $\lfloor \sqrt{p}\rfloor$  features are used in each split.

I understand that this is a fairly good educated guess and it was probably confirmed by empirical evidence, but are there other reasons why one would pick the square root? Is there a statistical phenomenon happening there?

Does this somehow help decrease the variance of the errors?

Is this the same for regression and classification?


texts: Is it possible to connect three neural networks in Matlab?

texts: If I have 3 separate feedforward neural networks in Matlab, is it possible to connect them so that, given input data and target data the 3 work in parallel to produce output? If so, how do I do this?


texts:
I'm reading the book Introduction to Data Mining by Tan, Steinbeck, and Kumar.
In the chapter on Decision Trees, when talking about the "Methods for Expressing Attribute Test Conditions" the book says :

  "Ordinal attributes can also produce binary or multiway splits. Ordinal
  attribute values can be grouped as long as the grouping does not
  violate the order property of the attribute values. Figure 4.10
  illustrates various ways of splitting training records based on the
  Shirt Size attribute. The groupings shown in Figures 4.10(a) and (b)
  preserve the order among the attribute values, whereas the grouping
  shown in Figure a.10(c) violates this property because it combines the
  attribute values Small and Large into the same partition while Medium
  and Extra Large are combined into another partition."

Why ordinal attribute values can be grouped as long as the grouping does
not violate the order property of the attribute values?


texts: Is Data Science the Same as Data Mining?

texts:
I am sure data science as will be discussed in this forum has several synonyms or at least related fields where large data is analyzed.

My particular question is in regards to Data Mining.  I took a graduate class in Data Mining a few years back.  What are the differences between Data Science and Data Mining and in particular what more would I need to look at to become proficient in Data Mining?


texts: Deep Learning Project to Predict Stock Prices

texts:
So I have a background in computer programming and a little in machine learning in general.  What I would like to do is create a fun project in A.I. with deep learning.

I have a dataset that has a whole bunch of stock prices at a certain date, with a bunch of features for each entry to go with it.  I also have some "experts" who made predictions on whether the stock will go up or down. As my dataset grows I can evolve the game to make selections from multiple stocks...etc

Essentially what I would love to do is create an A.I. app that will be fed the same data that the "experts" had and see if I can create something more accurate and beat them at it.  Is this a viable approach?


texts: Why and how BERT can learn different attentions for each head?

texts:
https://towardsdatascience.com/deconstructing-bert-distilling-6-patterns-from-100-million-parameters-b49113672f77

I read the blog above. It visualizes that different color/head has different attention of words.

Based on my understanding, the code implementation of each head is almost the same.


texts: What is Bits per Character (bpc) metric which has been used to measure the model accuracy with reference to text8 and enwiki8 datasets. I encountered the term bpc in transformer -XL paper here. How different is it from the perplexity as a metric?


texts: Generalized Hebbian Algoithm (GHA) stability issues

texts:
The GHA proposed by Sanger in 1989 is supposed to be very numerically stable. There are some implementations of it floating around, for example this one https://github.com/matwey/python-gha.
In that test example the implementation is working fine.
What happens is, if you send in for example MNIST data into the same code linked above and run it through a couple of data points the weight matrix quickly overflows to nan. Input size is obviously adjusted to 784 instead of 5 like in the test code.
Why does this happens, do anyone know any limitation to this algorithm. There are a lot of research using this rule in this way and I just can't get it to work. What am I missing?


texts:
Are the embedding values for a particular word using word2vec Skipgram model the weights of the first layer or the softmax output of the function?
Does the embedding value change according to the training corpus?


texts: SuperLearner Cross validation with iid time series

texts:
I created a number of ML models in R and I aim at combining them to form an ensemble.

I learned about SuperLearner library which cross validates many models and returns the weight to each model in the ensemble.

However, the cross validation mechanism randomly samples observation out of the whole training set. This carries the issue that in a regression model if the y is not iid (typical in financial time series) then this sampling method introduces a bias.

Is there a way to set the cross-validation set equal to an x% of consecutive observations within the training set so that serial autocorrelation is taken into account?


texts: Do we need the outer discount term when implementing REINFORCE algorithm

texts:
I am learning the REINFORCE algorithm, which seems to be a foundation for other algorithms.
I saw the $\gamma^t$ term in Sutton's textbook.

But later when I watch Silver's lecture on this, there's no $\gamma^t$ term.

I read several implementations of the REINFORCE algorithm and seems no one includes this term. (However, when calculating return $G_t$, they do use $\gamma$ to do discounting.)
I am wondering, is it OK to just omit the outer $\gamma^t$ while still using it to calculate the return ($G_t$ or $\text{v}_t$)? Does the outer $\gamma^t$ term have nothing to do with the inner $\gamma^{k-t-1}$ term?
Thanks.
FYI, this is the implementation I studied: https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py


texts: How can I use two different datasets as a training model for svm

texts:
I know that you're supposed to scale your test data using the parameters (mean and stdev) from your training data. This is relatively simple; but what if the number of samples is limited in one training data set (e.g. Set A = 5 samples) so I want to combine two data sets (i.e. Set A + Set B = 10 samples) to have enough samples for training, what can I do so that I can scale/normalize the two sets into one and then use those parameters on my test set? If I scale them individually I will have 2 means and 2 stdev.

The context is I'm trying to combine two microarray expression from two different microarray platform so their expression ranges are different.

Thank you for your help in advance


texts:
I have a R dataframe having a column with dates in multiple formats.
I want to convert this column to date format.
While some formats are converted others are not.
Why is the second date not converted? How can this be corrected?
Code and result:
> slno <- c(1,2)
> Date <- c('22-04-2015', '4/8/2015')
>
> df_data <- as.data.frame(cbind(slno,Date))
> df_data
  slno       Date
1    1 22-04-2015
2    2   4/8/2015
>
> date_formats <- c('%m/%d/%Y', '%d-%m-%Y')
>
> #df_data$Date <- mdy_hm(df_data$Date)
> df_data$Date <- as.Date(as.character(df_data$Date), tryFormats = date_formats, optional = F)
>
> df_data
  slno       Date
1    1 2015-04-22
2    2       <NA>
> str(df_data)
'data.frame':   2 obs. of  2 variables:
 $ slno: chr  "1" "2"
 $ Date: Date, format: "2015-04-22" ...
>


texts: How many pictures are needed to generate eigenfaces that correctly represent the input faces?

texts: I would like to show how eigenfaces work. Therefore, I would like to take 5 pictures of the person who is using my code and then after PCA reduction I would like to show the eigenfaces which represent the persons face. However, somehow the eigenfaces that get produced are just some weird unclear features not even similar to faces. I am wondering if it might need a larger training set to train on. And if so, I wonder if there might be some method to fulfill my idea. For example, a pretrained eigenface creator or adding the token pictures to a large dataset.


texts: Is train/test-Split in unsupervised learning of neural network necessary?

texts:
I am using autoencoder for anomaly detection in warranty data. It is unsupervised. I calculate the reconstruction error by the model and the records with high reconstruction error value is considered as an anomaly. I would like to know, if it is necessary to train/test-split the data.

Any help is much appreciated!


texts: R studio, one multi core CPU vs dual single cores

texts:
So far I was using R on my home pc: i3 CPU, two cores, 4 threads. In order to run the code faster I was using the package "DoSnow", utilizing 3 out of the 4 cores in order not to choke my system completely.

However, now I got a server which has a two single core Xenons and I wonder: can I use the same multi core code for multi cpus? If no, how can I run them together and at about 70% (again, in order to not choke the system)? So far, all the codes I found on the net refer to cores rather then cpus


texts: Which decision tree model is used in "standard" random forest?

texts:
Is that CART? Why not using C5.0 tree?

A perhaps more general question:

Since C5.0 tree frequently have better performance than CART, why people still use CART to build random forest (or people already are doing this)?


texts: Count and summarise ID's each day while creating a new column that shows the accumulated ID's

texts:
I have two column, first one being the ID of a customer and second one being the Date of purchase.

    ID       Date
     1 2017-01-17
     2 2017-01-17
     3 2017-01-17
     4 2017-01-17
     5 2017-01-17
     1 2017-01-17
     7 2017-01-17
     1 2017-01-17
     9 2017-01-18
     2 2017-01-18
     3 2017-01-18
     5 2017-01-18
     1 2017-01-18
     2 2017-01-18

I would like to summarise the Purchases made by a Customer at one day and create a third column that shows the amount of purchases for the customer on that date.


texts: from the naked eye itself, we can tell in the region 5161 the network usage is high so that is the anomaly in my case, then why do we want to apply k-means and other machine learning algorithms to find anomalies in our data


texts:
I have chemical plant data where the product is manufactured in batches.  Each batch takes about 4 hours and I have data for every 5 mins. My objective is to classify the batches as good and bad. How can I achieve this for a time related data like this one?

One method I have read is time unfolding where each time interval in combination with the feature becomes an individual feature (e:g if there is interval T0 and T1, features F0 and F1 then we will have 4 features T0 F0 and T1 F1) - please correct me if I am wrong about unfolding.

So basically we are converting each batch of N rows of data into single row. Is there any other method for this?


texts: Python Machine Learning/Data Science Project Structure

texts:
I'm looking for information on how should a Python Machine Learning project be organized. For Python usual projects there is Cookiecutter and for R ProjectTemplate.

This is my current folder structure, but I'm mixing Jupyter Notebooks with actual Python code and it does not seems very clear.

.
├── cache
├── data
├── my_module
├── logs
├── notebooks
├── scripts
├── snippets
└── tools

I work in the scripts folder and currently adding all the functions in files under my_module, but that leads to errors loading data(relative/absolute paths) and other problems.

I could not find proper best practices or good examples on this topic besides some kaggle competition solutions and some Notebooks that have all the functions condensed at the start of such Notebook.


texts: How to set weight in Weighted Kernel K-Means?

texts:
The objective function of kernel K-means is

$$      \sum\limits_{c=1}^k \sum\nolimits_{a_i \in c}   w_i \Vert \phi(a_i)- m_c \Vert^2 \ $$

where

$$ m_c = \frac{\sum\nolimits_{a_i \in c }w_i\phi(a_i)}{\sum\nolimits_{a_i \in c  }w_i}  $$
I need to know how to determine wi


texts: How to cluster and visualize 3D data in python

texts:
I have a 3D dataset of x,y,z points with 2 categories, category A and B. My end goal is to cluster all points in category B into volumes (spheroids/clouds) and find all points of category A close to edge of those volumes. I assume there won't be any points of category A inside the spheroids. The points of category B are very highly clustered in space, so clusters are probably very evident.
In 2D GIS I have used Kernel Density Estimation and K-Means clustering for similar tasks, but since I am dealing with 3D data, and non-geographic at that (relative to a fictional 0,0,0 origin), and since I am comfortable with the python data science tools, I think matplotlib/scipy/numpy/sklearn/pandas/etc are probably better tools for this. But I am not sure what tools and libraries specifically would be good to look at.
So my question is 2-fold:

what libraries would be well suited to find the 3D clusters and the points of category A "close" to them
what tool would allow me to visualize the clusters, preferably in an interactive plot that allows me to zoom/pan/rotate


texts: Cast regression problem into classification problem

texts:
In the TA session, my TA claimed, that regression problems should often be cast into classification problems by dividing the output range into bins and then using a multi-loss, since we have better classification than regression algorithms.
In my understanding, this is inherently wrong as it discards the property, that "close to correct is better than far correct". All wrong classes are equally wrong.
I asked my professor, but he just said, there are applications where it makes sense and did not want to discuss it more.
Am I wrong? When should I cast a regression problem into a classification problem?
Edit: I do not know if my TA referred to it, but here is a tweet from A. Karpathy: https://twitter.com/karpathy/status/708480082831024128

not-widely-enough-known-protip: Do not use L2 loss (regression) in
neural nets unless you absolutely have to. Softmax likely to work
better.


texts: Visualizing post-hoc results of many days

texts:
I'm looking for ideas how to plot many Tukey HSD results.
My origianl dataframe has 3 treatments. The values of each patient were measured every day for one month.
One-way ANOVA has shown that there is significant difference betwen the treatments, and now I want to see which treatment significantly differed from each other, but on a daily basis.
So I have run Tukey test for every day (that was significant in the ANOVA).
now  have many plots like this:
At te moment I just put all the days together in order to show the trend (in the begining treatment a differed only from b, then all differ ect), by put all those plots together, something like this:

...
My question is if there is any way I could "reduce" this visualization, so I can display it on one graph and tell there which groups differed from each other at which day.
Basically, looking for ideas.
*I have removed here the labels ,originally it has labels


texts: Error not decreasing in a 3 layer deep CNN using TensorFlow

texts:
I'm trying to train a CNN to play an online game by feeding images of the game along with the keyboard input.

By playing the game for some time and collecting the data, I gathered 342 images with size 110x42. I'm feeding these images in the network like so:

def convolutional_neural_network(x):
    weights = {'W_conv1': tf.Variable(tf.random_normal([3, 3, 1, 16])),
               'W_conv2': tf.Variable(tf.random_normal([5, 5, 16, 32])),
               'W_conv3': tf.Variable(tf.random_normal([5, 5, 32, 64])),
               'W_conv4': tf.Variable(tf.random_normal([5, 5, 64, 64])),
               'W_fc': tf.Variable(tf.random_normal([7 * 3 * 64, 1024])),
               'out': tf.Variable(tf.random_normal([1024, n_classes]))}

    biases = {'b_conv1': tf.Variable(tf.random_normal([16])),
              'b_conv2': tf.Variable(tf.random_normal([32])),
              'b_conv3': tf.Variable(tf.random_normal([64])),
              'b_conv4': tf.Variable(tf.random_normal([64])),
              'b_fc': tf.Variable(tf.random_normal([1024])),
              'out': tf.Variable(tf.random_normal([n_classes]))}

    x = tf.reshape(x, shape=[-1, 110, 42, 1])

    conv1 = tf.nn.relu(conv2d(x, weights['W_conv1']) + biases['b_conv1'])
    conv1 = maxpool2d(conv1)

    conv2 = tf.nn.relu(conv2d(conv1, weights['W_conv2']) + biases['b_conv2'])
    conv2 = maxpool2d(conv2)

    conv3 = tf.nn.relu(conv2d(conv2, weights['W_conv3']) + biases['b_conv3'])
    conv3 = maxpool2d(conv3)

    conv4 = tf.nn.relu(conv2d(conv3, weights['W_conv4']) + biases['b_conv4'])
    conv4 = maxpool2d(conv4)

    fc = tf.reshape(conv3, [-1, 7 * 3 * 64])
    fc = tf.nn.relu(tf.matmul(fc, weights['W_fc']) + biases['b_fc'])
    fc = tf.nn.dropout(fc, keep_rate)

    output = tf.sigmoid(tf.add(tf.matmul(fc, weights['out']), biases['out'], name='pred'))

    return output

def train_neural_network(x):
    prediction = convolutional_neural_network(x)
    cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=prediction, labels=y))
    optimizer = tf.train.AdamOptimizer(learning_rate=1).minimize(cost)

    hm_epochs = 6
    with tf.Session() as sess:
        sess.run(tf.initialize_all_variables())

        for epoch in range(hm_epochs):
            epoch_loss = 0
            for epoch_x, epoch_y, i in dataset.create_batches():
                epoch_x = epoch_x.reshape(-1,4620)
                _, c = sess.run([optimizer, cost], feed_dict={x: epoch_x, y: epoch_y})
                epoch_loss += c

            print('Epoch', epoch, 'completed out of', hm_epochs, 'loss:', epoch_loss)

        #correct = tf.equal(tf.argmax(prediction, 1), tf.argmax(y, 1))
        correct = tf.equal(tf.greater(prediction, 0.5), tf.equal(y, 1.0))
        accuracy = tf.reduce_mean(tf.cast(correct, 'float'))
        print('Accuracy:', accuracy.eval({x: dataset.dataset['test']['x_test'], y: dataset.dataset['test']['y_test']}))

train_neural_network(x)

The errors are getting stuck in a particular value and floating up and down around this value by a small amount.

I've tried incresing/decreasing the learning rate, improving the quality of images, changing the size of the batches... and nothing seems to make the network stable.

Do you guys know what I'm doing wrong?


texts: How to impute using simple imputer (custom function)

texts:
I am imputing my data using simple imputer from sklearn. i want to test many different ways of applying transformations to the data. i.e for logisitcic regression i would like to

remove nans and replace with mode
replace +infs with max and -infs with min
use standard scaler.

then for using xgboost i would like to:

simply replace -infs/+infs with very large or -ve large numbers.

i have been playing with sklearn pipeline and i would like to know how i can pass the custom imputers through the pipeline? e.g:

logistic_pipeline = Pipeline( steps = [('imputer', SimpleImputer(strategy = 'most frequent') ),
                                  ( 'std_scaler', StandardScaler() ),
                        ( 'model', LinearRegression() )] )

but how do i incorprate the following function into it where i am replacing infs from the training datase (df) with the max of that column . then using this max to populate it into the test.. how can i do this using pipeline?

def replace_pos_inf(df, dftest, numeric_features):

    for col in df[numeric_features].columns:
        m = df.loc[df[col] != np.inf, col].max()
        df[col].replace(np.inf,m,inplace=True)
        dftest[col].replace(np.inf,m,inplace=True)

    for col in df[numeric_features].columns:
        mini = df.loc[df[col] != -np.inf, col].min()
        df[col].replace(-np.inf,mini,inplace=True)
        dftest[col].replace(-np.inf,mini,inplace=True)

    return df,dftest


texts: Microsoft custom vision vs Tensorflow model?

texts:
I am planning to implement my own image classifier model using TensorFlow instead of a custom vision platform.
what is the biggest difference between custom vision(https://www.customvision.ai/) vs TensorFlow?


texts: Techniques for hyperparameter search in non-stationary environments

texts:
I'm tuning a supervised machine learning model over time, also called incremental learning. I do not want to assume the environment is non-stationary.

Grid search and random search do not appear to be the best choices.

Bayesian optimization has been used previously for this challenge.

Contextual bandits could possibly be used. However, I could not find any examples.

What are the best options for hyperparameter search in non-stationary environments?


texts:
I have a dataset with only 2 categorical attributes out of 9. How can I get a clustering analysis on it?
I am using R. Do you have any advices about instructions, how to do it, topics, ...?
here's my dataset

thanks


texts: When combined correlation of features decreases

texts:
I'm building a machine learning model in Python to predict soccer player values. I'm trying to predict a "player_value" column containing the value of a specific player. Consider a sample of the columns (features) I'm using.
---------------------------------
appearances | goals | goals_per_game
------------|-------|---------------
    20      |   2   |     0.1
    60      |   20  |     0.33
    54      |   30  |     0.55
    43      |   15  |     0.34
    30      |   17  |     0.56

I thought that the correct way to use those columns would be creating a goals per game statistic (goals divided by appearances), since a player can have more goals than another player, but with less matches played.
After that, the correlation of the refered columns with the column that I'm trying to predict (player value) decreased. The correlation of the "goals" and the "appearances" columns with the player value column was about 45% each, while the new "goals_per_game" column has a correlation around 18%.
Should I use the columns "appearances" and "goals_per_game" columns individually and not use the "goals_per_game" column? Is my analysis wrong and it does not makes sense to use a "goals_per_game" metric since the player value is higher when using those features individually?


texts: Best framework for recognizing a specific cartoon character's face?

texts:
I have a supply of images of a specific cartoon character's face. I have hours of video. I would like to automatically find the sections of the video in which this cartoon character appears.

https://github.com/ageitgey/face_recognition doesn't seem to work very well on cartoon characters (it fails to recognize any face in my images).

What is the state of the art on this? Is there an open-source library or framework that is good at this?


texts: Isn't computing the "tractable error" in Restricted Boltzmann Machines (RBM) intractable?

texts:
Let $v \in \{0,1\}^M$ be the visible layer, $h \in \{0,1\}^N$ be the hidden layer, where $M$ and $N$ are natural numbers. Given the biases $b \in \Re^M$, $c \in \Re^N$ and weights $W \in \Re^{M \times N}$, the energy and probability of an RBM is given by:

Energy, $E(v,h; b,c,W) = -b^T v - c^T h - h^T W^T v$

Probability, $p(v, h; b, c, W) = \frac{e^{-E(v,h; b,c,W)}}{Z}$

where $Z = \sum_{v,h} e^{-E(v,h; b,c,W)}$

The negative log likelihood error for a Restricted Boltzmann Machine (RBM) is given by:

$\mathcal{L}(b,c,W) = \frac{1}{T} \sum_{t=1}^{T} \left( -\log \sum_{h} e^{-E(v^t, h; b,c,W)} \right) + \log Z$

where:

$T$ is size of training dataset; and

$v^t$ represents $t^{th}$ data point in the training dataset

It is clear that computing $Z$ (and hence $\log Z$) is intractable because we have to sum over $2^{M+N}$ configurations of $v$ and $h$ - exponential time algorithm.

However, shouldn't computing $\sum_{h} e^{-E(v^t, h; b,c,W)}$ be intractable as well? Aren't we summing over all the $2^N$ configurations of $h$ here? If say $N = 64$, then we are already reaching exascale computations ($2^{64} = 1.84 \times 10^{19}$)!


texts: Would having data with different values coming from different sources but having the same timestamp affect the training of my LSTM model?

texts:
If I have two readings of the same data as follows
timestamp         concentration1 concentration2
2016-01-01 01:00  30.4           10
2016-01-01 01:00  36.4           16

Would this somehow impact the training of model negatively or confuse it in some sense or does the model know that these readings are from two difference sources?
If it does affect it would I then create a new feature that specifies where a concentration comes from as one of the two following ways?
1-
timestamp         concentration 1-A       concentration 2-A      concentration 1-B      concentration 2-B
2016-01-01 01:00  30.4                    10                     36.4                    16

2-
  timestamp         concentration1 concentration2  source
  2016-01-01 01:00  30.4           10              A
  2016-01-01 01:00  36.4           16              B


texts:
How do you compute MAP in python for evaluating recommender system effectiveness? Is there any library in sklearn or code in python for it?

I would like to compute the effectiveness of my Recommender System by computing Mean Average Precision.


texts: What are the business metrics I should track to evaluate a recommender model deployed on an e-commerce website?

texts: Can you suggest some google analytics metrics such as (click or impressions etc) to evaluate a recommender model deployed on an e-commerce website.


texts: How does one define the possibility space of valid priors (models)?

texts:
When one trains a model on data of any complexity one inevitably ends up with a one particular model among a vast many possible models that would make similar (or even, the same) predictions.

For example, we can both train a neutral net on mnist and the model I produce will have different weights and biases from yours, even if they make the same predictions.

But are there any algorithms that don't produce one model, but instead produce a description of the space of all possible models that could describe the data?

Does that even make sense? In some way, as near as I can tell, a model represents a set of priors (biases and weights). Is this sort of thing not done because it's too expensive?

Take the following riddle as a conceptual example, '''A man and his son are in a terrible accident and are rushed to the hospital in critical care. The doctor looks at the boy and exclaims "I can't operate on this boy, he's my son!" How could this be?'''

You might have a prior that says, "most surgeons are men" and another prior that says, "most parents are heterosexual." You have a space of two models that would explain the data, and of course they are mutually exclusive: either the boy has two fathers, or the surgeon is his mother.

The key is, both of these models would work to explain the data and you don't know which model is true until you get more data. Also, just as importantly, you can map each model's relationship to your prior beliefs so when you learn which one is valid you can update you're priors accordingly.

A technique or algorithm that produces a probability space of all possible models seems like a potentially powerful tool. Yet I can't find this kind of thing anywhere.

Are there any algorithms or machine learning techniques that do this? And furthermore how do you even go about mathematically describing a possibility space of all possible valid models according to their likelihood, according to your priors?

Thanks for your help!


texts: Human Height Estimation using walking stride

texts: Are there any papers or research showing a correlation between walking stride and human height? My purpose is to estimate height from walking stride of a person.