texts:
I was implementing a neural network with the MNIST dataset, and I am having issues in terms of classifying 10 classes. I am pretty sure the implementation of feed-forward and back propagation is correct, and I have used Xavier initialization to initialize the weights. When running mini-batch gradient descent, with only $1$ epoch, the lowest loss I have obtained with standard deviation is $0.3288$ as shown in the graph below:

Complete graph of $J(\theta)$:

Note that I'm using a zero-indexed implementation, which means $\frac{\partial{J(\theta)}}{\partial{\Theta^{(l)}}} = a^{(l)}\delta^{(l - 1)}$, and $a^{(l)} = \sigma((\Theta^{(l - 1)})^Ta^{(l)})$
The model has $2$ hidden layers with $10$ nodes each (excluding the bias term). Activation functions for layers $a^{(l)}, \text{ where } l > 0$  Cross-Entropy loss is used for every node in every layer.
As my training data is normalized, normalized test data is used to test the accuracy of the model. Despite a low cost, my model only predicts one class.
You can find the source code here.
To summarize, I have two questions:

What does negative cost indicate?
Why is my model only predicting one class?


texts:
While solving the questions for machine learning, I got two values for R square from 2 different regressors, i.e, 0.9999 and 0.9769. So, which should go for as both could lead to overfitting?
Thanks in advance


texts: How to fill missing numeric if any value in a subset is missing, all other columns with the same subset are missing

texts:
There is a clear pattern that show for two separate subsets (set of columns); If one value is missing in a column, values of other columns in the same subset are missing for any row.

Here is a visualization of missing data

My tries up until now, I used ycimpute library to learn from other values, and applied Iterforest.

I noted, score of Logistic regression is so weak (0.6) and thought Iterforest might not learn enough or anyway, except from outer subset which might not be enough? for example the subset with 11 columns might learn from the other columns but not from within it's members, and the same goes for the subset with four columns.

This bar plot show better quantity of missings

So of course, dealing with missings is better than dropping rows because It would affect my prediction which does contain the same missings quantity relatively.

Any better way to deal with these ?

[EDIT]
The nullity pattern is confirmed:


texts: What algorithms should I choose for a recommender system and why?

texts:
To my knowledge, recommender systems are broadly classified into collaborative and content. Collaborative in turn is divided into 1) Memory (uses similarity metrics) and 2) Model (well known Matrix/Tensor factorization). Content based involves constructing a user profile and then an algorithm like SVM to classify and recommend items. Now here are my questions:

What other algorithms can I use for recommendation and why?
Can I use neural networks? (understanding them has been a bit difficult for me)
Is it true that neural networks (NN) are only suited for text and image processing and numerical data doesn't need complex algorithms like NN?


texts: What are the exact differences between Deep Learning, Deep Neural Networks, Artificial Neural Networks and further terms?

texts:
After having read some theory I am getting a bit confused about the following terms:

Deep Learning
Deep Neural Network
Artificial Neural Network
Feedforward Neural Network

So, what seems clear to me is that Deep Neural Networks are Artificial Neural Networks with multiple layers (usually more than 1 hidden layer). However I have read several times that:

  "Deep Neural Networks are feedforward Neural Networks with many layers."

I know what a feedforward Neural Network is, but to my understanding Deep Neural Networks is a term for ALL Artificial Neural Networks with multiple layers between input and output layer? Shouldn't there also be for instance Deep Recurrent Neural Networks? Is it correct that Deep Neural Networks must feedforward Neural Networks? This would in turn mean that Deep Recurrent Neural Networks cannot be referred to as Deep Neural Networks.

Moreover, I see there is a large variety of deep learning architectures, such as:

Convolutional Neural Networks
Residual Neural Networks
Deep Belief Networks
Deep Boltzmann machines
...

However, now also Wikipedia gives me a hard time to distinguish all the terms, saying: "Deep learning architectures such as deep neural networks, deep belief networks, recurrent neural networks and convolutional neural networks have been applied to fields including..."

So, my concrete questions rising from all the thoughts above are:

Does the term Deep Neural Networks ONLY belong to feedforward Neural Networks?
If the answer to 1) is yes: is the Wikipedia definition correct as you can read it up there? This would mean that i.e. a Convolutional Neural Network with multiple layers must be referred to as a Deep Convolutional Neural Network, which is not a subclass of Deep Neural Networks?
Is the term Deep Neural Networks a collective term for ALL Artificial Neural Networks with multiple layers, or just for all feedforward Neural Networks with multiple layers?
Would it be more accurate to use the collective term "Deep Learning Architectures" with strictly separated subclasses as Wikipedia suggests?


texts: What is the benefit of splitting tfrecord file into shards?

texts:
I'm working on speech recognition with Tensorflow and plan to train LSTM NN with massive waves dataset. Because of the performance gains, I plan to use tfrecords. There are several examples on internet (Inception for ex.) where tfrecords files are split into shards.
My question is: what is the benefit of having tfrecords file into shards? Is there any additional performance gain of this split?


texts: Is deduction, genetic programming, PCA, or clustering machine learning according to Tom Mitchells definition?

texts:
Tom M. Mitchell defines machine learning as

  A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

For some algorithms that are usually counted towards the field of machine learning, I heavily doubt that they are learning algorithms according to the definition. Interestingly, all of them are part of Tom Mitchell's book "Machine Learning".

Deduction

The "symbolist tribe" uses inverse deduction according to Pedro Domingos (see The 5 tribes of ML). While I'm not too sure what inverse deduction is, the idea of having a set of true statements and inference rules like Modus Ponens to create more/other true statements is clear.

Comming to the definition: What is experience here? How is performance measured? What does a typical task look like?

Genetic programming

From Wikipedia:

  Genetic programming (GP) is a technique whereby computer programs are encoded as a set of genes that are then modified (evolved) using an evolutionary algorithm (often a genetic algorithm, "GA").

What is experience in this case? Maybe "iterations" / "epochs"? (I'm not sure about the terminology)

Tasks: What are the typical use cases of Genetic Programming?

PCA

PCA and some other dimensionality reduction algorithms are in some way "fixed". The algorithm does not change with more data. It does not have parameters. Thinking about it, one might argue that it estimates the projection in a better way with more data.

But then: What is better? What is the performance measure P here?

Clustering

I don't see a clear performance measure for Clustering (e.g. k-means). Also, the only kind of experience I can see is the number of samples.


texts: Doc2Vec for dataset with several text fields: concatenate or separate models?

texts:
I have a dataset with several fields: description, name, header. I want to train doc2vec out of it, so that I could use vectors for classification.

So I wonder, should I concatenate all the fields together into a single document for each sample or should I train separate model for each field?. Some fields may be missing (e.g. there may be no description).

If I train as a single document, should I emphasize context of each filed (e.g. 'NameStart'+ sampleName + 'NameEnd' + 'HeaderStart' + sampleHeader + 'HeaderEnd'...) or just concatenate everything together?

I was inclined to train separate vectors for each field, but if some fields are missing for some samples, there'll be no representation for them, am I right?


texts: What is the number of neurons for the input layer of the BERT?

texts: I think it is the vocab size. However I am not sure and I appreciate your help.


texts: Naive Bayes for SA in Scikit Learn - how does it work

texts:
Okay so i scrape data from the web on movie reviews. I also have already got my own 'dictionary' or 'lexicon' with words and their labels (1-poor, 2-ok, 3-good, 4-very good, 5-excellent).

SO the input are paragraphs of movie reviews and i use Scikit Learn Naive Bayes to evaluate the sentiment of each comment , which would be a paragraph. I would like to know how it works under the hood. I ASSUME it uses a bag of words concept.

So , I am describing my assumption about how Scikit Learn Naive Bayes works for SA. How close am i to the reality ?

I am neither a data scientist nor a statistician, but this is a summary of what i THINK happens in  Naive Bayes algorithms for Sentiment Analysis, in Scikit Learn.

Training Phase

Lets assume i am using labels like 1,2,3,4,5 for each paragraph in the training set. Each paragraph is one unit for training as well as evaluation.

STEPS :-

1) Drop unwanted words like THE, BUT, AND and so on

2) Read the first word say 'BEACH', pick it's label from it's parent paragraph, say '5'. So attach
5 to BEACH and put it back in the bag.

3) So add up the number of times each word matched a given label.   Same word 'BEACH' could occur with multiple labels across the input paragraphs. So keep count of each label for the given word. So maybe ['BEACH' - label 1 - 10], ['BEACH' - label 2 - 8] and so on.

4) After above step for all words, sum up the probability of getting a certain label for a given word. So "BEACH" may have 1/6 probability of label 1, 2/6
probability of 2, 1/6 probability of 3, 1/6 probability of 4 and 1/6
probability of 5. So these 5 probabilities for 'BEACH" are put back in the bag of words. Remove the Word-label-count entities from Step 3 , which were used to obtain these probabilities. So now each word can have a maximum of 5 probabilities in the bag.

Evaluating Sentiment Analysis - New Data

1) Now when we feed the real data, when it hits a word, it checks if it is
found in its bag. If not it omits it

2) Say it finds BEACH in one comment. It checks highest probability
for 'BEACH' is 2/6 for label 2. So it picks label 2 for the word
BEACH. Similarly it picks corresponding labels with highest probability for all the words in bag matching the input.

3) Now it sums up all the labels for all words for the paragraph whose SA needs to be computed. Let us say they add up to 20. It has to
convert this 20 to somewhere between 0 and 1. So it uses a logarithmic
conversion which converts 20 to somewhere between 0 and 1. Let us say
0.75.

4) So 0.75 is the weight for this comment.

Is this how it works ? Thanks for any inputs.


texts: Semantic segmentation to instance segmentation

texts: Having a mask from semantic segmentation I want to split it to list of non-overlapping instances masks(like an output from instance segmentation). Do you know some fast algorithm, approach, tool or library to do this? I have found one python library. But it is very slow. Naive python implementation will also be slow.


texts: Loss function that penalizes under-predictions more than over-predictions?

texts:
I have a regression problem where I am predicting a continuous variable. Loss functions used most often in these cases (RMSE, MAE, etc.) don't treat over- or under- predictions differently.

I am in a scenario where under-predicting would be a much worse outcome than over-predicting.

What type of loss function would appropriately capture this?


texts: Speed Date or Dating Conversation Dataset / Corpora

texts: I'm working on an LSTM model that uses dating conversation dataset. Any suggestion as to where I can get such data? Any ideas will be much appreciated.


texts: Efficient algorithm to find the lowest value cluster in a series of values

texts:
Let's say I have a list of numeric values that tend to be grouped into some number of clusters of values that are close to one another.
I'm aware of things like k-means to group these into groups of similar values, but I'm curious as to whether there's a faster and perhaps simpler algorithm to simply find all the values in the first or lowest valued cluster.
Example: take this sequence:
9,8,22,45,46,41,20,19,7,10,11,20,21,24

There are three clusters: one near about 10, one near 20, and one near 42. The algorithm in question might return:
8,9,7,10,11

These values are part of the lowest valued cluster of grouped values. Whether the results are sorted or not doesn't matter.
I can think of some easy improvised ways but I'm wondering if there's a robust general solution.


texts:
We have some skill levels (beginner, advance, expert) which users assign themselves. Then they get some rating (2, 3, 4, 5) stars from others…. So an expert may have 2-star overall rating and an advance may have a 5-star overall rating. I think advance user with 5-star overall rating should be valued more than expert user with 2-star overall rating.

How can we compose a recommender algorithm with this reversal stuff?


texts: How does combining two linear perceptrons create non-linear boundaries?

texts:
I don't understand the equation that you get from combining the two linear perceptrons is non-linear?

The video starts with two linear perceptrons with the equations:
$$e1 = 5x_1 -2x_2 - 8 = 0 \hspace{10ex} e2 = 7x_1 - 3x_2  + 1 = 0 $$

  Note: The bias unit signs flip between the written equation and the
  neural network diagram. I am using the negative sign since it is
  continued throughout the rest of the video

Then we go on to combine them with respective weights and biases as follows:
$$ 7e_1 + 5e_2 -6 = 0$$
When I do the math, I get:
$$ 7e_1 + 5e_2 - 6 = 0 $$
$$ 7(5x_1 -2x_2 - 8) + 5(7x_1 - 3x_2 + 1) - 6 = 0$$
$$ 35x_1 - 14x_2 - 56 + 35x_1 -15x_2 +5 -6 = 0$$
$$ 70x_1 - 29x_2 - 57 = 0$$

The resulting equation is very much linear, however, the idea is that that this generates a non-linear equation (and model).

What am I doing incorrectly in the way I am combining the two models?


texts: Does testing a training dataset guarantee successful results?

texts:
If I test an image that has been previously used to train a classification model, is it guaranteed to classify correctly?
My guess is that since the parameters have been trained with other images as well, there is no guarantee of getting a correct classification, just a high probability.


texts: Same confusion matrix when changing DecisionTreeClassifier parameters

texts:
I'm trying to build my first Decision Tree Classifier using the Iris dataset in the sklearn library. This is my first sample code:
from sklearn.datasets import load_iris
from sklearn.metrics import accuracy_score
from sklearn.metrics import f1_score
from sklearn.model_selection import cross_val_score
from sklearn.metrics import confusion_matrix
from sklearn import tree
import numpy as np
import graphviz

iris = load_iris()

clf_ex1 = tree.DecisionTreeClassifier(criterion="entropy",random_state=300,min_samples_leaf=5,
                                      class_weight={0:1,1:10,2:10})

np.random.seed(0)

indices = np.random.permutation(len(iris.data))
indices_training=indices[:-10]
indices_test=indices[-10:]

iris_X_train = iris.data[indices_training]
iris_y_train = iris.target[indices_training]
iris_X_test  = iris.data[indices_test]
iris_y_test  = iris.target[indices_test]

clf_ex1 = clf_ex1.fit(iris_X_train, iris_y_train)

predicted_y_test = clf_ex1.predict(iris_X_test)

print(confusion_matrix(iris_y_test, predicted_y_test))

print("Predictions:")
print(predicted_y_test)
print("True classes:")
print(iris_y_test)
print("--------")
print(iris.target_names)

# print some metrics results
acc_score = accuracy_score(iris_y_test, predicted_y_test)
print("--------")
print("Accuracy score: "+ str(acc_score))
print("--------")
f1=f1_score(iris_y_test, predicted_y_test, average='macro')
print("F1 score: "+str(f1))
print("--------")

scores = cross_val_score(clf_ex1, iris.data, iris.target, cv=5)
print(scores)

dot_data = tree.export_graphviz(clf_ex1, out_file=None,
                         feature_names=iris.feature_names,
                         class_names=iris.target_names,
                         filled=True, rounded=True,
                         special_characters=True)
graph = graphviz.Source(dot_data)
graph

As you can see, in my DecisionTreeClassifier function I set the weight of the classes by giving a major value to the second one and the third one and I've given 300 to the random_state parameter. Then, I made another example by changing these parameters on this way:
clf_ex2 = tree.DecisionTreeClassifier(criterion="entropy",random_state=300,min_samples_leaf=5,
                                  class_weight={0:1,1:1,2:10})

and on this way:
clf_ex3 = tree.DecisionTreeClassifier(class_weight=None, criterion='entropy',
                                  max_depth=2,
                                  max_leaf_nodes=None,
                                  min_samples_leaf=15,
                                  min_samples_split=5,
                                  random_state=100,
                                  splitter='best')

The problem is that all the values that I print (the confusion matrix, the accuracy, the predicted_y_test and the f1 score) do not change between the three codes. The only value that gets affected is the Cross Validation Score. Why?


texts: Why do a lot of people use ipython notebook over python file when doing analyzing data? Is it the same in industry?

texts:
I have seen that a lot of people write code in ipython notebook when doing statistical analysis on data, apart from easy visualization after each step rather than running the whole code every time on a .py file. What are the other advantages,if there are any?

Also in industry which one is used more often?


texts: naive bayes classifier for non-binary feature values

texts:
Given a training set $\{{(x^{(i)},y^{(i)});i=\{1,...,m}\}\}$ where $x^{(i)}\in\{1,2,...s\}^n$ and $y^{(i)}\in{0,1}$. We model the label as a biased coin with $\theta_0=P(y^{(i)}=0)$ and $1-\theta_0=P(y^{(i)}=1)$. We model each non-binary feature value $x_j^{(i)}$ (an element of $x^{(i)}$) as a biased dice for each class. This is parametrized by:

$$P(x_j=k|y=0) = \theta_{j,k|y=0}, k = 1, \ldots, s-1;$$

$$P(x_j=s|y=0) = \theta_{j,s|y=0}=1-\sum_{k=1}^{s-1}\theta_{j,k|y=0};$$

$$P(x_j=k|y=1) = \theta_{j,k|y=1}, k = 1, \ldots, s-1;$$

$$P(x_j=s|y=1) = \theta_{j,s|y=1}=1-\sum_{k=1}^{s-1}\theta_{j,k|y=1};$$

[![enter image description here][1]][1]

(1) Using the Naive Bayes assumption, write down the joint probability of the data:
$$P(x^{(1)},\ldots,x^{(m)},y^{(1)},\ldots,y^{(m)})$$
in terms of the parameters $\theta_0, \theta_{j,k|y=0},$ and $\theta_{j,k|y=1},$ Using the indicator function 1(.) may be useful

(2) Maximizing the joint probability you get in (1) with respect to $\theta_0, \theta_{j,k|y=0},$ and $\theta_{j,k|y=1},$, write down your resulting $\theta_0, \theta_{j,k|y=0},$ and $\theta_{j,k|y=1},$ and show intermediate steps. Comment on the meaning of your results.

My attempt:

I know that $P(Y,X_1,X_2,\ldots,X_d)=P(Y)\cdot \Pi_{i=1}^{d}{P(X_i|Y)}$. To expand this to non-binary, I'm not sure how to proceed. Any help is appreciated.


texts:
I am new in deep learning and I am trying to build a classification module which can classify text to one of 9 classes and then use the result of the classification to classify them to another set of classes (sub-classes) and then again using the second classification result to classify them to (sub-sub-classes)

I hope that made sense one way or another, I will try to explain it better if need be, any guidance on the right track would be much appreciated

Edit: Whats the best deep learning algorithm for text classification? and should I use TenserFlow, Keras or PyTorch?


texts: Skewed Data doesnt follow normal distribution

texts:
My data (continuous) is highly skewed and it doesn't follow the normal distribution. Using sns.distplot I found out that exponweib fits the data better.

How to deal with this?

My end goal is to use the data for machine learning model (SVM).


texts: How to get KNN linearly hybridised by two similarities?

texts:
I'm writing a KNN (collaborative filtering) hybrid similarity recommender and I need some advice. It is based on this paper.
I've currently got 2 datasets. The first one is ratings (approximately 3500 films, 1,028,751 ratings) and the second one is personality (1812 users) where the personality of the user is measured using a FFM vector. They are from here.
The similarity between 2 users is calculated as follows:
$$sim(u,v) =  A*sim_{ratings}(u,v) + (1-A)*sim_{personality}(u,v) $$
The rating similarities and personality similarities are both done using a Pearson correlation, as opposed to only ratings similarity (which is supported in some recommender modules).
I read up on the module Surprise but as far as I can tell, I wouldn't be able to write my own algorithm as it won't support compound similarity calculation. I went on to write things from scratch i.e. build a personality-personality similarity matrix and then a ratings-ratings similarity matrix between users with the intention of combining them, but as you can imagine, that is very computationally intensive and I simply don't get the results in time, not to mention the testing that would then come afterwards. I even tried to parallelise some aspects, but I'm simply not skilled enough to do that (but if someone believes this is the way to go, please comment)
I'm not sure how to leverage sklearn for this use, but I'd be really open to hearing suggestions (for sklearn or other modules) as I feel totally at a loss.
Alternatively, if you know of a better architecture in literature, please do share. I'm currently using Python.


texts: How many trees does a Random Forest need?

texts:
At first, I did a GridsearchCV and the best parameter I found was 100, i.e., a random forest with just 100 trees. My trainset has 80,000 rows and 669 columns. My test set has 20,000 rows and 669 columns.
How is it possible that such small number of trees is enough?


texts: How can I replace some words in my text file by keywords provided in another file using python?

texts:
I'm new to Python.
I have two files. one file contains text, the second contain keywords.Here is a sample of the keywords file :

Parental authority

Rights and Responsibilities

Health and Safety

I want my code to parse my text, and whenever he finds a derivation of one of the keywords, he would replace them with the keyword as it is in my file.

Exemple : if "healthy and safe" is found in the text, he would replace it with "Health and Safety" mentioned in the keywords file.

I already have a function "Vocabulary" that does the lemmatization:
print(j --> Vocabulary[j]) would display a word and its derivations. Exemple :

Safety --> Safely, Safe

Here is my code :

import pandas as pd
import numpy as np
import spacy
from itertools import product
import re

text = pd.read_csv(name_file, sep=';', header=None, index_col=0)
text.shape
for i in text.index:
  document = i.lower
    for j in vocabulary:
       for t in vocabulary[j]:
        document.replace(t,j)

As a result, I'm having the following error :

AttributeError: 'builtin_function_or_method' object has no attribute 'replace'


texts:
Question

Is Poisson model the best method for predicting counts among multiple  levels within nominal variable?

Details
Imagine data of 7000 observations, where output= Obs.Count {numeric,0,1,2..8} and features=location {factor, 13 levels} .

When conducting Poisson regression, the output returns:

## function for glm
#p1 <- glm(Count ~ Loc,family = poisson, data = dat)

Call:
glm(formula = Count ~ Loc, family = "poisson", data = p1)

Deviance Residuals:
     Min        1Q    Median        3Q       Max
-2.49116  -1.32852   0.00775   1.02579   1.55985

Coefficients:
               Estimate Std. Error z value            Pr(>|z|)
(Intercept)    1.112766   0.032880  33.843 <0.0000000000000002 ***
LocLocation E -0.006774   0.039251  -0.173               0.863
LocLocation G -0.005369   0.045309  -0.118               0.906
LocLocation H -0.020067   0.039208  -0.512               0.609
LocLocation L -0.018632   0.044483  -0.419               0.675
LocLocation N -0.032309   0.039875  -0.810               0.418
LocLocation O -0.023674   0.044166  -0.536               0.592
LocLocation P -0.015914   0.044296  -0.359               0.719
LocLocation Q  0.019584   0.039434   0.497               0.619
LocLocation R -0.023361   0.039597  -0.590               0.555
LocLocation S  0.003840   0.039222   0.098               0.922
LocLocation U -0.033202   0.040190  -0.826               0.409
LocLocation V -0.007011   0.044393  -0.158               0.875
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for poisson family taken to be 1)

    Null deviance: 11647  on 6999  degrees of freedom
Residual deviance: 11642  on 6987  degrees of freedom
AIC: 29771

Number of Fisher Scoring iterations: 5

Result of dispersion testing:

    Overdispersion test

data:  p1
z = 19.391, p-value < 0.00000000000000022
alternative hypothesis: true dispersion is greater than 1
sample estimates:
dispersion
  1.306968

A couple of thoughts:

If I am reading the output from dispersion correctly, the model is good in this respect (true dispersion is >1).
The coefficients & pvals from the glm() seem to point away from any meaningful relationship between locations (based on count. This seems odd as I would expect that, with the count of 7000, there would be a least a couple of Loc observations that are related.

Is Poisson the right choice for modeling such data? Given the data is count, I hesitate to apply other methods (albeit, I did apply extensions, such as zero-inflated with similar results). The existing literature is sparse with alternatives to Poisson and even less so when wanting to identify relationships between counts of multiple levels of a categorical (nominal) variable.

Thoughts/suggestions would be welcome.


texts: Feature engineering and longitudinal data

texts:
I need some advice for my feature engineering.
Suppose I have 90 days follow-up data. on 12 patients and I have the vital status of the patients at the end of these 90 days (deceased=1, alive=0)
ID<-as.factor(c(1,1,1,2,2,2,2,3,3,4,4,4))
time<-c(0,12,36,0,7,23,68,0,23,0,32,45)
Age<-rnorm(12,45,9)
Sexe<-c("F","F","F","M","M","M","M","M","M","F","F","F")
biology1<-rnorm(12,12,3)
biology2<-rnorm (12,100,20)
biology3<-rnorm(12,45,9)
biology4<-rnorm(12,20,2)
sign1<-c(1,1,1,1,0,1,0,1,0,0,0,1)
sign2<-c(1,0,0,1,1,0,1,0,1,1,0,1)
stage<-c(3,3,4,2,2,1,1,3,2,3,2,3)
Death<-c(1,1,1,0,0,0,0,0,0,1,1,1)
data<-data.frame(ID,time,Age,Sexe,sign1,sign2,biology1,biology2,biology3,biology4,stage,Death)

Patients were seen at irregular rhythms where they benefited from clinical and biological examinations.
I would like to make a model of mortality prediction.
What kind of transformations will be adapted to the numerical variables for my case, and which take into account the evolutivity of the data, knowing that the data are not periodic. For the moment I am thinking of calculating the slopes for the numerical variables. What other transformation could I do that takes into account the evolutivity of the variables?
The variable "stage", is an ordinal variable and in principle the higher the stage, the more critical the state of the patient is, and the more likely he is to die, Would it be a good idea to do the One-Hot-Encoding? Or should I do Ordinal Encoding? And in the latter case is it a good idea to leave the variable with its 1,2, 3,4? (knowing that there is no interval property)
What type of encoding could I do that takes into account the evolutivity of the binary variables (sign1 and sign2)?  For example patient 1 has 1,then 0, then 0 for sign 2 Would it be a good idea to make a slope here as well?


texts:
I understood that Mercer's Theorem extends the definition of kernels also for infinite input space.

In Machine Learning realm our training set is always finite and hence the input space is always finite. So why are we interested in infinite input spaces?

Where is the flaw in my reasoning?


texts: 2D Pytorch tensor doesn't have independent random values

texts:
I've written some Python to create a pytorch tensor of random values, sampled from a Student's t distribution with 10 degrees of freedom:

t = torch.Tensor(())
def random_from(shape):
    return torch.distributions.StudentT(10, t.new_zeros(shape), t.new_ones(shape)).sample()

If shape is of the form $(n, m)$, all values in the resulting 2D tensor are identical. I don't understand why. I did try reading Pytorch's documentation, but I couldn't find anything that helped me understand what would be a better syntax. I suppose I could create separate samples then concatenate them, but apart from the speed implications I'd like to know where my existing syntax goes wrong.


texts: Deep Learning Binary Text Classification

texts:
I am looking to build a model for specific news and blog articles which merge fashion with patterns in biology. I have 35 websites that I read daily (it is exhausting). I am wondering how to approach creating such a model so that I can send articles to it daily and it can predict whether or not they deserve reading (ie relevant vs irrelevant). For example, in a perfect world I send it 230 articles of which 12 are returned as relevant based on previous training and testing articles. Assuming all 12 are relevant I save them to the training dir and recompile  to strengthen the model.

After reading Deep Learning with Keras I was hoping Chapters 5 and 6 on Word Embeddings and RNNs(simple, LSTM, GRU) would point me to how to develop such a model. I was also reading how to implement a CNN for text classification but cannot seem to construct a basic conceptual framework for how to start with a few articles and adding to the training data every day to strengthen the model.

Is there a term for this type of strategy? Has it been done in some form on Kaggle, Github, etc?


texts: Manually creating plants images dataset for machine learning plant type classification and leaf segmentation

texts:
A group I work with wants to create its own plants data set that will be used for multiple projects like plant type classification and leaf segmentation for starters.
They are willing to provide all the means necessary for the creation of this data set. A partner agriculture company is willing to assist with planting what ever is needed and have someone monitor and capture images every X duration as per requirements. Different growth stages are desired as well.

I was asked to write the requirements for how the images should be acquired, however I'm not quite sure of that myself. What are some key points one should know for such a task and are there any known guidelines or tutorials available for data sets creation?

Note that currently for this task only 1 plant will be captured in each image as the non-learning methods that will be used to label and segment the images support only 1 plant per image.

I was thinking that based on the time it takes a specific plant to fully grow (which is known, approximately), N images should be captured every X days covering 360° of the plant. Also, images are to be captured from different angles and distances.

Some questions I have in mind that I cant seem to answer yet,

Number of images, the more the merrier as long as there are differences?
What about the location of the (single) plant in the image? Is it crucial or could it be augmented after the data set has been acquired?
What about camera resolution? is any decent camera sufficient for the task or should it be high end?

I'm still researching what other considerations there are and would appreciate any information in that matter


texts: What are the approaches to aggregate categorical variables?

texts:
I am working on a clickstream dataset. I have come up with the following example dataset to explain my problem:

ClickTimeStamp        | SessionID | ART_weekOfYear | PagenameClicked | TimeSpentPerSession | CustID | ContractID | ... | TARGET |
2017-01-04 16:48:00   | 1         | 1              | P1              | 1                   | abc    | xyz        |     | 1      |
2017-01-04 16:48:53   | 1         | 1              | P2              | 1                   | abc    | xyz        |     | 1      |
2017-01-11 10:09:57   | 2         | 2              | P1              | 2                   | abc    | xyz        |     | 1      |
2017-01-11 10:11:24   | 2         | 2              | P4              | 2                   | abc    | xyz        |     | 1      |
2017-01-27 13:22:39   | 3         | 4              | P1              | 2                   | abc    | mnp        |     | 0      |
2017-01-27 13:24:01   | 3         | 4              | P7              | 2                   | abc    | mnp        |     | 0      |

The above dataset has clicks on its each row and TARGET is (let's say) contract was retained (1) or not (0). Keep in mind the TARGET is at contract level.

Now, I aggregate the above dataset as per my need (i.e. aggregate on contractID) and training set looks like this:

CustID | ContractID | ... | SessionID_conct | ART_weekOfYear_conct | PagenameClicked  | TimeSpentPerSession_avg | TARGET |
abc    | xyz        |     | "1-2"           |"1-2"                 | "P1->P2->P1->P4" | 1.5                     | 1      |
abc    | mnp        |     | "3"             |"4"                   | "P1->P7"         | 2                       | 0      |

PROBLEM:
For numerical features I just took average (as for TimeSpentPerSession_avg) but for categorical features it is not straightforward. In reality, my categorical features have very high cardinality, such as "PagenameClicked". So I cannot simply convert my categorical features to dummy variables and then aggregate them as numerical features.

I would like to know possible solutions to treat categorical features in such a way that dimensionality doesn't explode and I can also aggregate new representation on the contractID.

I have tried Entity Embeddings and read this paper for details. I transformed each categorical feature to an embedding representation of 16 dimension. However, now I am stuck at aggregating these embedding vectors for each contractID. Kindly let me know if anyone has worked in this direction or has a better solution.

Thanks allot for reading this question. :)


texts: Algorithm Suggestions for a Self Project

texts:
So, I am doing a small self project on data analytics. I am collecting the android apps data from the play store sites by web scraping.
I am basically trying to collect all possible information related to a particular app, like it's type/genre and sub-genre, ratings, size, number of downloads, if it's paid/unpaid, and other possible information regarding the apps, wherever possible.

I was wondering after I'm done with the data extraction and cleaning work, what data analytics related algorithms I might be able to apply on my play store apps data to analyze it and find patterns in it, predict the links amongst the data and do other predictions and data analysis on it?

A few, I came across were :- TFID, Clustering, Decision trees, Sequence Analysis and maybe Regression.

So, please review the above ones and suggest any new algorithms, which might be useful to me for my project. Some specific algorithms under the above mentioned broad algorithm(if there are, which might be useful for this project ) please suggest them.
Details on them will very much helpful.

Thanks!


texts: Why are ensembles so unreasonably effective

texts: It seems to have become axiomatic that an ensemble of learners leads to the best possible model results - and it is becoming far rarer, for example, for single models to win competitions such as Kaggle. Is there a theoretical explanation for why ensembles are so darn effective?


texts: Why deep feedforward neural networks are restricted to solve classification problems?

texts:
I want to use deep neural networks for regression problems, but as far as I've read through, it's mainly for classification. I was wondering why can't a regular convolutional network, or a multilayer perception work as a linear regression, just with multiple layers.
I tried connecting two convolutional layers with max pooling on each, and a fully connected layer together, without activation function, with 10000 MNIST images, 100 epochs, and with 100 batch size. Used mean squared error as the loss function, and regular stochastic gradient descent as the optimizer. Instead of one hot encoding, I used the actual expected numbers as labels.
The results are very promising even though I haven't played optimizing the parameters. The loss after the training is under 1, and I've got R2 values over 0.9 at testing.
So the question is, why neural networks are mainly used for classification while it could handle regression as well?


texts: How can I use my own dataset for Image segmentation using Tensorflow

texts:
I have a huge problem using my own created dataset for image segmentation using Tensorflow. The dataset that I've build contain images like the one shown below:

The problem that I have is: How do I use my own dataset specifically for image segmentation? I've looked at the documentation on how to create datasets but all the examples either only use object detection with a single class or classify the entire image. I want to assign every pixel to a class (image segmentation) and use it to train my model.
I've also found examples of image segmentation but they all use existing datasets such as Cityscapes, ADE20k etc. How can I use my own images and data and transform it to a tensorflow dataset so that I can use it for training?


texts: Trying to figure out how to set weights for convolutional networks

texts:
I am working on CNN, and I have some doubts. Let's assume I only want one feature map, just to make things easier. And let's suppose my image is grayscale, to make things even easier. So, let's say my image is (32,32) --grayscale, hence just a channel and we don't need to write it explicitly, and my filter is (3,3) --again, one feature map, so I won't bother writing 1.
I understand this will map to a (30,30) layer.

How many parameters will I have? If I understand it correctly, I will have 9 weights and one bias, so a total of 10, because we map each (3,3) subregion using the same weights. Back-propagation will determine the best values for those weights and that will give me one feature map, or a filter.

So far, so good. What I don't understand is how does the training work? I need to keep the same weights and bias when moving across the image (that's why I only have 10 parameters), but won't those change when I do back-propagation? How can I apply back-propagation and keep the same values for the weights regardless of the subregion they are applied to?


texts:
I've created a pipeline using tf.Data (or more accurately a mix of Pandas and then tf.Data). My specific requirement is to create sequences using the window command (which applies a moving window with a specified window size, step and stride) and then batch these windows in order to train time series RNN's. The network is actually implemented in JAX/FLAX so I use as_numpy_iterator() to cast from Tensor to np.array

I'm considering moving to PyTorch and would like to migrate my pipeline. What is the equivalent in PyTorch? I've found torch.utils.data but I don't see any mention of windowing etc.


texts: Is over-fitting a matter of features engineering or a matter of parameters set?

texts:
I am using sklearn package to make models.

I tried randomly to set some paramater to a sklearn.ensemble.RandomForestClassifier in order to have ideas about parameters to search for a future grid search.

Well, here the results of this attempt:

Accuracy :
Training: 0.9209427371993345 , Test: 0.7035561005518087

Clearly, I know it means the model is over-fitting because it's not able to generalize to new data. I know a cross-validation would be more accurate because it is possible the test-set is unfortunately too different from training-set just by random, but that's not what I am looking for in the current problematic of my topic.
Besides, the two classes I try to target are imbalanced. Class 1 is more present than Class 0.

Class 0: 34% of test set/training set, Class 1: 66% of test set/training set.

So because it's imbalanced I checked about precision and recall metrics:

On test set:

+---------+-----------+--------+----------+----------+
| classes | precision | recall | F1-score | support  |
+---------+-----------+--------+----------+----------+
|       0 |      0.60 |   0.38 |     0.47 |     3326 |
|       1 |      0.73 |   0.87 |     0.80 |    6460  |
+---------+-----------+--------+----------+----------+

On training set:

+---------+-----------+--------+----------+----------+
| classes | precision | recall | F1-score | support  |
+---------+-----------+--------+----------+----------+
|       0 |      0.98 |   0.81 |     0.89 |   31265  |
|       1 |      0.91 |   0.99 |     0.95 |  59492   |
+---------+-----------+--------+----------+----------+

Then, on training set I see both classes are well predicted. Intuitively I think: if it's well predicted on the training set, it means features are good enough to split the two classes. So, it's just a matter of parameters set.
But intuition is not as valuable as good experience is. So I am asking for more experienced people from this community if my intuition is wrong, and if so why it is?

UPDATES:

After checking variables, it shows distribution are the same between the training-set and testing-set through histograms.


texts: What models do Create ML and Turi Create use

texts:
I'm taking a course on Apple's machine learning technologies.  I just came across this paragraph:

  Turi Create and Create ML are task-specific, rather than
  model-specific. This means that you specify the type of problem you
  want to solve, rather than choosing the type of model you want to use.
  You select the task that matches the type of problem you want to
  solve, then Turi Create analyzes your data and chooses the right model
  for the job.

My question is when you select a task like binary image classification, and Create ML / Turi Create selects an appropriate model for you, what models does it have at its disposal.  Aren't there MANY models in the market that do this?  Did Apple simply pick the one they thought was best?

Thanks!


texts: MAPE over 100% after normalization of dataset

texts:
I try to forecast power demand for next 24 hours. Years 2017 and 2018 are my training set, 2019 is test set. I use multistep vanilla LSTM . In first step I used original data with any preparation and I get MAPE score about 7,5%. Then I read some article about doing the same thing in Matlab and they use to types of normalization of dataset:
First one
$$
P_n(d,h)=\frac{P(d,h)}{max_P}
$$
Second one
$$
P_n(d,h)=\frac{((P(d,h)-mean(P(d)))}{std(P(d))}
$$
When I use first one Make drop to 3,5% and in article second one gets even better scores. But when I use it my forecast was very wrong with MAPE 130% and very big errors. I don't find in internet anyone with the same problem. I tried using MinMax scaler but MAPE was around 20%. I think it have something with very small values because when modified second normalization to have every value greater than zero MAPE drop to 78%.
this is my neural network

def build_model(train, n_input):
    # prepare data
    train_x, train_y = to_supervised(train, n_input)
    # define parameters
    verbose, epochs, batch_size = 2, 100, 72
    n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]
    # define model
    model = Sequential()
    model.add(LSTM(400, activation='relu', input_shape=(n_timesteps, n_features),return_sequences=True))
    model.add(LSTM(200, activation='relu', input_shape=(n_timesteps, n_features)))
    model.add(Dense(100, activation='relu'))
    model.add(Dense(100, activation='relu'))
    model.add(Dense(n_outputs))
    model.compile(loss='mse', optimizer='RMSprop')
    # fit network
    model.fit(train_x, train_y, epochs=epochs, batch_size=batch_size, verbose=verbose)
    return model

this is link to my Colab colab

this is normalization

#first one
load_norm=data['load'].max()
data2=data.copy()/data['load'].max()

#second one
import datetime
def no_trend(x):
  mean=x.mean()
  std=x.std()
  return (x-mean)/std
def no_trend_transform(data):
  index_range=data.copy()
  index_range=index_range.resample('D').mean()
  index_range.reset_index(inplace=True)
  index_range['index']=index_range['index'].apply(lambda x: x.strftime("%Y-%m-%d"))
  for i in index_range['index']:
    data[i]=no_trend(data[i])
  return data.dropna()
data2=no_trend_transform(data.copy())

Dataset dataset

Do you have any idea what is a problem? I tried this that a set with other types of LSTM and MLP and I have the same problem.


texts:
On a CNN, what is use of using Activation function in convolution layer?

Does single weight is used for full matrix or for every pixel or box it may vary?


texts: Evaluation metrics for Decision Tree regressor and KNN regressor

texts:
I have started working on the Decision Tree Regressor and KNN Regressor.

I have built the model and not sure what are the metrics needs to be considered for evaluation. As of now I have considered Root mean squared error.

Can we use $R^2$ values for Decision Tree and KNN or is it applicable only to Linear Regression Model

I have also pasted the $R^2$ and MSE values.
As per my understanding Linear regression model is stable, Decision tree is over fitting and KNN is having less error. Which model needs to be considered here?

R2 Score for train - KNN regression 0.8215942683102192
R2 Score for test - KNN regression 0.7160388084850589
Mean squared error for train - KNN regression 49.92162362176166
Mean squared error for test - KNN regression 78.30907381395349

R2 Score for train - linear regression 0.6141419744748021
R2 Score for test - linear regression 0.6117893766210736
Mean squared error for train - linear regression 107.97107771851036
Mean squared error for test - linear regression 107.0583420197463

R2 Score for train - Decision Tree regression 0.9962039204515297
R2 Score for test - Decision Tree regression 0.7866182225490949
Mean squared error for train - Decision tree regression 1.0622217832469776
Mean squared error for test  - Decision tree regression 58.84511637596899

Updating the AIC and BIC values

AIC value - Test - KNN       :   -3.8272461328797505
BIC value - Test - KNN       :   1169.4748549110836
AIC value - Test - Linear Reg:   -4.452667046616746
BIC value - Test - Linear Reg:   1250.154152783156
AIC value - Test - Decision T:   -3.2766787253336602
BIC value - Test - Decision T:   1098.4516593376377

Thank you.


texts: How to approach this multi label classification problem and what will be its accuracy metric?

texts:
I have a dataset for people doing trade in various segments (classes) .I am trying to build a multi-label classifier to predict people trading in various segments (classes).

My dataset :

Client_id [Demographic data] orders_perday Traded_days Segment
 A123      ..............     3             4           equity
 A123      ..............     2             2           commodity
 A123      ..............     1             9           currency
 B789      ..............     7             8           equity
 B789      ..............     3             2           futures
 C456      ..............     2             7           currency
 D987      ..............    10             1           equity
 C183      ..............     2             9           currency

Demographic data includes age, sex, city, income ,etc.

I need to build a model which will predict new clients belong to which class(segment) can be 1 or more.
Please give some advice of how to approach this problem


texts: Interpretability of RMSE and R squared scores on cross validation

texts:
I'm working on a regression problem with 30k rows in my dataset, decided to use XGBoost mainly to avoid processing data for a quick primitive model. And i noticed upon doing cross-validation that there's a noticeable difference between R² for train and R² for CV => clear signs of overfitting. Here's my code for CV :

oof_train = np.zeros((len(train_maisons)))
ind = 0
cv_scores = []
train_scores=[]
for ind,(ind_train,ind_val) in (enumerate (kfolds.split(X,y))):
    #ind=+1
    X_train,X_val = X.iloc[ind_train],X.iloc[ind_val]
    y_train,y_val = y.iloc[ind_train],y.iloc[ind_val]
    xgb = XGBRegressor(colsample_bytree=0.6,gamma=0.3,learning_rate=0.1,max_depth=8,min_child_weight=3,subsample=0.9,n_estimators=1000,objective='reg:squarederror',eval_metric='rmse')
    xgb.fit(X_train,y_train)
    val_pred = xgb.predict(X_val)
    train_pred = xgb.predict(X_train)
    oof_train[ind_val] += val_pred
    score_fold_validation=np.sqrt(mean_squared_error(y_val, val_pred))
    score_fold_train=np.sqrt(mean_squared_error(y_train, train_pred))
    train_scores.append(score_fold_train)
    cv_scores.append(score_fold_validation)
    #r2_score(y_val, grid.best_estimator_.predict(X_val))
    print('Iteration : {} - CV Score : {} - R² Score CV : {} - Train Score : {} - R² Score train : {}'.format(str(ind+1),score_fold_validation,r2_score(y_val, val_pred),score_fold_train,r2_score(y_train,train_pred)))
end_train_score=np.mean(train_scores)
train_scores.append(end_train_score)
end_cv_score=np.mean(cv_scores)

Using SquaredError as objective ( loss function ) , evaluating with RMSE and R², here are the metrics' outputs :

CV Score : 96416.84137549331 - R² Score CV : 0.6545903695464426 - Train Score : 30605.655815355676 - R² Score train : 0.9730563148067477

My question: is this considered an overwhelming overfitting problem? or is it mild? and should I do more feature engineering or tune hyperparameters more? ( used GridSearchCV for the current hyperparameters ). And one last thing, is my result on X_train indicative that my features are informative enough to learn the target? or is the R² train score somehow biased?

Note : In this code i'm using 10 folds for CV. Used 3 folds gave me a better result on CV, if someone can also explain that , it would be great.


texts:
I have a CNN architecture with two cross entropy losses $\mathcal{L}_1$ and $\mathcal{L}_2$ summed in the total loss $\mathcal{L} = \mathcal{L}_1 + \mathcal{L}_2$. The task I want to solve is Unsupervised Domain Adaptation.
I have attested the following behavior:

The gradients coming from $\mathcal{L}_1$ have a different magnitude than those coming from $\mathcal{L}_2$ such that the supervision coming from the first loss is negligible.
$\mathcal{L}_1$ has a positive constant value and does not decrease during the training, while $\mathcal{L}_2$ does decrease.

How can I minimize $\mathcal{L}_1$ and how can I make the gradient from $\mathcal{L}_1$ more important? Currently I have two options:

Add a tradeoff parameter to one of the two losses $\mathcal{L} = \mathcal{L}_1 + \gamma \cdot \mathcal{L}_2$
Normalize the gradients at some step

The last option would be to leave everything as it is, with the motivation that one loss does not provide supervision to the task I want to solve. Do you have some advice on the road to follow?


texts: How to calculate a location prior in CT's based on atlas segmentation?

texts:
I'm working myself through the paper "Anatomical Priors in Convolutional Networks for
Unsupervised Biomedical Segmentation" (https://arxiv.org/abs/1903.03148)

I lack to understand how the atlas based prior is calculated in 4.3, any hints for me? I've spent some time on research about this, but unfortunately couldn't find a satisfying answer for myself..

  The decoder is a mirror of this design, but upsamples instead of
  downsampling and ends with a sigmoid activation. In addition, we use a
  final layer that implements a pixelwise spatially-varying voxel-wise
  (location) prior ploc(s), which is multiplied with dec(z) (in
  practice, we add the logarithms). As is common in the atlas-based
  segmentation literature, the prior ploc(s) was computed as the
  frequency of labels in the held out prior dataset, in
  affine-normalized coordinate system. This layer discourages any
  extreme decodings of z but does not capture shape properties, which is
  encoded in dec(z).

Thanks!


texts: How to split the data as per the label name count using sklearn

texts:
Before training a model, I would like to split the data in a 80/20 ratio.

For example, in my dataset of 3000 rows, I have different labels. Out of which A is one label name, has 100 records in the dataset. How can I divide the dataset so that 80% of the label A data is in training and 20% is in the test set.

Do we have any pre defined functions for this?


texts: What's a good machine learning algorithm for low frequency trading?

texts:
I'm trying to train an algorithm to copy some of the top traders on various forex social trading sites. The problem is that the traders only trade around say 10 times per month so even if I only look at minute resolution numbers that's .02% of the time [ 10/(60*24*30)*100 ].

I've tried using random forest and it gives an error rate of around 2% which is unacceptable and from what I've read most machine learning algorithms have similar errors rates.

Does anyone know of a better approach?


texts: Generative adversarial networks for multiple distribution noise removal

texts:
I am working on a project where I need to denoise images, and my dataset is composed of a big chunk of pairs <natural image, same image with synthetically added noise>. The fact is, I have multiple sources of noise (with labels), e.g. Gaussian noise, salt&pepper, distortion, saturation to name a few. Different noises types are on the same original images, meaning that for each undistorted image, I have one pair for each noise type, which I believe is relevant for training.

Thanks to the recent success of GANs for image translation tasks, I was looking into recent architectures and how I can adapt them for my task. The question is: do you think that is possible for a GAN to learn a many-to-one mapping between distributions, i.e. different noise distributions (the many) and the undistorted image distribution (the one) or do I need to train multiple networks for different kinds of noise?

Given that I have paired images I was looking into the pix2pix architecture. What about the recent CycleGAN? Learning the backward mapping function looks a lot tougher if there are multiple distributions, which I guess is why when they release the pre-trained models they indeed have multiple models for multiple tasks.


texts: What are some appropriate models to use for inventory forecast based on consumption history or trends?

texts:
I am working on an inventory management system where I have daily/monthly/yearly consumption history for a particular item, which may or may not follow a repeating trend. In order to forecast demands within an upcoming time-frame, which model / distributions are recommended?

Data Format: Data is a series of (x, y) pairs where x is a timestamp and y is count of consumption.

As far as I studied, random forest or tree based models do not handle trend patterns effectively, while linear models do not seem to be appropriate either given the fact that data do not seem to be following a linear pattern.

Is this problem solvable using a single model? Should Linear Regression models be used for this problem?


texts:
I have trained a DL image classification model. The model size is 250 MB. I want to expedite the inference timing for both GPU and CPU run. On searching a bit, the general trend for faster inference on GPU is using NVIDIA Tensor RT. While on CPU, it is to do pruning or quantization.
I read a few articles about doing it on C++ but most of them are not succinct.
Can anyone please suggest the tried and tested way to solve the problem, I am a beginner and can't really find the mental framework to approach this problem. I can also save the model in ONNX and use TensorFlow framework, if they are faster and easier to use.
(Also if this is not a place ask questions on ML-Ops then please let me know the suitable forum.


texts: Why are precision and recall used in the F1 score, rather than precision and NPV?

texts:
In binary classification problems it seems the F1 score is often used as a performance measure. As far as I've understood the idea is to find the best tradeoff between precision and recall. The formula for the F1 score is symmetric in  precision and recall. However, (and that's what bothers me) there is an asymmetry between precision and recall. While recall is a property of the classifier that is independent of prior probabilities, precision is a quantity that does depend on the prior probabilities.

Can anyone tell me what's so special about the combination of precision and recall? Why don't we use precision (which is the positive predictive value) and negative predictive value?


texts: Questions about LSTM cells, units and inputs

texts:
I'm trying to learn how LSTM networks work, and even if I get the basics, the details of the internal structure is not clear for me.

On this blog link, I found this scheme of a LSTM architecture

Where apparently, every circle should correspond to an individual LSTM unit like this

Is this correct?
Is each unit in the cell independent from the others? Or do they share information?
Imagine I have the following configuration:
Number of samples = 1000
Number of time-steps = 10
Number of features = 5

In this case, each unit in a cell will take as input a vector of size 5 right?

But what will be the size of the output for one unit? 1?

Thanks


texts: how to get the number of columns that have the same value between 2 rows

texts:
I want to compare two records and get the number of columns that have the same value , for example
I have two rows, the first row is :

rank
rank1
rank2

1
1
3

and the second row is

rank
rank1
rank2

1
1
2

the result should be 2 columns have the same value. is there a function in python that is capable of performing something like this?


texts: Interactive heatmap slice as separate plot for browser

texts:
I want to create an interactive online version of a static plot I generated with gnuplot5. I have limited experience with generating dynamic plots for webpages and need a rough direction where to go to waste the least amount of time researching and trying different frameworks.

desired plot: 2d heatmap with of a file with 3d data. mouse position over plot is selector for 2d slice of heatmap which is displayed next to the heatmap. (bonus realtime capabilities)

Is D3 a viable candidate?

Thank you in advance


texts: Why are support vector machines good at classifying images?

texts:
I'm trying to figure out if I can identify styles of art using support vector machines and I don't quite understand what makes them good. Perhaps I can make the question more accurate. Support vector machines are basically a way of doing classification with a linear model. You find $\vec{\beta}$ and $\alpha$ such that $sgn(\beta \cdot x + \alpha)$ determines what the class is. So I believe that in order to do image classification, you need logic. It does not suffice to find the average color of an image and decide whether it is greater or smaller than a number to decide which number an image represent. You need logic. I believe you need intermediate logic, like comparing one part of the image to another part of an image. Kind of like a multilayered support vector machine, which now looks like a neural net. Is this intuition correct?

Sincerely,
Still figuring it out.


texts: Can pooling ever increase accuracy in convolutional neural networks?

texts:
In ConvNets, pooling is used to downsize the input volume, leading to fewer parameters, leading to computational efficiency and possibly helping with overfitting.

But can pooling ever increase the accuracy of your CNN model? I suppose this could happen if your CNN was overfitting when pooling was not used, and introducing pooling prevented overfitting, which could lead to better accuracy performance during testing. But if overfitting was not a problem to begin with, and you applied pooling, is it possible for accuracy to increase as a result of the pooling?
I don't see how this would happen since pooling is essentially making a lower-order computation, so your accuracy should always be $\leq$ un-pooled CNN?


texts:
newbie speaking:

Commonly we can say that accuracy is defined as total positive/ total nbr cases.

But I read that, when it is a binary classifier we should consider: TP+TN/ total nbr cases.

Can someone explain to me why?
Thanks


texts: How to understand Inconsistent and ambiguous dimensions of matrices used in the Attention layer?

texts:
Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original "Location-based Attention" in Bahadanau well-known paper at https://arxiv.org/pdf/1506.07503.pdf. (it seems this attention is used in various forms of GNMT and text-to-speech sythesizers like tacotron-2 https://github.com/Rayhane-mamah/Tacotron-2).

Even after repeated readings of this paper and other articles about Attention-mechanism, I'm confused about the dimensions of the matrices used, as the paper doesn't seem to describe it. My understanding is:

If I have decoder hidden dim 1024, that means ($s_{i-1}$} vector is 1024 length.
If I have encoder output dim 512, that means $h_{j}$ vector is 512 length.
If the maximum number of inputs to the encoder is 256, then the number of $j$ can be from 1 to 256.
Since $W \times S_{i-1}$ is a matrix multiply, it seems $\text{cols}(W)$ should match $\text{rows}(S_{i-1})$, but $\text{rows}(W)$ still remain undefined. The same seems true for matrices $V$, $U$, $w$, $b$.

This is page-3/4 from the paper above that describes Attention-layer:

I'm unsure how to make sense of this. Am I missing something, or can someone explain this?

What I don't understand is:

What is the dimension of previous alignment (denoted by $\alpha_{i-1}$)? Shouldn't it be total values of $j$ in $h_{j}$ (which is 256 and means total different encoder output states)?
What is the dimension of $f_{i,j}$ and convolution filter $F$? (the paper says F belongs to $k\times r$ shape but doesn't define $r$ anywhere). What is $r$ and what does $k \times r$ mean here?
How are these unknown dimensions for matrices $V$, $U$, $w$, $b$ described above determined in this model?


texts: What are practical differences between kernel k-means and spectral clustering?

texts:
I've been lately wondering about kernel k-means and spectral clustering algorithms and their differences.

I know that spectral clustering is a more broad term and different settings can affect the way it works, but one popular variant is using K-means clustering on the spectral embedding of affinity matrix.
On the other hand kernel K-means applies K-means clustering directly to the affinity matrix. Therefore one immediate, theoretical difference is it omits spectral embedding step, i.e. it doesn't look for the lower-dimensional representation of data with eigenvectors.

I think it could be beneficial in high-dimensional settings (with many observations to cluster), but does it provide any boost with small sample size, for example from 10 to 20 observations?

What are other practical implications of using either of these algorithms versus another (e.g. which one is more sensitive to any changes in affinity etc.)?


texts: Is it possible to create a neural network with two inputs, with sequential layers?

texts:
Is there a natural way, in terms of structure of the layers of a NN, in order to pass 2 inputs vectors to the NN?

Example: text authorship identification

Input #1: sentence1 by unknown author encoded, as list of words from a dictionary

"The sky is blue" => x1 = [2, 23, 7, 76, 0, 0, 0, ..., 0] (zero-padded to have a length of 1000 items)
Input #2: sentence2 by unknown author, idem

"The cat is sleeping" => x2 = [2, 65, 7, 121, 0, 0, 0, ..., 0]
Output: y = a single number in [0, 1].
0 = different authors
1 = same author
0.9 = high probability of same author, etc.

Of course I could build the layers like this:

  Input-size: None, 1000, 2    (x1, x2 stacked into a 1000x2 matrix)
  CNN ...
  ...
  Dense: None, 1

Then I could train with a dataset with 10,000 pairs of sentences of the same author (desired output: 1), and 10,000 pairs of sentences with different authors (desired output: 0).

But I don't know if it would work by just stacking x1 and x2 as a 1000x2 matrix.

TL;DR: What is the classical approach to build a NN taking 2 inputs and a single number as output which is a similarity index from 0% to 100%? (if possible with Sequential structure)


texts: Which metric to choose for tracking model performance?

texts:
I am working on a binary classification problem with class proportion of 33:67.

Currently what I am doing is running multiple models like LR,SVM,RF,XgBoost for classification.

RF and Xgboost models perform better.

However I am reading online that AUC, F1-score or Accuracy comparison between models may not be a good metric (as they are sensitive) to measure the model performance.

I see these are the mostly used scoring metrics but can you let me know which metric should I look for measuring binary classification model performance?

Can you let me know why should we choose a different metric?

These are the metrics that I see in scikit-learn


texts:
I am using kmeans to cluster some data with 2 features. Not sure I understand why kmeans is producing the clusters I see:

Why would kmeans not cluster these points in a way that matches what we would expect visually looking at the data? Why are seemingly random points in the middle of visual clusters being put into a second cluster like that?

The code I am running:

cols = ['col1', 'col2']
features = map(lambda x: df[x], cols)
input = np.matrix(list(zip(*features)))

scaler = StandardScaler()
scaler.fit(input)
input_scaled = scaler.transform(input)

algo = KMeans(n_clusters=2)
algo.fit(input_scaled)
df['cluster'] = pd.Series(algo.labels_)

sns.lmplot(x=cols[0],y=cols[1],data=df, fit_reg=False, hue='cluster')


texts: Backpropagation: Relevance of the error signal of a neuron

texts:
During my quest to understand back propagation in a more rigorous approach I have come across with the definition of error signal of a neuron which is defined as follows for the $j^{\text{th}}$ neuron in the $l^{\text{th}}$ layer:
\begin{eqnarray}
  \delta^l_j \equiv \frac{\partial C}{\partial z^l_j}
\tag{1}\end{eqnarray}

Basically, $\delta^l_j$ measures how much the total error changes when the input sum of the neuron is changed and is used for calculating the weights and biases of the neural network as follows:

\begin{eqnarray}
  \frac{\partial C}{\partial w^l_{jk}} = a^{l-1}_k \delta^l_j
\tag{2}\end{eqnarray}

\begin{eqnarray}  \frac{\partial C}{\partial b^l_j} =
  \delta^l_j.
\tag{3}\end{eqnarray}

Besides being useful for the calculation of the weights and the biases in the sense it makes it possible to reuse its value several times, is there any other reason why this definition is always brought up when addressing back propagation?


texts: ValueError when reshaping to 4 dim array for the Keras API

texts:
DL beginner here. I'm trying to implement LeNet using Keras and apply it on good ol' MNIST.

class LeNet:
@staticmethod
def build(height, width, depth, classes):
    model= Sequential()
    inputshape= (height,width, depth)

    if K.image_data_format()== 'channels_first':
        inputshape= (depth, height, width)

    #build model
    model.add(Conv2d(20, (5,5), padding= "same", input_shape= inputshape))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size= (2,2), strides=(2,2)))

    #replicate what's above again
    model.add(Conv2d(50, (5,5), padding= "same"))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size= (2,2), strides=(2,2)))

    #Fully Connected Layers
    model.add(Flatten())
    model.add(Dense(500))
    model.add(Activation('relu'))

    #softmax
    model.add(Dense(classes))
    model.add(Activation('softmax'))

    return model

Then I load the dataset

print('[INFO] Accessing MNIST...')

dataset= loadmat('mnist-original.mat')
data=dataset['data']

Now I'm trying to reshape the data whose current shape is (784, 70000) by

data= data.reshape(data.shape[0], 28, 28, 1)

But I get an error that says ValueError: cannot reshape array of size 54880000 into shape (784,28,28,1)

Where am I going wrong here? Please help. Thanks.


texts: distribution difference between image and text

texts: Once for the task of image captioning I've read that, the features extracted from image and text by deep networks are from two different worlds and got different distribution. My question is how is the distribution in two of them and how are they different?


texts: Tensorflow does not learn - same answer for various inputs

texts:
The code:

import numpy as np
import tensorflow as tf
import pandas as pd
from sklearn.model_selection import train_test_split

x_data = np.linspace(0, 1000, 100000)
y_true = np.square(x_data)
y_true += np.random.randn(len(x_data))

feature_columns = [tf.feature_column.numeric_column('x', shape=[1])]
estimator = tf.estimator.DNNRegressor(feature_columns=feature_columns,         hidden_units=[2, 2], optimizer=lambda:
                                  tf.train.AdamOptimizer(
                                      learning_rate=0.001
                                  ))

X_train, X_test, y_train, y_test = train_test_split(x_data, y_true, test_size=0.3)

input_function = tf.estimator.inputs.numpy_input_fn({'x': X_train},y_train,
                                                batch_size=20, num_epochs=10000,
                                                shuffle=True)

train_input_function = tf.estimator.inputs.numpy_input_fn({'x': X_train},y_train,
                                                      batch_size=8, num_epochs=10000,
                                                      shuffle=False)
test_input_function = tf.estimator.inputs.numpy_input_fn({'x': X_test},y_test,
                                                     batch_size=8, num_epochs=10000,
                                                     shuffle=False)

estimator.train(input_fn=input_function, steps=1000)

train_metrics = estimator.evaluate(input_fn=train_input_function, steps=1000)
test_metrics = estimator.evaluate(input_fn=test_input_function, steps=1000)

print('TRAINING DATA METRICS')
print(train_metrics)
print()

print('TEST DATA METRICS')
print(test_metrics)
print()
###
new_data = np.linspace(0, 1000, 10)
input_function_predict = tf.estimator.inputs.numpy_input_fn({'x':new_data},     shuffle=False)
print(list(estimator.predict(input_fn=input_function_predict)))

Gives the following output:

TRAINING DATA METRICS
{'average_loss': 200498430000.0, 'label/mean': 332774.78, 'loss': 1603987400000.0, 'prediction/mean': 0.97833574, 'global_step': 1000}

TEST DATA METRICS
{'average_loss': 197508330000.0, 'label/mean': 332257.22, 'loss': 1580066700000.0, 'prediction/mean': 0.97833574, 'global_step': 1000}

[{'predictions': array([0.9783435], dtype=float32)}, {'predictions': array([0.9783435], dtype=float32)}, {'predictions': array([0.9783435], dtype=float32)}, {'predictions': array([0.9783435], dtype=float32)},
{'predictions': array([0.9783435],....

So to conclude, the loss is huge, because TF predict same value of Y for all X. What is wrong in the code?


texts:
I am trying to do sentiment analysis. In order to convert the words to word vectors, I am using Word2Vec model. Suppose I have all the sentences in a list named 'sentences' and I am passing these sentences to word2vec as follows:
model = word2vec.Word2Vec(sentences, workers=4 , min_count=40, size=300,   window=5, sample=1e-3)

Since I am noob to word vectors, I have two doubts:
1- Setting the number of features to 300 defines the features of a word vector. But what these features signify? If each word in this model is represented by a 1x300 numpy array, then what do these 300 features signify for that word?
2- What does downsampling as represented by 'sample' parameter in the above model do in actual?


texts: How to make a Python code execute on GPU and not CPU

texts:
I am doing some image pre-processing using Python 3. As I know the code is executed on CPU for which the process is a little bit slow, its a good idea to make it run on GPU for the faster process because GPU can do graph operation much faster than CPU. Is there a way to make it run on GPU without using a library?
Please note that it is required to make it like a GPU version of Keras or TensorFlow.


texts: Algorithms and tools for ranking text as a job description

texts:
We're working on a ranking texts by degree of the similarity to vacancies. We have 4-year data set (≈1M texts) of custom search feeds from social network. We also have vacancies (≈30K) manually selected from this set.

Now the vacancies are selected in 2 stages:

We make several specific search queries (i.e. "need translator") on the social network by API, receiving and combining the search feeds into one.
Then we look through each post in it for real vacancies. Generally, it should contain a request for interpreter services. Often it contains some job details (such as a place, salary, subject, deadlines, languages, contacts) and stylistic features of speech (i.e. greetings, exclamations, request to write in private, ask for recommendations, etc.).

Before the point 2, we would like to have first in our final moderation feed the texts, that are most likely to be vacancies. We're going to try ranking by using the Naive Bayes classifier or the Natural Language Processing, but it's not as simple as it seems: there are a lot of various tools, schemes, models.

The question is, what algorithms and tools could help in our case? What we should pay attention to?


texts: What is feature channels mentioned in U-Net?

texts:
I was reading the U-Net paper for medical image segmentation. I had a doubt in the architecture. The authors mention that the max pooling layers in contraction path double the number feature channels while Downsampling.
Can anyone explain what are feature channels and how are they doubled while max pooling? In simple language please.


texts:
I used clustering on my dataset. Now when I'm trying to use a LASSO with cv to predict a response, one of the variables it takes into consideration is which cluster a new point is classified into.(I included the cluster variable as a predictor to see if being in a particular group affects the response)
Since the information on all variables is already captured by the cluster variable,using it again in the Lasso model with some other variables,does it become redundant/biased?


texts:
I have implemented an auto-encoder that takes sentence vectors as input and at decoder the last layer outputs sentence vectors. I would like to convert sentence vectors to sentences. Is there any way to convert sentence vectors to sentences?
class AutoEncoder(nn.Module):

def __init__(self, embedding_dim, hidden_dim):

    #Constructor
    super().__init__()

    self.fc=nn.Linear(embedding_dim,hidden_dim)
    self.act1=nn.ReLU(0.2)
    self.fc1=nn.Linear(hidden_dim,hidden_dim)
    self.act2=nn.ReLU(0.2)
    self.fc2=nn.Linear(hidden_dim,hidden_dim)
    self.act3=nn.ReLU(0.2)
    self.fc3=nn.Linear(hidden_dim,hidden_dim)
    self.act4=nn.ReLU(0.2)
    self.fc4=nn.Linear(hidden_dim,hidden_dim)
    self.act5=nn.ReLU(0.2)
    self.fc5=nn.Linear(hidden_dim,embedding_dim)
def forward(self, X):
    l1=self.fc(X)
    a1=self.act1(l1)
    l2=self.fc1(a1)
    a2=self.act2(l2)
    l3=self.fc2(a2)
    a3=self.act3(l3)
    l4=self.fc3(a3)
    a4=self.act4(l4)
    l5=self.fc4(a4)
    a5=self.act5(l5)
    l6=self.fc5(a5)
    return l6

Here I have considered word embeddings initially and converted them to sentence vectors by averaging. These vectors are given for auto encoder model.
Let me know whether we have any other approach where we can generate a sentence.
Any kind of reference is helpful.


texts: How to most effectively utilize historical data to train churn model

texts:
Suppose we have some historical data of users activity on a website and we want to build a churn prediction model (let's say we want to predict churn in a 2 month window). The usual approach, as I understand it, is to take a slice of that historical data at time $t$ and see which users churn in the time interval $(t, t + 2 \text{ month})$, so we take some features at time $t$ and train our model.

However, that way we only use a small part of our data, feeding our model only users who were active at time $t$. But what if we want to use all historical data? One way that comes to mind is take a lot of slices of our data at times $t_1, t_2, t_3, \dots$ and just merge them in one dataset, however different slices could have the a lot of same users even if we take these slices very far apart from each other. So our model could potentially learn that if a particular set of features occurs many times in our data set, then the user with these features is less likely to churn, (e.g. if we take two time slices at $t_1$ and $t_2>1$ then, if a user is presented in both of these slices, he can't churn at least at time $t_1$). So it doesn't seem to be the right way to do it...

How can I extract as much information as possible from a historical data over large period of time without spoiling the model?


texts:
I am using Keras LSTM (Tensorflow backend) to fit a time series model. Here is my model:

model = Sequential()
model.add(LSTM(LSTM_nums, input_shape=(timestep,data_dim)))
model.add(Dense(2))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(trainX, trainY, epochs=100, batch_size=1, verbose=2)

I am wondering is there a constraint or rule of thumb (or best practice) for setting the LSTM_nums value with a given timestep? I.e. should there be a relationship between LSTM_nums ~ timestep ? Thanks!


texts: How to access images directly from Google Cloud Storage (GCS) when using Keras?

texts:
I have developed a model in Keras that works perfectly when reading data stored locally.  However, I now want to take advantage of Google Cloud Platform's GPUs for training the model.  I have set up the GPU on GCP and am working in a Jupyter notebook.  I have moved my images to Google Cloud Storage.

My question is:

How can I access these images (specifically the directories -  training, validation, test) directly from Cloud Storage using the Keras' flow_from_directory method of the ImageDataGenerator class?

here's my directory structure in Google Cloud Storage (GCS):

mybucketname/
      class_1/
          img001.jpg
          img002.jpg
          ...
      class_2/
          img001.jpg
          img002.jpg
          ...
      class_3/
          img001.jpg
          img002.jpg
          ...


texts: Performing machine learning on small datasets

texts:
As a beginner at machine learning, I wanted to work on a small project in which the dataset has only 80 rows and 5 columns. The dataset I am working with is related to a medical condition, with 4 columns as biomarkers, and the 5th column indicates whether the row (a patient) has the condition or not.
 So far, I have fitted the following 5 models (with accuracy and MCC scores):

KNN (Accuracy: 43.5%, MCC:-0.164)
Logistic Regression (Accuracy: 65.2%, MCC: .312)
SVM (Accuracy: 60.9%, MCC: .214
Random Forest (Accuracy: 86.95%, MCC: .769)
Decision trees (Accuracy: 65.2%, MCC: .312)

I have used 5-fold cross validation to prevent overfitting, and yet most of my models are underperforming. I was also considering ensembling and bootstrapping, but with these lacking results, I am not sure how effective they would be. Do you have any tips concerning either:

Better algorithms for small datasets
Improvements I could make on the algorithms I have so far
Another method (e.g. Regularization)


texts: Some data sets values are missing. Missing values are spread along 1.5 standard deviation from the median with distribution mean = 0 & variance = 5. How much data would remain unaffected in %. Why?


texts:
I did a lot of Googling but could not find a paper that presents an algorithm which will produce dense feature vectors for short text input. I would be happy to find feature extraction algorithm which would perform at least as good as sparse word unigram and bigram feature vectors.

Currently I am exploring the idea of using LDA (Latent Dirichlet Allocation) but there are problems with processing short text (2-7 words per document).

The task at hand is short text classification. The number of classes for my data ranges from 10 to 20 classes. The classes are fairly well represented and the word unigram and bigram features work well. I would like to compute dense feature vectors for other experiments.

Any pointers to papers, preferrably simple to implement, would be appreciated.


texts: How to represent target variable for chess AI

texts:
Inspired by Google's recent AlphaGo project, I've decided that as a fun personal challenge I'd like to use deep learning and convoluted neural networks to build an algorithm that can beat an ordinary chess program when the difficulty is set to expert. I'm awful at chess and generally can't beat a computer beyond easy/medium, so making something that's a lot smarter than I am (at chess, at least) sounds fascinating.

Question:
What should the target variable look like for predicting the next best chess move?

I've been building predictive models for a long time, but I can't wrap my head around this question. Some things that have me confused:

I believe the target variable for Go is simply the entire grid of spaces on the board. In chess there are a limited number of legal moves, which continuously diminish as pieces are removed from the board throughout the game. How do I represent a shrinking target space?
Not all pieces have the same set of rules (ex: queen vs knight). Do I need to somehow explicitly code these rules or will deep learning automatically learn them?

Notes:

I know that alpha-beta pruning and quasi-brute-force search
techniques have been shown to do well in chess, however, I
specifically want to use deep learning instead.
AlphaGo started with supervised learning (SL) on documented expert human moves to make a policy network, among other things. In my case, I'm just trying to do the supervised learning part, and when I do that, the target variable is difficult for me to define. That's the root of my question.


texts: Effect of adding extra unrelated features to linear perceptron

texts:
Suppose that we are training a linear regressor (perceptron). Adding extra features that are not related to the target (e.g. randomly generated values) before training will typically ____ our training error.

Would it typically increase, decrease or stay the same?

Listed Answer: decrease

The answer to this question is listed as decrease, but wouldn't adding extra unrelated features not change the training error, because the model would just learn to ignore those features (add 0 weights to them). Even if the model didn't learn to ignore the extra features, if anything wouldn't the training error increase as it doesn't seem like the model would be able to learn a relationship that would benefit the training error from a completely random unrelated feature?


texts: Using vector of floating points as labels for input data in a CVAE

texts:
The Conditional Variational Autoencoders (CVAE) (and other classification networks) I have come across use a one hot vector encoding for labeling categorial data sets.

In my case, I do not have a categorial data set. I have 2 vectors states and costs which both contain floating point values. states has dim 4 and costs has dim 3. I want to train my CVAE in such a way that at test time I feed the decoder a states vector and it should output an approximate costs vector.

In other words my decoder should return: p(costs|states,z)

How should I train and organise my CVAE in this case? Should I simply concat both vectors?


texts: What is the difference between concept learning and classification?

texts:
I've recently read about concept learning in a machine learning class. They defined a concept as (translated from German):

  Concept:

  Describes a subset of objects or events defined on a bigger set.
  Boolean function defined on bigger set

  Given: Examples which are labeled as members or not-members

  Searched: Automatically infer the definition of the underlying concept

  Definition of concept learning: Infering a boolean-valued function from training data of its input and its output.

I would call this a 2-class classification problem. Is there a difference?


texts: MLP Parameter tuning - gridsearchCV cannot fit?

texts:
I'm making an MLP classifier for binomial classification from 145 features.

I want to get the best parameters on my MLP classifier to get a better prediction so I followed the answer to this question, which is to use gridsearchCV from sklearn. However, when I get to

clf.fit(DEAP_x_train, DEAP_y_train)

I get the ff error:

TypeError: '<=' not supported between instances of 'str' and 'int'

I checked my train data and both X and y have already been changed to become float data type, so I don't understand why I keep getting this response.

For reference, I have 145 features to predict a binary response, and this is my parameter_space:

parameter_space = {
    'hidden_layer_sizes': [(368,), (555,), (100,)],
    'activation': ['identity', 'logistic', 'relu'],
    'solver': ['sgd', 'adam'],
    'alpha': [0.0001, 0.05],
    'learning_rate': ['constant','adaptive'],
    'max_iter': ['200', '1000', '5000', '10000']
}

I've tried to search for other cases online that this error response is shown when trying to fit, but I haven't come across any.

Would appreciate any response. Thanks!


texts: Convert time series data to supervised learning problem

texts:
I have a similar dataset like the one below. Each row represents a person and there are 3 different variables m1,m2,m3 with 3 measurements each.
I am trying to frame this time series problem as a supervised learning problem so i can train an SVM and/or random forest.

id
Age
gender
m1_1
m2_1
m3_1
m1_2
m2_2
m3_2
m1_3
m2_3
m3_3
label

1
20
M
12.4
34
12
13
324
34
34
232
12
0

2
30
M
123.4
324
2
32
32
4
3
2
2
1

My idea would be to reshape the data like this: Note that I only did for the first row on the dataset above

id
Age
gender
m1
m2
m3
Label

1
20
M
12.4
34
12
0

1
20
M
13
324
34
0

1
20
M
34
232
12
0

Basically, if there are 3 measurements per variable, I'd have 3 rows to reflect this information.
Am I going in the right direction with this approach? If not, could you point me to the right direction?
Thank you for taking your time in advance.


texts: Which learning tasks do brains use to train themselves to see?

texts:
In computer vision is very common to use supervised tasks, where datasets have to be manually annotated by humans. Some examples are object classification (class labels), detection (bounding boxes) and segmentation (pixel-level masks). These datasets are essentially pairs of inputs-outputs which are used to train Convolutional Neural Networks to learn the mapping from inputs to outputs, via gradient descent optimization. But animals don't need anybody to show them bounding boxes or masks on top of things in order for them to learn to detect objects and make sense of the visual world around them. This leads me to think that brains must be performing some sort of self-supervision to train themselves to see.

What does current research say about the learning paradigm used by brains to achieve such an outstanding level of visual competence? Which tasks do brains use to train themselves to be so good at processing visual information and making sense of the visual world around them? Or said in other words: how does the brain manage to train its neural networks without having access to manually annotated datasets like ImageNet, COCO, etc. (i.e. how does the brain manage to generate its own training examples)? Finally, can we apply these insights in computer vision?


texts: Does modeling with Random Forests require cross-validation?

texts:
As far as I've seen, opinions tend to differ about this. Best practice would certainly dictate using cross-validation (especially if comparing RFs with other algorithms on the same dataset). On the other hand, the original source states that the fact OOB error is calculated during model training is enough of an indicator of test set performance. Even Trevor Hastie, in a relatively recent talks says that "Random Forests provide free cross-validation". Intuitively, this makes sense to me, if training and trying to improve a RF-based model on one dataset.

Can someone please lay out the arguments for and against the need for cross-validation with random forests?


texts: statsmodels.tsa.holtwinters.ExponentialSmoothing: what do “additive”/“multiplicative” trend and seasonality actually mean?

texts:
There are additional concepts of additivity and multiplicativity for

trend (trend{“add”, “mul”, “additive”, “multiplicative”, None}) and
seasonality (seasonal{“add”, “mul”, “additive”, “multiplicative”, None})

in the Statsmodels implementation [1, 2] of the Triple Exponential Smoothing (Holt-Winter's Method).
Actually, this means different methods of calculating the initializing values of seasonality and trend correspondingly (according to source code):
m = self.seasonal_periods
l0 = initial_level
b0 = initial_trend
if has_seasonal:
    l0 = y[np.arange(self.nobs) % m == 0].mean() if l0 is None else l0
    if b0 is None and has_trend:
        # TODO: Fix for short m
        lead, lag = y[m : m + m], y[:m]
        if trend == "mul":
            b0 = np.exp((np.log(lead.mean()) - np.log(lag.mean())) / m)
        else:
            b0 = ((lead - lag) / m).mean()
    s0 = list(y[:m] / l0) if seasonal == "mul" else list(y[:m] - l0)
elif has_trend:
    l0 = y[0] if l0 is None else l0
    if b0 is None:
        b0 = y[1] / y[0] if trend == "mul" else y[1] - y[0]
    s0 = []
else:
    if l0 is None:
        l0 = y[0]
    b0 = None
    s0 = []

In other words, when there is seasonality

the initial additive trend will be:

$$ b_0 = \frac{1}{N} \sum^{N}_{i=0} \frac{y_{i+m} - y_i}{m}$$

the initial multiplicative trend will be:

$$ b_0 = \frac{ \ln \left( {\frac{1}{m}\sum^{m}_{i=0}y_{i+m}} \right) - \ln \left({\frac{1}{m}\sum^{m}_{i=0}y_{i}} \right)}{m} $$
where $m$ is the length of the one period, and $\mathbf{y}$ is the input vector (time series).
When there is no seasonality

the initial additive trend will be:

$$ b_0 = y_1 - y_0 $$

the initial multiplicative level will be:

$$ \ln(b_0) = \ln(y_1) - \ln(y_0) $$
The zero value of the seasonality (zero period) for its additive or multiplicative form is defined as the difference or ratio between the first m samples and the zero value of the level, respectively.
AND this is NEITHER a classical additive/multiplicative decomposition or additive/multiplicative Exponential smoothing as I understand. Moreover, trend and seasonality can be additive or multiplicative independently of each other in Statsmodels.
So, the questions:

Does somebody have got any reference about the right selection of the values of the parameters?
Can anybody explain for what cases these parameters should be used?

I'll be appreciated your replies!
References:

Statsmodels.Tsa.Holtwinters.Exponentialsmoothing — Statsmodels.
GitHub. 2020. Statsmodels/Statsmodels.


texts: How to find correlation between categorical data and continuous data

texts:
I'm working on imputing null values in the Titanic dataset.
The 'Embarked' column has some. I do NOT want to just set them all to the most common value, 'S'.
I want to impute 'Embarked' based on its correlation with the other columns.
I have tried applying this formula to the 'Embarked' column:
def embark(e):
        if e == 'S': return 1
        if e == 'Q': return 2
        if e == 'C': return 3
        else: return 4

This allows me to check out data.corr(), but I think it's trickier than that since I'll get a different correlation with different value assignments (right??). I also thought about using a four-dimensional (for S,Q,C,NaN) one-hot vector, but I doubt that would work.
Is there a skLearn method that does this some way? Any further insights on the matter?


texts:
Let's say I'm building a medical assistance chatbot.

How do I validate that my model is working well or better than others.

Is there any standard validation metrics for it ?


texts: Using sigmoid in binary DNN output layer instead of softmax?

texts:
For a binary DNN, the output is $y_0 + y_1 = 1$ since they are the probability distribution, hence the sum must equate to 1. However, I've been told that $y_1$ is sufficient to represent the output of the DNN where:

$y_1 = \frac{e^{h_1}}{{e^{h_0} + e^{h_1}}}$

Hence, $h = h_1 - h_0$, why is it that $y$ which is the final output can now be represented as $y = \sigma(h)$, where $\sigma$ is the sigmoid function. Could anyone please explain why? How did it derive to the point where $y = \sigma(h)$?


texts: Memory issue when trying to initiate zero tensor with pytorch

texts:
I am facing a memory issue while trying to initialize a torch.zeros -
torch.zeros((2000,2000,3200), device=device)
Getting the following error:
RuntimeError: CUDA out of memory. Tried to allocate 47.69 GiB (GPU 0; 8.00 GiB total capacity; 1.50 KiB already allocated; 6.16 GiB free; 2.00 MiB reserved in total by PyTorch)

My question is: why does a zero tensor need that big memory? Or am I doing any mistake?
P.S. I was checking with getsizeof in another system - the size of this tensor is showing as 72bytes only.


texts: Should I resize my test images when feeding to a CNN model?

texts:
I am new to deep learning, I have made my first CNN network which accepts an input of (32,32,3) while training.
Now I want to test some random images on my trained model.
Do I have to resize my test images to (32,32,3). OR Can I leave the images as it is. OR Can I resize all of them to a higher resolution?
Plz give reasons.


texts:
I have used the KMeans algorithm to create an engine that can guess the cluster that a particular set of input data will fall into.

Can I use it to guess the 2nd closest cluster, 3rd closest, and so on?

Currently, I am using the sklearn.cluster.KMeans library if that's any help - it doesn't seem like the API provides this functionality already.


texts: MATLAB @ symbol for fminunc / fmincg function

texts:
I am following Andrew Ng's Coursera class on machine learning, and I came across this syntax for the fminunc and fmincg functions:

fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)),initial_theta, options);

Specifically, it is the @ that's confusing me. I read up on it and I still don't understand what we're doing with it here. Can someone clarify what this is doing, as well as how the t in the function call to lrCostFunction is related to the @(t) in the beginning.


texts: Change priority of cell execution in jupyter notebook?

texts:
I am sometimes in the following situation:

I want to execute two cells: Cell 1 takes on the order of 10 minutes or hours. Cell 2 will take 1 second.
I'd rather execute cell 2 first and see the result, but I didn't know that beforehand and hence already started cell 1.
cell 1 is already significantly into the computation, so it would be wasteful to just abort it.

How do I first pause cell 1, then start cell 2, and then start cell 1 again? Or even better, how do I pause cell 1, start cell 2, and have cell 1 automatically start again when cell 2 is done?


texts: CycleGAN vs. AutoEncoder for transforming sketches into images

texts:
I'm playing around with the use of deep learning on images and done quite works : colorizing black and white images for example, or maybe fixing old damaged photos.
Today I want to tackle a new problem, concerning the conversion of sketches into real look like images as shown in the figure.

Performing this could be done in various ways. I want to know if it is correct to think of CycleGANs for this task, since in truth they are not made trivially for that, and the pros and cons for CycleGANs and AutoEncoders for such task.
PS: Generating data is not a big deal for me, I have already a large database, and the right scripts to do the conversions in the both directions.


texts: Can colors be detected using Neural Nets?

texts:
How do I represent a color as an activation value within a neuron? ( might be off-topic)

I want to detect colors Using Neural Nets

In my Knowledge any activation function which is generally used will push the values in between 0-1 which isn't of any use to me...

Just wondering how Windows 10 decides on the theme color change when we switch themes.. (led me to think about this or its just simple averaging on pixel values of the channels?)

So seems like an impossible task for Neural Nets then?


texts:
I searched for the term and it appears in a few articles but it is used without explanation. The only explanation I could find is in a PhD thesis: "Regret bounds are the common thread in the analysis of online learning algorithms. A regret bound measures the performance of an online algorithm relative to the performance of a competing prediction mechanism, called a competing hypothesis."

I am still confused after reading this (I did not read the rest of the thesis as it is way above my understanding in that field). Could someone please explain? Many thanks in advance!


texts:
I have images with and without watermark. There is only one type of watermark. I have tried VGG16 transfer learning, but results were bad.

What are the methods to do image classification if it has a watermark or not?


texts: When to choose character instead of factor in R?

texts:
I am currently working on a dataset which contains a name attribute, which stands for a person's first name. After reading the csv file with read.csv, the variable is a factor by default (stringsAsFactors=TRUE) with ~10k levels. Since name does not reflect any group membership, I am uncertain to leave it as factor.

Is it necessary to convert name to character? Are there some advantages in doing (or not doing) this? Does it even matter?