texts:
I've done perceptron with one output already, but now i want to try doing it with at least two outputs. I can't find any example in google. Is it possible? Can you give me a minimal example? I want to create neural network which is predicting binary numbers. I don't know how to add more outputs. It will be the best option if i can create 8 output neurons.
Thats what i've written:
def sigmoid(x):
    return 1.0 / (1 + exp(-x))

def sigmoid_deriv(x):
    return sigmoid(x) * (1 - sigmoid(x))

training_data = [
    (array([0,0,1]), [0, 0]),
    (array([1,0,1]), [1, 0]),
    (array([0,1,1]), [0, 1]),
    (array([1,1,1]), [1, 1]),
]

def binary_to_int(binary_number_array: list):
    #print(binary)
    decimal = 0
    i = 0
    #print(binary_number_array)
    for number in binary_number_array:
        if number == 1:
            decimal += pow(2, i)
        i += 1
    #print(decimal)
    return decimal

errors = []
eta = 0.6
input_layer = 3
hidden_layer = 2
output_layer = 2

epoch_count = 500

weights_input_to_hidden = np.random.uniform(-1, 1, (input_layer, hidden_layer))
weights_hidden_to_output = np.random.uniform(-1, 1, hidden_layer)

preActivation_H = np.zeros(hidden_layer)
postActivation_H = np.zeros(hidden_layer)

training_count = len(training_data[:])
#print(training_count)

for epoch in range(epoch_count):
    for sample in range(training_count):
        data, expected = training_data[sample]
        for node in range(hidden_layer):
            #data, expected = training_data[sample]
            #print(data, weights_input_to_hidden[:, node])
            preActivation_H[node] = np.dot(data, weights_input_to_hidden[:, node])
            postActivation_H[node] = sigmoid(preActivation_H[node])
            #print(data, weights_input_to_hidden[:, node])
            #print(preActivation_H[node], postActivation_H[node])

        preActivation_O = np.dot(postActivation_H, weights_hidden_to_output)
        postActivation_O = sigmoid(preActivation_O)

        decimal_expected = binary_to_int(expected)
        error = postActivation_O - decimal_expected
        errors.append(error)
        #print(error)

        # back propagation
        for hidden_node in range(hidden_layer):
            S_error = error * sigmoid_deriv(preActivation_O)
            #print(S_error)
            gradient_hidden_to_output = S_error * postActivation_H[hidden_node]
            #print(gradient_hidden_to_output)

            for input_node in range(input_layer):
                # tu moze byc zle
                input_value = data[input_node]

                print(input_value)
                gradient_input_to_hidden = S_error * weights_hidden_to_output[hidden_node] * \
                                           sigmoid_deriv(preActivation_H[hidden_node]) * input_value

                weights_input_to_hidden[input_node, hidden_node] -= eta * gradient_input_to_hidden

            weights_hidden_to_output[hidden_node] -= eta * gradient_hidden_to_output

validation_data = []
for i in range(training_count):
    data, expected = training_data[i]
    for node in range(hidden_layer):
        preActivation_H[node] = np.dot(data, weights_input_to_hidden[:, node])
        postActivation_H[node] = sigmoid(preActivation_H[node])

    preActivation_O = np.dot(postActivation_H, weights_hidden_to_output)
    postActivation_O = sigmoid(preActivation_O)

    #print(postActivation_O)
    if postActivation_O > 0.5:
        output = 1
    else:
        output = 0

    validation_data.append((postActivation_O, expected, output))
```


texts:
Below are few questions where I unable to find out where I am wrong. I added screen shot of image and explanations of the each options that I am understanding. Questions are purely discussion based and short. Please help me out.

In below question, I checked

A. We need to predict the author gender and it can be either male or female. I think  it is  classification problem, so it is supervised learning.

B.  We have given group of spam emails and need to predict does there sub-types are spam or not. I think  it is  classification problem, so it is supervised learning.

C. We need to predict data based on height and age. It is a linear  regression problem because we create graph height vs age will find out the test case. It is supervised learning.

D. Grouping data is a cluster problem, so it unsupervised learning.

In below question, I had checked C and D options because feature scaling creates our dataset in same range which helps to predict the best theta in less iterations and contour graph will be more cleared and symmetric. Ref : https://medium.com/greyatom/why-how-and-when-to-scale-your-features-4b30ab09db5e


texts: SVD Kernel and Linear Algebra Kernel, is there a conceptual difference?

texts: Is the term kernel used in Sklearn to execute the  SVD machine learning algorithm conceptually related to the notion of a kernel in linear algebra ( null space )? Or do they happen to use this same name?


texts: Abnormal value of ROC AUC score in Binary Classifier Model

texts:
I have developed the following model for a Binary Classifier. I have to evluate using roc_auc_score. I am getting unusual values for roc_auc_score.
When I use
epoch = 1, roc_auc_score = 0.8
epoch = 2,  roc_auc_score =0.53
And after this for any epoch it is constant at 0.5. Why is this happening? Is there something wrong with my model?
DATA PREPROCESSING
#Load Dataset
test = pd.read_csv('test.csv')
train = pd.read_csv('train.csv')

#Combine Train and Test set for Data Cleaning
train['set'] = 'train'
test['set'] = 'test'
df = pd.concat([test, train])

#One Hot Encoding
df = pd.get_dummies(df, columns=['Gender','Driving_License','Previously_Insured','Vehicle_Age','Vehicle_Damage'])

#Moving Target Column to End
target = df['Response']
df.drop(labels=['Response'], axis=1, inplace = True)
df.insert(16, 'Response', target)

#Separating Train and Test Data
train = df[df['set']=='train']
test = df[df['set']=='test']
train = train.drop('set', 1)
test = test.drop('set', 1)

#Creating Input Features and Target Variables
X= train.iloc[:,1:15]
y= train.iloc[:,[15]]

#Standardizing the Input Features
scaler = StandardScaler()
X = scaler.fit_transform(X)

#Train Test Split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

CODE
#Model
model = Sequential()

model.add(Dense(14, activation='relu', kernel_initializer='random_normal', input_dim=14))

#Output Layer
model.add(Dense(1, activation = 'sigmoid', kernel_initializer='random_normal'))

#Compiling the neural network
model.compile(optimizer ='adam',loss='binary_crossentropy', metrics =['accuracy'])

#Fitting the data to the training dataset
model.fit(X_train,y_train, batch_size=32, epochs=1, verbose=0)

#Make predictions and convert to binary value
pred_train = model.predict(X_train)
pred_test = model.predict(X_test)

#ROC AUC Score
print('Train AUC = {:.5f}'.format(roc_auc_score(y_train,pred_train)))
print('Test AUC = {:.5f}'.format(roc_auc_score(y_test,pred_test)))

#Accuracy
print('Train Accuracy = {:.3f}'.format(accuracy_score(y_train,pred_train.round())))
print('Test Accuracy = {:.3f}'.format(accuracy_score(y_test,pred_test.round())))


texts: Demand forecasting with marketing budget data

texts: I'm trying to build a demand forecasting model to predict future daily orders of an online food takeout service (similar to UberEats or DoorDash). My first model uses a univariate approach, which is basically an ensemble of statistical models such as Auto ARIMA, ETS, BSTS, etc. Now, I want to build upon this by adding features related to marketing budget/spend because that's always been a big push for sales, however, the data that I have is only on a monthly level, which makes sense because things like promotional campaigns can only be planned at a high level. So my question is, how can I integrate this data into my forecast model? Any other tips on improving the model accuracy (e.g. what features can be added) are also appreciated. Thanks!


texts: MSE vs Cross Entropy for training with facial landmark (pose) heatmaps

texts:
I am trying to reimplement the excellent paper https://github.com/1adrianb/face-alignment-training in tensorflow.  I have successfully defined the network and downloaded the LSD3D-W dataset.  I am able to train the model however I am running into a serious issue.

Ground Truth and Loss

For training, I generate ground truths by converting the x,y landmark coordinates into gaussian heatmaps where x,y is the mean of the gaussian.
I first trained with MSE loss as given in the original implementation.  After some iterations, the loss becomes extremely small but the output is completely white!

loss = tf.losses.mean_squared_error(
    predictions=heatmaps, labels=labels_tensor
)

When I tried with cross entropy, I am getting better results.  But they are not sharper

loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=heatmaps, labels=labels_tensor), name= 'cross_entropy_loss')

gtmap means groundtruth map.


texts: Triplet loss function for face recognition?

texts:
In the Andrew-NG coursera course on Convnets he talked about triplet loss function for one shot face recognition.

The formula given in the video is,
$$\to \small \small \small ||f(A)-f(P)||^2 \;+\;\alpha \leq\;||f(A)-f(N)||^2$$
$$\to \small \small \small D(A, P) + \alpha \leq D(A, N)$$
$$L(A,P,N) = max(||f(A)-f(P)||^2 - ||f(A)-f(N)||^2 + \alpha,  0)$$
Here,
$$f(A) - \small \text{ Person }A$$
$$f(P) - \small \text{Different Picture of Person }A$$
$$F(N) - \small \text{Another Person}$$

I couldn't understand why did we use $\alpha$ in the formula. I understood that the ideal loss function is to decrease $\small \small D(A, P)$ and increase $\small \small D(A, N)$ but if we add $\alpha$ to $\small \small D(A,P)$ it will increase it which is not we require right?


texts: What happens when the vocab size of an embedded layer is larger than the text corpus used in training?

texts:
Full disclosure this question is based on following this tutorial: https://tinyurl.com/vmyj8rf8
I am trying to fully understand embedded layers in Keras. Imagine having a network to try and understand basic sentiment analysis as a binary classifier (1 positive sentiment and 0 negative sentiment). The toy dataset for this is as follows:
# Define 10 restaurant reviews
reviews =[
          'Never coming back!',
          'horrible service',
          'rude waitress',
          'cold food',
          'horrible food!',
          'awesome',
          'awesome services!',
          'rocks',
          'poor work',
          'couldn\'t have done better'
]#Define labels
labels = array([1,1,1,1,1,0,0,0,0,0])

This data can be used to train a really simple network as follows:
Vocab_size = 50
model = Sequential()
embedding_layer = Embedding(input_dim=Vocab_size,output_dim=8,input_length=max_length)
model.add(embedding_layer)
model.add(Flatten())
model.add(Dense(1,activation='sigmoid'))
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['acc'])
print(model.summary())

In order to feed this data into he network, we can one hot encode it using Keras one_hot as follows:
encoded_reviews = [one_hot(d,Vocab_size) for d in reviews]
print(f'encoded reviews: {encoded_reviews}')

We get the following output:
encoded reviews: [[14, 45, 43], [8, 2], [6, 43], [24, 1], [8, 1], [11], [11, 21], [16], [34, 40], [2, 25, 36, 15]]

I understand that the purpose of setting Vocab_size = 50, even though there are only around 20 unique words in the corpus is to give a large enough hashing space for the hashing algorithm behind one_hot to avoid collisions when the text is encoded.
If I train the model on these words (assume fixed length input and padding) and then get the weights of the embedded layer:
print(embedding_layer.get_weights()[0].shape)

(50, 8)

We can see this it is an array of 50 vectors that look like this as an example:
[ 0.17051394  0.13659576 -0.05245572 -0.12567708  0.06743167  0.05893507
-0.14506021  0.06448647]
My understanding is that each of these vectors corresponds to a word embedding for each word in the corpus. But if there are only 20 unique words in the corpus and Vocab_size is set larger than this then that can't be completely true? If Vocab_size > corpus_vocab_size, then what do these embeddings represent? Any help would be appreciated.


texts: what is the difference between positional vector and attention vector used in transformer model?

texts:
what is the difference between positional vector and attention vector used in transformer model ? , i saw a video in youtue and the defintion for positional vector was give as :* "vector that gives context based on postion of word in sentence "*
defintion for attention vector was give as "For ever word we can have attention vector generated which captures contextual relationship between words in sentence"
Capturing context information based on distance(postional vector) and attention (attention vector ) sounds same right? or is it different ?


texts: Is it a good idea to normalize the outputs of a Neural Network for Regression, when the different outputs vary in magnitude?

texts:
I understood that it is not necessary to scale the output of a neural Network when I predict a single value via regression.

Is it necessary do normalize the Outputs of my neural Network if I have multiple outputs that vary in magnitudes between 10^-2 and 10^4?
My intuition would tell me that the loss function might ignore the smaller values and only focus on the values of a bigger magnitude.


texts: What does the term 'Facet' in Seaborn FacetGrid imply?

texts:
I am a newbie to data science. I have a very basic understanding of Seaborn. My understanding is that it is used to plot grids. That said, I intend to have a better understanding of 'FacetGrid' and the term 'facet'.

  FacetGrid is a multi-plot grid for plotting conditional relationships.
  FacetGrid object takes a DataFrame as input and the names of the
  variables that will form the row, column, or hue dimensions of the
  grid. The variables should be categorical and the data at each level
  of the variable will be used for a facet along that axis.

The confusion is in the line 'the data at each level of the variable will be used for a facet along that axis'. Here, what does it mean by facet?

I am adding a code example where FacetGrid is used(Here FacetGrid is used to plot a 2D scatter plot):

sns.set_style('whitegrid')
sns.FacetGrid(haberman, hue ='Survival Status', size = 8) \
.map(plt.scatter, 'Age of Patient', 'Positive Axillary Nodes') \
.add_legend()
plt.show()

Can someone give a better explanation for a better understanding?


texts:
I've been thinking about the Recurrent Neural Networks (RNN) and their varieties and Convolutional Neural Networks (CNN) and their varieties.

Would these two points be fair to say:

Use CNNs to break a component (such as an image) into subcomponents (such as an object in an image, such as the outline of the object in the image, etc.)
Use RNNs to create combinations of subcomponents (image captioning, text generation, language translation, etc.)

I would appreciate if anyone wants to point out any inaccuracies in these statements.  My goal here is to get a more clearer foundation on the uses of CNNs and RNNs.


texts:
I used to tried making a fake correlation in my mocking dataset and found that if the score is more than 0.5 I can reduce feature to avoid singularity

In the given example they are many correlated variables. Therefore I am thinking about dimensionality reduction. The one feature here that I plan to remove is Hum-6D05 because it has strong correlation between Hum-6D12, Hum-6F04, and Hum-6F14

Question:
Am I correct?
Any comment or advice are welcomed


texts: Classification methods using one overlapping feature

texts:
I'm currently working on an assignment related to house prices in the UK. I have a list of prices that depend on the size of the property (small, medium, large) and the property location in terms of how close the property is to urban areas (Yes = close to Urban areas | No = far away).

The data that I have is from two extremes, so all the properties are either close to urban areas or far away. As you can assume, the prices are high for large properties that are close to urban areas. In contrast, the prices are low for small properties that are far from urban areas.

However, the data is quite overlapping and a small house in urban area might have the same price (or close) to a large property outside the urban zone.

Now, my task is to determine whether the property is located in urban areas just by using house prices (the only available feature). But to do so I need to determine the size of the property as this has a significant affect on the price as well.

In reality, I would like to test my algorithm(s) on a range of property prices even those that are somewhere in between that is, not far to urban areas but no so close either. Can someone share their ideas on how one would tackle such a problem using only the house prices as my features? Thanks in advance for your input!


texts: Why/When should I use VGG16 to do fine-tuning?

texts:
Why or When should I use VGG16 in my cnn? what is the pros and cons to use this model?

I search but not found this answer. If you have references, I appreciate


texts: use the same gradient to maximize one part of the model and minimize another part of the same model

texts:
I want to calculate the gradient and use the same gradient to minimize one part and maximize another part of the same network (kind of adversarial case). For me, Ideal case would be, if there are two optimizers responsible for two part of the network/model and one of the optimizers has a negative learning rate. But it seems that PyTorch does not allow negative learning rate.
In this case what I am doing is:
loss.backward()
optimzer_for_one_part of the model.step()

and then
(-loss).backward()

Problem is, This time the again calculated gradient will not be the same(values are different but flopped of course) because some weights of the same network (same computation graph) have already been changed. But, Ideally, I want to use the flipped version of the previous gradient.
How can I achieve this?


texts: Trained Tensorflow model performs poorly on inference

texts:
I trained an image classification model using Keras with Tensorflow backend. The model got good accuracy on validation dataset as well as on the testing data, I save the entire model to .h5 format, here is my checkpoint callback.

checkpoint = ModelCheckpoint(model_name+".h5", monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=False, mode='auto', period=1).

As I was hoping to use this model on Android so I refreeze the model to binary protobuf (.pb) using keras_to_tensorflow.

When performing inference using the model on mobile I noticed the model gives very wrong and random predictions. I have tried exploring other reasons why this could be the case like I found here still, it seems clear the issue is not with loading the images.

Also, performing inference with the converted model on Tensorflow Python still gives the same wrong/random predictions. Here is my code for performing inference in Python.

def model_predict( model_path, image_path, model_input, model_output, class_names ):

    with tf.Graph().as_default() as graph: # Set default graph as graph

        with tf.Session() as sess:
            # Load the graph in graph_def
            print("load graph")

            # We load the protobuf file from the disk and parse it to retrive the unserialized graph_drf
            with gfile.FastGFile(model_path,'rb') as f:

                print("Load Image...")
                # Read the image & get statstics
                np_image = Image.open(image_path)
                np_image = np.array(np_image).astype('float32')/255
                np_image = np.resize(np_image, (224, 224, 3))
                np_image = np.expand_dims(np_image, axis=0)

                # Set FCN graph to the default graph
                graph_def = tf.GraphDef()
                graph_def.ParseFromString(f.read())
                sess.graph.as_default()

                # Import a graph_def into the current default Graph (In this case, the weights are (typically) embedded in the graph)

                tf.import_graph_def(
                graph_def,
                input_map=None,
                return_elements=None,
                name="",
                op_dict=None,
                producer_op_list=None
                )

                # INFERENCE Here
                m_input = graph.get_tensor_by_name(model_input) # Input Tensor
                m_output = graph.get_tensor_by_name(model_output) # Output Tensor

                print ("Shape of input : ", tf.shape(m_input))
                #initialize_all_variables
                tf.global_variables_initializer()

                # Run model on single image
                Session_out = sess.run( m_output, feed_dict = {m_input : np_image} )

                print("Predicted class:", class_names[Session_out[0].argmax()] )

How do I perform inference using Tensorflow Python/Android with a save .pb model?

Others have suggested I save the session used for training and load them to Tensorflow when performing inference. If this is the case how do I load the saved session in Tensorflow android?

I am sure the model did not overfit, it performs very well when using Keras.


texts: Difference between NCE-Loss and InfoNCE-Loss

texts:
I started looking into word2vec and was wondering what the connection/difference between the NCE-Loss and the infoNCE-Loss is. I get the basic idea of both.
I have a hard time deriving one from another, do you have any idea ?
Thank you in advance!


texts: Why is my model accuracy decreasing after the second epoch?


texts: Vapnik Chervonenkis dimension of a classifier from the Wikipedia page

texts:
The Vapnik Chervonenkis dimension is defined by the wikipedia page here for a classification model as:
A classification model $f$ with some parameter vector $\theta$ is said to shatter a set of data points $(x_1, \ldots x_n)$ if, for all assignments of labels to those points, there exists some $\theta$ such that the model $f$ makes no errors when evaluating that set of data points.
I am trying to understand the second example. Suppose I have a model $f$ with only one parameter $\theta$, and two points $\{a,b\}$ where $a,b \in \mathbb{R}$ and $a<b$. Then there are four possible labellings:
$\{a = 1, b=1\}, \{a = 0, b=1\}, \{a = 1, b=0\}, \{a = 0, b=0\}$
Is the author saying that I have to choose one value of $\theta$ that would correctly classify the $a,b$ for each of these scenarios? Or rather that I can choose a different $\theta$ for each possible labelling.
For instance in the case $ t < a < b$ where $a,b$ are both labelled $1$, then by simply classifying points above $t$ as positive, I would correctly classify both examples. However this scheme would fail for the case where $ a = 1, b=0$.
Any insights appreciated.


texts: Long range forecasting with sequence-to-sequence models

texts:
I have a task where I want to forecast daily observations for 1 year or 2 years in advance at multiple locations--so 365 or 730 days in advance. I actually have a pretty good dataset, meaning daily observations going back 40 years at those sites. So I have a lot of data to train with. I am working with climate and geologic data, so the time series is strongly cyclical or periodic with some random variation.
I was trying to use a Sequence-to-Sequence LSTM network with attention to start with. I have 365 days input and trying to predict 365 days output. I am just starting to collect the results so far. But I was wondering if there are any specific models designed for long-range time-series forecasting? I thought sequence to sequence models still only worked well for say 20-30 day output, but I was not sure if there has been any developments on that front.
Thanks.


texts: Calculation of Output in LSTM Many-to-One Architecture

texts:
I'm new to Recurrent Neural Network but I want to train my data with LSTM but I'm having a trouble to understand LSTM Many-to-One architecture.
Suppose the size of my data is time_step x num_features say 2 x 2 and I have to use many-to-one LSTM architecture because I want to do classification. So in the last time_step I have to add dense (a) which contain sigmoid activation function to predict the sequence class which is 0 or 1.
My questions are,

When I compute a, do I need to include all the hidden state (h1 and h2) or just the last hidden state h2?
If I just include the h2, how do I calculate the derivatives of the loss function (cross entropy) w.r.t h1?

The derivation for h1 is highly recommended. Thank you :)


texts:
I recently put together an entry for the House Prices Kaggle competition for beginners. I decided to try my hand at understanding and using XGBoost.

I split Kaggle's 'training' data into 'training' and 'testing'. Then I fit and tuned my model on the new training data using KFold CV and got a score with scikit's cross_val_score using a KFold with shuffle.

the average score on the training set with this cross validation was  0.0168 (mean squared log error).

Next, with the fully tuned model, I check its performance on the never before seen 'test' set (not the final test set for the Kaggle leader board). The score is identical after rounding.

So, I pat myself on the back because I've avoided over-fitting... or so I thought. When I made my submission to the competition, my score became 0.1359, which is a massive drop in performance. It amounts to being a solid 25 grand wrong on my house price predictons.

What could be causing this, if not overfitting?

Here is the link to my notebook, if it helps: https://www.kaggle.com/wesleyneill/house-prices-walk-through-with-xgboost


texts: How do I determine if a neural network can be computed by a network that contains only an input layer and an output layer?

texts:
I have a homework problem where the neural network below is given with its description.  We have been asked "Can the functions that the above network computes also be computed by a network that contains only an input layer and an output layer that has a single node?" If we can, we're to provide the network, weights, and activation function. And if not, we have to explain why.

I am very unsure how to approach this problem as I am not that strong with neural networks. Any starting point would be very helpful.

Thanks in advance.


texts: Where can I find free spatio-temporal dataset for download?

texts: Where can I find free spatio-temporal dataset for download so that I can play with it in R ?


texts: Rate of convergence - comparison of supervised ML methods

texts:
I am working on a project with sparse labelled datasets, and am looking for references regarding the rate of convergence of different supervised ML techniques with respect to dataset size.
I know that in general boosting algorithms, and other models that can be found in Scikit-learn like SVM's, converge faster than neural networks. However, I cannot find any academic papers that explore, empirically or theoretically, the difference in how much data different methods need before they reach n% accuracy. I only know this from experience and various blog posts.
For this question I am ignoring semi-supervised and weakly supervised methods. I am also ignoring transfer learning.


texts: metric accuracy for linear regression and non linear regression

texts:
Is there a one size fit all metric to measure accuracy / error rate for both linear or non linear regression models?

For example adjust R2 is only for multi linear regression (or so they say).
RMSE seems the best choice for majority except for curve lines of best fit as it can be misleading.

How do I know which metric to use?


texts: What representation of data should I choose, pandas, numpy or tensors to train neural network model

texts: I am having a pandas dataframe divided to X_train, X_test, y_train, y_test by train_test_split and I am using it to train my neural network model for binary classification. It is taking some time and I was wondering if it would be faster if I changed it from pandas.dataframe to numpy.array or tensor.


texts: Plotting in Multiple Linear Regression in Python 3

texts:
So I'm working on linear regression. So far I've managed to plot in linear regression, but currently I'm on Multiple Linear Regression and I couldn't manage to plot it, I can get some results if I enter the values manually, but I couldn't manage to plot it. Below is my code block and dataset and error, what can i change to plot it?

Dataset:

deneyim maas    yas
0.5 2500    22
0   2250    21
1   2750    23
5   8000    25
8   9000    28
4   6900    23
15  20000   35
7   8500    29
3   6000    22
2   3500    23
12  15000   32
10  13000   30
14  18000   34
6   7500    27

Code block:

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.linear_model import LinearRegression

dataset = pd.read_csv("multiple-linear-regression-dataset.csv",sep = ";")

x = dataset.iloc[:,[0,2]].values
y = dataset.maas.values.reshape(-1,1)

multiple_lr = LinearRegression()
multiple_lr.fit(x,y)

b0 = multiple_lr.intercept_
b1 = multiple_lr.coef_
b2 = b1

multiple_lr.predict(np.array([[10,35],[5,35]]))

array = np.array([0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]).reshape(-1,1)
y_head = multiple_lr.predict(array)

plt.scatter(x,y)
plt.plot(array, y_head, color = "red")
plt.show()

It says ValueError: shapes (16,1) and (2,1) not aligned: 1 (dim 1) != 2 (dim 0) when I try to compile it.


texts: Linear regression - LMS with gradient descent vs normal equations

texts: I wonder when to use linear regression with stochastic or batch gradient descent to minimize the cost function vs when to use normal equations? The algorithms using gradient descent are iterative, so they might take more time to run, as opposed to the normal equation solution, which is a closed form equation. But it does use matrices to store the training data. Does this mean gradient solutions require more processing power, but using the normal equation method requires more memory because of the matrices? Which method is optimal in what scenario?


texts: How to calculate similarity between 2 users based on the images they share?

texts:
Say there are 2 users, A and B, and they each shared 10 images (in some social media site), which I have collected in 2 folders separately.
I want to calculate the similarity between the 2 users based on all the images they shared. For example, there is a sense of the 2 users being more similar if they both share pictures of food; and the 2 users would be dissimilar if one of them shares pictures of food while the other posts selfies. And the 2 users should be even more similar if they both share pictures of food containing chicken dishes only. And so on.
How do I calculate this similarity? I thought of adding up the similarities of each pair of images, and then averaging them. But this clearly doesn't work, because take only 1 user for instance. If you calculate this user's similarity to itself, the similarity will not be 1. But we expect the similarity of something with itself to be 1.
So, how do we measure this similarity of users based on their images shared?
EDIT: I am getting the following similarities in my actual use case:
import time

img_path = 'Images/All_Users/'
user_list = ['User '+str(i) for i in range(1,11)]
df = pd.DataFrame(columns=user_list, index=user_list)
for user1 in user_list:
    for user2 in user_list:
        t = time.time()
        sum_img_sim = 0
        user1_files = [x for x in os.listdir(img_path+user1+'/') if "jpg" in x]
        user2_files = [x for x in os.listdir(img_path+user2+'/') if "jpg" in x]

        for image1 in user1_files:
            for image2 in user2_files:
                sim = image_similarity(img_path+user1+'/'+image1, img_path+user2+'/'+image2)
                sum_img_sim += sim

        df[user1][user2] = sum_img_sim/(len(user1_files)*len(user2_files))
        print(user1 +' and '+ user2 + ' done in ' + str(time.time()-t) + ' secs. Similarity: ' + str(df[user1][user2]))

So, as you can see, there's not much difference in the similarities, and I'm getting anomalies like User 2 has a similarity of 0.47 with itself, while User 1 and User 10 have a similarity of 0.49


texts: What's the best way to detect bible verse mentions in a text?

texts:
I have a set of 10 verses from the Bible in English. I want to detect the occurrence of any of these verses in a text. What would be the best way to go about doing this?
Note that verses of the Bible are worded differently from one translation to another. For example, the verse "Cast all your anxiety on him because he cares for you" exists in other translations as:
KJV
Casting all your care upon him; for he careth for you.
ESV
casting all your anxieties on him, because he cares for you.
NLT
Give all your worries and cares to God, for he cares about you.
Also, people may make typos or make mistakes when writing a verse.
Would you say Semantic Search is the best approach for this? If so, do I use a pertained word embeddings model, retrain it on my set of 10 verses, and then search my text for the occurrence of these verses?
How would the search mechanism work? Do I get the word embedding sentence by sentence and compare each sentence's embedding one by one with my Bible corpus of 10 verses' embeddings?


texts: Multivariate time series prediction with binary target

texts:
I have an electronic component whose sensors record temperature, current and voltage values of various sub-elements. These readings are taken at regular intervals of time and I organized them as records of a dataset. In addition to these features, the dataset has a column with value 1 or 0 if at that moment the component is experiencing a malfunction or not. Here is an example of the dataset structure:

My goal is to build a model that is able to predict the occurrence of a malfunction with some time advance, so I think this is a multivariate time series prediction problem.

What approaches could I use to achieve this goal? I have read that it is possible to use Recurrent Neural Networks / LSTM but I do not have an adequate GPU.


texts:
I've been using SQL since 1996, so I may be biased. I've used MySQL and SQLite 3 extensively, but have also used Microsoft SQL Server and Oracle.

The vast majority of the operations I've seen done with Pandas can be done more easily with SQL. This includes filtering a dataset, selecting specific columns for display, applying a function to a values, and so on.

SQL has the advantage of having an optimizer and data persistence. SQL also has error messages that are clear and understandable. Pandas has a somewhat cryptic API, in which sometimes it's appropriate to use a single [ stuff ], other times you need [[ stuff ]], and sometimes you need a .loc. Part of the complexity of Pandas arises from the fact that there is so much overloading going on.

So I'm trying to understand why Pandas is so popular.


texts: Is Vector in Cosine Similarity the same as vector in Physics?

texts: I'm new to Data Science. I'm trying to understand cosine similarity and it seems like the equation is about finding the distance between two vectors. From what I've Googled, a vector needs to have magnitude and direction. But in CS, it seems like it's a 1-dimensional array. Is vector in CS the same as vector in Physics? If so, what is the direction of a vector. And if a vector is like this [1, 0, 1, 0] what is the magnitude of this vector?


texts: Using Apache Spark to do ML. Keep getting serializing errors

texts:
so I'm using Spark to do sentiment analysis, and I keep getting errors with the serializers it uses (I think) to pass python objects around.

PySpark worker failed with exception:
Traceback (most recent call last):
  File "/Users/abdul/Desktop/RSI/spark-1.0.1-bin-    hadoop1/python/pyspark/worker.py", line 77, in main
    serializer.dump_stream(func(split_index, iterator), outfile)
  File "/Users/abdul/Desktop/RSI/spark-1.0.1-bin-    hadoop1/python/pyspark/serializers.py", line 191, in dump_stream
    self.serializer.dump_stream(self._batched(iterator), stream)
  File "/Users/abdul/Desktop/RSI/spark-1.0.1-bin-    hadoop1/python/pyspark/serializers.py", line 123, in dump_stream
    for obj in iterator:
  File "/Users/abdul/Desktop/RSI/spark-1.0.1-bin-    hadoop1/python/pyspark/serializers.py", line 180, in _batched
    for item in iterator:
TypeError: __init__() takes exactly 3 arguments (2 given)

and the code for serializers is available here

and my code is here


texts: Ranking algorithm based on a handful of features

texts:
I am trying to determine the apt algorithm for a ranking problem that I am working on. I have social media metrics - engagement, sentiment, audience size etc for several brands and am looking for a ranking / classification algorithm to rank them.

I am not sure if I have a dependent variable or label class for classical classification algorithms.

The data is aggregated by brands and algorithms needs to rank the brands based on the metrics.

Any ideas would be much appreciated.


texts: How to add previous predictions for new predictions in LSTM?

texts:
I am trying to train a model on a big data sequence like this [0.2 0.1 0.1 ..... 0.4 0.8] . I create X vectors with length 60 for inputs and Y scaler numbers as labels(It means the LSTM reads first 60 numbers as input(one row in X_train) and the 61'th number as the output label(rows in y_train)).

model = Sequential()
model.add(LSTM(units = 50, return_sequences = True , input_shape = (X_train.shape[1], 1)))
model.add(Dense(units = 1))
model.compile(optimizer = 'adam', loss = 'mean_squared_error')
model.fit(X_train, y_train, epochs = 100, batch_size = 32)

And:

X_train.shape = (1000,60,1) ,       y_train.shape = (1000,)

It's OK till here, but the problem is in the prediction part. What I am trying to do is create a (60*1) input vector and use it to predict the next number in my sequence. Then adding the new predicted number to my sequence to predict the next number(second predicted  number) and so on.
For that goal, I created a new model, retrieved the weights from previous model, then fed new_model only with one (60*1) vector to predict the next number. then added the predicted number to the sequence and shift input vectors one number to the right(to use new predicted number for the next number prediction).

new_model = Sequential()
new_model.add(LSTM(units = 50, return_sequences = True , input_shape = (1 , 60 , 1 )))
new_model.add(Dense(1))
old_weights = model.get_weights()
new_model.set_weights(old_weights)
new_model.compile(optimizer = 'adam', loss = 'mean_squared_error')

inputs = []
for i in range(10):
    inputs = dataset_total[len(dataset_total) - 60:].values
    inputs = np.reshape(inputs, (1 , 60, 1))
    predicted = new_model.predict(inputs)
    inputs.append(predicted)

But what do I get is such errors:

ValueError: Input 0 is incompatible with layer lstm_61: expected
ndim=3, found ndim=4

I don't know how to solve this problem!

There is also a similar but without related answer here(for clarification):
How to Predict the future values of time horizon with Keras?


texts: How to use GridSearch for LinearSVC / Random Forest with time series data

texts:
I have a question related on how to use the GridSearch to find the best models for my problem with time series data.
Every 3 rows is 1 one row in the original dataset. To make my time series problem a supervised one, I parsed like the one below. This was resolved from one of my previous question.

id
Age
gender
m1
m2
m3
Label

1
20
M
12.4
34
12
0

2
20
M
13
324
34
0

3
20
M
34
232
12
0

4
45
F
1.3
32
19
1

5
45
F
14
132
19
1

6
45
f
94
232
19
1

My question is: How can I use GridSearch for example to find my best machine learning model configuration using time series data? As far as I understand, using cross validation wouldn't work in this case because of the time series nature of the dataset.
I'm not sure how to proceed with this.


texts: How to bypass ID column without being used in the training model but have it as output - Azure ML

texts:
The input data in the model includes column ControlNo.

But I don't want this column being part of learning process so I'm using Select Columns in Dataset to exclude ControlNo column.
But as a output I want those columns:
ControlNo, Score Label, Score Probability

So basically I need NOT to include column ControlNo into learning process,
but have it as output along with Score Label column.
How can I do that?


texts:
I am working on a dataset to classify facial expressions.
Dataset has 7 classes, training images 28000 and test images 7000. I created 2 models
Model1:
this model has 11 layers. Initially model was working fine but now accuracy is increasing very slowly after 60% and drooping sometimes. model 1 link
then I thought that due large dataset my model is over fitting and I reduced the images by randomly deleting in each class and reducing train set images to 15000 and test set images to 3900. and created another model.
Model 2:
This model has 9 layers. The graph for first 40 epochs was fine but after then when I resumed training its seems to overfit.model 2 link
first 40 epoch

this seems fine i guess
after resuming training

I have also tried model with fewer layers nut cant figure out why my accuracy is stuck at and model is not improving
dataset link, I don't understand is there a problem with dataset, can anyone provide me link to another dataset. Figure out what changes are required in model.


texts: What is the origin of YOLO/darknet coordinates

texts:
I am aware of that darknet/yolov3 outputs relative coordinates, would the coordinate (0,0) be at the bottom left, or at the top left?

I am confused as opencv2 seems to have the y=0 at the top of the image as is usual when working with matrices. Yet mathematicaly and intuitively (for me at least) the coordinate (0,0) should be at the bottom left.


texts:
I have seen here but I really cannot realize that.

  In this framework, the learner receives samples and must select a generalization function (called the hypothesis) from a certain class of possible functions. The goal is that, with high probability (the "probably" part), the selected function will have low generalization error.

Actually we do that in every machine learning situations and we do latter part for avoiding over-fitting. Why do we call it PAC-learning?
I also have not get the meaning of the math too. Is there anyone that can help?


texts:
Most of the classification problems I have seen assume that the classes are kind of independent. For example, a digit classification uses a softmax output that penalizes the output prediction error the same way for any digits other than the correct label. However, let's say we have an image of number 5. I ideally want to predict as closely as possible to 5. Digit 5 is the correct prediction, but 6 or 4 are the best bets after 5. And 0 or 9 are the worst predictions.
Is there a known way to do this?


texts: found \N in my data does not count as missing values in r

texts:
So scrolling through my columns I find \N embedded. I need to count them, but I get an error.  Would it be considered a missing value, it's new to me.

# Check whether attribute HeadOfState in country has any missing values, and if so, how many.
country$headOfState[country$headOfState==""] <- NA
country$headOfState[country$headOfState==\N] <- NA
sum(is.na(country$headOfState))

# Check whether attribute IndepYear in country has any missing values, and if so, how many.
country$IndepYear[country$indepYear==""] <- NA
country$indepYear[country$indepYear=='\N'] <- NA
sum(is.na(country$indepYear))

Error: unexpected input in "country$headOfState[country$headOfState==\"

  country$headOfState[country$headOfState==\N] <- NA
  Error: unexpected input in "country$headOfState[country$headOfState==\"


texts: Python library to compute some metrics for multioutput-multiclass classification task

texts: Is there any Python library that provides ready-to-use metrics to analyze the performance of a classifier for a multioutput-multiclass classification task? scikit-learn doesn't have this option yet (as stated in the documentation and in the corresponding feature request on GitHub).


texts: Perpendicularity between random variables?

texts:
I am reading Bechavod et al. (2017) [1], and at page 3 there is written:

  In the example, each data point lies in $X = (X_1,X_2) = \{0, 1\}^2$
  and has two features—$X_1 = A$ is the protected attribute, and $X_2$
  is a non-protected attribute—and a label in $Y = {0, 1}$. Given
  $\epsilon \in (0,\frac{1}{4})$, we define a distribution $\mathcal{D}$
  over labelled examples as follows:

  $\mathbb{P}[Y = 1] = 0.5$

  $\mathbb{P}[A = y|Y = y] = 1 − \epsilon $

  $\mathbb{P}[X2 = y|Y = y] = 1 − 2\epsilon$

  Note that $\mathcal{D}_\epsilon$ is defined s.t. $A \,\bot\, X_2|Y$.

What does it mean, mathematically, that $A \,\bot\, X_2|Y$? Aren't those random vectors?

[1] Yahav Bechavod, Katrina Ligett, "Penalizing Unfairness in Binary Classification" (link)


texts: high root mean square error in regression model

texts:
I am applying regression to a data of 110 rows and 7 columns ,each having targets. When I applied Lasso to the data and calculated the RMSE value ,the RMSE value is coming to be 13.11.I think the RMSE value should be close to zero.What is the permissible values of RMSE in a regression model.What could have gone wrong in the computation.

from sklearn import linear_model
reg = linear_model.Lasso(alpha = .00001)
reg.fit(Xt,Yt)
ans=reg.predict(Xts)
print(ans)
from sklearn.metrics import mean_squared_error
print(mean_squared_error(Yts, ans))

whereas when I try cross validation the MSE scores are way below 0.35

kfold = KFold(n_splits=10)
results = cross_val_score(reg, full_data, target, cv=kfold)
print("Results: %.2f (%.2f) MSE" % (results.mean(), results.std()))
results
Results: -0.13 (0.45) MSE


texts: Dealing with the test set of imbalanced data

texts:
I am working on a problem dealing with unbalanced data that has a very specific request.

I would like to know the following:

When I have an imbalanced dataset and I do train test split, the test samples also have an imbalance. How can I at least balance the test set so that when I do the confusion matrix equal numbers are tested?

Here's an example (Classification):

If I have three groups: A 60,B 30,C 20. For simplicity, the test set is 10%. So, instead of having a train/test set of 54/6, 27/3, 18/2, I would like 58/2, 28/2 and 18/2 OR 54/2, 27/2, 18/2.

How can I do this?


texts: Padding in Keras with output half sized input

texts:
Here is my Keras model I'm working on:

model = Sequential()
model.add(Conv2D(64, kernel_size=(7, 7), strides = 2, padding = 3,
                 input_shape=input_shape)) # (224,224,64)

model.add(MaxPooling2D(pool_size=(2, 2), strides = 2)) # (112,112,64)
model.add(Conv2D(192, kernel_size = (3,3), padding = 1)) #(112,112,192)
model.add(MaxPooling2D(pool_size = (2,2),strides = 2)) #(56,56,192)
model.add(Conv2D(128, kernel_size = (1,1))) #(56,56,128)
model.add(Conv2D(256, kernel_size = (3,3), padding = 1)) #(56,56,256)
model.add(Conv2D(256, kernel_size = (1,1))) #(56,56,256)
model.add(Conv2D(512, kernel_size = (3,3),padding = 1)) #(56,56,512)
model.add(MaxPooling2D(pool_size = (2,2), strides = 2)) #(28,28,512)
model.add(Conv2D(256, kernel_size = (1,1))) #(28,28,128)
model.add(Conv2D(512, kernel_size = (3,3), padding = 1)) #(28,28,512)
model.add(Conv2D(256, kernel_size = (1,1))) #(28,28,128)
model.add(Conv2D(512, kernel_size = (3,3), padding = 1)) #(28,28,512)
model.add(Conv2D(256, kernel_size = (1,1))) #(28,28,128)
model.add(Conv2D(512, kernel_size = (3,3), padding = 1)) #(28,28,512)
model.add(Conv2D(256, kernel_size = (1,1))) #(28,28,128)
model.add(Conv2D(512, kernel_size = (3,3), padding = 1)) #(28,28,512)
model.add(Conv2D(512, kernel_size = (1,1))) #(28,28,512)
model.add(Conv2D(1024,kernel_size = (3,3), padding = 1)) #(28,28,1024)
model.add(MaxPooling2D(pool_size = (2,2), strides = 2)) #(14,14,1024)
model.add(Conv2D(512, kernel_size = (1,1))) #(14,14,512)
model.add(Conv2D(1024,kernel_size = (3,3), padding = 1)) #(14,14,1024)
model.add(Conv2D(512, kernel_size = (1,1))) #(14,14,512)
model.add(Conv2D(1024,kernel_size = (3,3), padding = 1)) #(14,14,1024)
model.add(Conv2D(1024, kernel_size = (3,3), padding = 1)) #(14,14,1024)
model.add(Conv2D(1024, kernel_size = (3,3), strides = 2, padding = 3)) #(7,7,1024)
model.add(Conv2D(1024,kernel_size = (3,3), padding = 1)) #(7,7,1024)
model.add(Conv2D(1024, kernel_size = (3,3), padding = 1)) #(7,7,1024)

model.add(Flatten())
model.add(Dense(4096))
model.add(Dense(7*7*30))
model.add(Reshape(7,7,30))

When I compile it, I got an error for padding because Keras knows only 'same', valid' and 'casual'. I understand these, but I really need padding somewhere to be equal to 3 because my output should be a half of input (we have strides equal to 2). I really don't know how to fix it. How to do padding if we want to half size the input with strides 2?


texts:
How do you manage the code (and notebooks) in the ML Scenario Manager workspace in SAP Data Intelligence?

It seems that SAP DI does not support a direct integration of Git.


texts: Calculate probability vector from sample data

texts:
I'd like to compare two distributions using Jensen-Shannon Divergence metric. To do this, I need two probability vectors to plug into distance.jensenshannon(p, q). From the scipy.spatial documentation.

  scipy.spatial.distance.jensenshannon(p, q, base=None)[source]

  Parameters:

  p(N,) array_like left probability vector

  q(N,) array_like right probability vector

Question

How can I calculate probability vectors from sample data?

Example:

from scipy.spatial import distance
import numpy as np

x1 = np.random.normal(size=100)
x2 = np.random.normal(size=100)

p =

q =

jsd_metric = distance.jensenshannon(p, q)

Can I accomplish this using scipy.stats.norm.pdf()?

p = scipy.stats.norm.pdf(x1)
q = scipy.stats.norm.pdf(x2)


texts: Should I modify my data to reflect time dependence of my rules?

texts:
Say I have 100 points. Each of these points have a date associated with them and have a "yes" label. I can artificially add "no" points to create more data. Now, I want, using these 100 "yes" points + X "no" points, to predict if a given set of features (a new point) will be categorized "yes" or "no".

I can develop a model that does so.

Now, let's imagine that some "rules of the game" can change. Something that in 1967 was a "yes" point, becomes a "no" point after a (unknown) rule change in 1989. Most factors stay the same, but one change causes the point to change classification drastically.

Can I modify my data to add more importance to recent dates (that is, duplicate recent values so that in "importance", 1 value from 2010 equals 3 values from 1990 and 5 values from 1970 for example), because it's a better representation of our prediction for a future point? Or is this a terrible idea?

Basically, to simplify, if my data is 5 points, labeled "1939", "1967", 1980", "1982", "2010", can I artificially modify it to be 12 points, "1939", "1967", 1980", "1980", "1982", "1982", "1982", "2010", "2010", "2010", "2010", "2010"?

Of course, chosing the "importance" of a given more recent point is pretty difficult to begin with.


texts: Difference Between Feature Engineering and Feature Learning

texts:
I am playing with features (input data) to improve my model's accuracy.

If I have a raw time-series dataframe, does feature engineering mean extracting properties or characteristics of my raw data and feed it as input? Or will the algorithm learn these from the time-series itself?

In other words, should I create a column that is comprised of the moving average, or will the algorithm pick up on moving average from the raw data?

Is feature engineering just the munging of independent variables? Or is it extracting features that are dependent on other raw data?

EDIT:

Here's another question: If I have a categorical feature, would it be better to have it as a one-hot vector (say, 5 binary inputs), or to have it as one input with range [0,4]?

How does one intuitively know the answer to these questions??


texts: Gene innovation numbers in NEAT implementatoins

texts:
In NEAT (neuroevolution through augmenting topologies) algorithm description, an innovation number, e.g. id, is assigned to each gene so that genomes can be crossed over meaningfully: genes having corresponding innovation numbers are aligned and then crossover takes place.

Is this innovation number global only within distinct genemoes ? Meaning that each genome has its own counter or is it the case that there is a unique counter for all genomes ?

Description of algorithm: https://www.cs.cmu.edu/afs/cs/project/jair/pub/volume21/stanley04a-html/node3.html


texts: Connection between Embedding and LSTM and Dense layer

texts:
I am building a "predict next word" model using the following model architecture. The codes fine, but I have a few questions:

# define model
model = Sequential()
model.add(Embedding(vocab_size, 50, input_length=seq_length))
model.add(LSTM(100, return_sequences=True))
model.add(LSTM(100))
model.add(Dense(100, activation='relu'))
model.add(Dense(vocab_size, activation='softmax'))
print(model.summary())

Between the Embedding and LSTM layers, are there 50 connection weights, or 50 * 100 connection weights? Based on the documentation, this LSTM block contains 100 cells. So I am not sure how exactly does the Embedding layer connect to the LSTM layer?

Also, between LSTM(100) layer and the Dense(100, activation='relu') layer, are there 100 connection weights or 100 * 100 connection weights?

Thanks!


texts: MLP on Iris Data not working but it does fine on MNIST - Keras

texts:
So I'm a little bit baffled. I have just started working with the Keras framework for Python (which is awesome by the way!).

However just trying a few simple test of neural networks has got me a bit confused. I initially tried to classify the Iris data as it was a small, quick and simple dataset. However when I constructed a neural network for it (4 input dimensions, 8 node hidden layer, 3 node output layer for binary classification of the 3 classes). This however didn't produce any predictive powers at all (100 samples in the training set, a separate 50 in the test set. Both had been shuffled so as to include a good distribution of classes in each).

Now I thought I was doing something wrong, but I thought I'd give the network a quick test on the MNIST dataset just in case. So I used basically the exact same network (other than changing the input dimensions to 784, hidden nodes to 30, and output nodes to 10 for the binary encoded 0-9 output). And this worked perfectly! With a 97% accuracy rate and a 5% loss.

So now I'm not sure why the Iris dataset isn't playing ball, does anyone have any clues? I've tried changing the number of hidden layer nodes and also normalized the X input.

Here's an output I've produced when I've manually run some of the test set through the trained Iris model. As you can see it's basically producing a uniform random guess.

Target: [ 1.  0.  0.]  | Predicted: [[ 0.44635904  0.43874186  0.45729554]]
Target: [ 0.  0.  1.]  | Predicted: [[ 0.44618103  0.43869928  0.45735642]]
Target: [ 0.  1.  0.]  | Predicted: [[ 0.44612524  0.43863046  0.45729461]]
Target: [ 0.  0.  1.]  | Predicted: [[ 0.44617626  0.43870446  0.45736298]]
Target: [ 0.  0.  1.]  | Predicted: [[ 0.44613886  0.43865535  0.45731983]]

And here's my full code for the Iris MLP (the MNIST one is essentially the same)

import numpy as np
import random
from keras.models import Sequential
from keras.layers import Dense, Activation
from sklearn.datasets import load_iris
from sklearn import preprocessing

# Model Layers Defnition
# Input layer (12 neurons), Output layer (1 neuron)
model = Sequential()
model.add(Dense(8, input_dim=4, init='uniform', activation='sigmoid'))
model.add(Dense(3, init='uniform', activation='sigmoid'))

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Iris has 150 samples, each sample has 4 dimensions
iris = load_iris()
iris_xy = zip(iris.data, iris.target)
random.shuffle(iris_xy) # Iris data is sequential in it's labels
iris_x, iris_y = zip(*iris_xy)
iris_x = preprocessing.normalize(np.array(iris_x))
iris_y = np.array(iris_y)

# Encode decimal numbers to array
iris_y_enc = np.zeros(shape=(len(iris_y),3))
for i, y in enumerate(iris_y):
    iris_y_enc[i][y] = 1

train_data = np.array(iris_x[:100]) # 100 samples for training
test_data  = np.array(iris_x[100:]) # 50 samples for testing
train_targets = np.array(iris_y_enc[:100])
test_targets  = np.array(iris_y_enc[100:])

model.fit(train_data, train_targets, nb_epoch=10)

#score = model.evaluate(test_data, test_targets)

for test in zip(test_data, test_targets):
    prediction = model.predict(np.array(test[0:1]))
    print "Target:", test[1], " | Predicted:", prediction


texts: Which metric should I use for classifying an imbalace data with fewer labels for the negative class?

texts:
From reading, I understand that when we have fewer positive class labels, it is better to use precision or recall as the evaluation metric. Which metric should I use when we have fewer negative samples?
I'm looking for an approach other than switching the labels.
Problem setting: I'm developing parametrized fragility functions for predicting damage to a structure (for example trees). An example of fragiltiy function is here The fragility function will estimate the probability of exceeding a damage state given some parameters (say wind load). The damage state can be expressed in terms of damage ratio (0-1 with 1 being fully damaged). Now, we are interested in estimating the probability of exceeding a damage ratio given features. To elaborate, the probability of any damage would be P(Damage_ratio>0.0|features). Logistic regression can be used to learn this curve from data after categorizing 0-1 continuous damage ratio to damaged (- class)/no damage (+ class) for a particular threshold. Now, as we move from the threshold from 0 to 1, the dataset transforms from imbalanced data dominated by damaged cases to a balanced state and then finally to another imbalanced data dominated by non-damage case.
Now, when learning the model, 'AUR-ROC' performs really well when the data is balanced. Precision performs well when the data is imbalanced with more no-damage case (P(Damage_ratio>0.1|features)). These metrics don't do well for the case with few negative case (P(Damage_ratio>0.9|features)). I tried switching the label with very limited success. Are there any other 'metrics' that perform well in an imbalanced data setting?


texts: How do autoencoders reconstruct images/color?

texts: I am abit confused by how autoencoders are able to reconstruct colors in images. According to me a CNN has feature detectors that convert the image into a sequence of feature or activation maps. These activation map corrospond to the degree of presense of a structural feature. Then how is the color information preserved in autoencoders if it is not a structural feature? To me the color information gets lost very quickly but most visuals of autoencoders show they preserve color very well.


texts: Logitic Regression cost function - what if ln(0)?

texts:
I am building logistic regression from scrap.

The simplified cost function I am using is (from machine learning course on coursera):

in specific case during learning,
one observation in training set y is 0 - but the specific choice of betas in:

makes g(z) = h(x) = 1 , because e.g. z > 50.

in this case my right side od J is (1 - 0) * log(1 - 1) what is -inf (I am doing my calculations in python)

I understand that in this case value of cost function should be high because the probability of y = 1 is very big while the truth is that it actually is 0.

Is the problem approximation of g(50) being 1 instead of something like: 0.999999? Or there is some more fundametal error with my logic?

because this specific example the summation of cost of all observations is nan (not a number) in my code.


texts: How to tune the proportion of minority class after oversampling in imbalanced data with cross-validation?

texts:
I have an imbalanced data set and I want to balance them with SMOTENC with cross-validation. In order to determine the performance of the classifier on the original data, I will cross-validation as follows:
Here, feature X is divided into X_train and X_val.
scoring = {'accuracy' : make_scorer(accuracy_score),
           'precision' : make_scorer(precision_score),
           'recall' : make_scorer(recall_score),
           'f1_score' : make_scorer(f1_score)}

model=RandomForestClassifier(n_estimators=50,random_state=10)

results = cross_validate(estimator=model, X=X, y=Y, cv=10, scoring=scoring)

Based on this code, I can find the performance of random forest on the original data.
However, I am a bit confused about how I should compare the oversampled data with the original data. I want to determine the best proportion of data and I use the following code. Should I split the original data to train and val and oversample train data and find the performance on val data?
I am wondering if the results of these codes are comparable?
percentage = np.arange(0.20,0.5,0.02)
Measurements = np.zeros( (np.size(percentage),4) )

skf = StratifiedKFold(n_splits=10)

for train_index, test_index in skf.split(X, y):
    X_train, X_val = X[train_index], X[test_index]
    y_train, y_val = y[train_index], y[test_index]

    counter = 0
    for per in percentage:

        smote_nc = SMOTENC(sampling_strategy = per, categorical_features=[0,1,2,3,4,6,8,9], random_state=0)
        X_resampled, y_resampled = smote_nc.fit_resample(X_train, y_train)

        X_resampled = pd.DataFrame(X_resampled, columns=X_val_train.columns)

        after_X_resampled = encoding(X_resampled)

        RF = RandomForestClassifier(n_estimators=50,random_state=10)
        RF.fit(after_X_resampled, y_resampled)

        prediction = RF.predict(X_val)

        Measurements[counter,0] += accuracy_score(y_val, prediction)
        Measurements[counter,1] += precision_score(y_val, prediction)
        Measurements[counter,2] += recall_score(y_val, prediction)
        Measurements[counter,3] += f1_score(y_val, prediction)

        counter+=1


texts: How to use Cross Entropy loss in pytorch for binary prediction?

texts:
In the pytorch docs, it says for cross entropy loss:

  input has to be a Tensor of size (minibatch, C)

Does this mean that for binary (0,1) prediction, the input must be converted into an (N,2) tensor where the second dimension is equal to (1-p)?

So for instance if I predict 0.75 for a class with target 1 (true), would I have to stack two values (0.75; 0.25) on top of each other as input?


texts:
I am new to machine learning and trying to apply it to my problem.
I have a training dataset with 44000 rows of features with shape 6, 25. I want to build a sequential model. I was wondering if there is a way to use the features without flattening it. Currently, I flatten the features to 1d array and normalize for training (see the code below). I could not find a way to normalize 2d features.
dataset2d = dataset2d.reshape(dataset2d.shape[0],
                              dataset2d.shape[1]*dataset2d.shape[2])
normalizer = preprocessing.Normalization()
normalizer.adapt(dataset2d)
print(normalizer.mean.numpy())

x_train, x_test, y_train, y_test = train_test_split(dataset2d, flux_val,
                                                    test_size=0.2)

# %% DNN regression multiple parameter
def build_and_compile_model(norm):
    inputs = Input(shape=(x_test.shape[1],))
    x = norm(inputs)
    x = layers.Dense(128, activation="selu")(x)
    x = layers.Dense(64, activation="relu")(x)
    x = layers.Dense(32, activation="relu")(x)
    x = layers.Dense(1, activation="linear")(x)
    model = Model(inputs, x)
    model.compile(loss='mean_squared_error',
                  optimizer=keras.optimizers.Adam(learning_rate=1e-3))
    return model

dnn_model = build_and_compile_model(normalizer)
dnn_model.summary()
# interrupt training when model is no longer imporving
path_checkpoint = "model_checkpoint.h5"
modelckpt_callback = keras.callbacks.ModelCheckpoint(monitor="val_loss",
                                                     filepath=path_checkpoint,
                                                     verbose=1,
                                                     save_weights_only=True,
                                                     save_best_only=True)
es_callback = keras.callbacks.EarlyStopping(monitor="val_loss",
                                            min_delta=0, patience=10)
history = dnn_model.fit(x_train, y_train, validation_split=0.2,
                        epochs=120, callbacks=[es_callback, modelckpt_callback])

I also tried to modify my model input layer to the following, such that I do not need to reshape my input
inputs = Input(shape=(x_test.shape[-1], x_test.shape[-2], ))

and modify the normalization to the following
normalizer = preprocessing.Normalization(axis=1)
normalizer.adapt(dataset2d)
print(normalizer.mean.numpy())

But this does not seem to help. The normalization adapts to a 1d array of length 6, while I want it to adapt to a 2d array of shape 25, 6.
Sorry for the long question. You help will be much appreciated.


texts: Time Complexity notation in Big Data platforms

texts:
I am redesigning some of the classical algorithms for Hadoop/MapReduce framework. I was wondering if there any established approach for denoting Big(O) kind of expressions to measure time complexity?

For example, hypothetically, a simple average calculation of n (=1 billion) numbers is O(n) + C operation using simple for loop, or O(log) I am assuming division to be a constant time operation for the sake for simplicity. If i break this massively parallelizable algorithm for MapReduce, by dividing data over k nodes, my time complexity would simply become O(n/k) + C + C'. Here, C' can be assumed as the job planning time overhead. Note that there was no shuffling involved, and reducer's job was nearly trivial.

I am interested in a more complete analysis of algorithm with iterative loops over data and involve heavy shuffling and reducer operations. I want to incorporate, if possible, the I/O operations and network transfers of data.


texts: Backpropagation Through Time (BPTT) of LSTM

texts:
I am currently trying to understand the BPTT for Long Short Term Memory (LSTM) in TensorFlow. I get that the parameter num_steps is used for the range that the RNN is rolled out and the Error backpropagated. I got a general question of how this works.

For reference a repitition of the formulas. I'm referring to:
Formulas LSTM (https://arxiv.org/abs/1506.00019)

Question:
What paths are backpropagated that many steps? The constant error carousel is created by the formula 5, and the derivate for backpropagation ($s(t)\rightarrow s(t-1)$) is $1$ for all timesteps. This is why LSTMs capture long range dependencies. I get confused with the dependencies of $g(t)$, $i(t)$, $f(t)$ and $o(t)$ of $h(t-1)$. In words: The current gates do not just depend on the input, but also on the last hidden state.

Doesn't this dependency lead to the exploding/vanishing gradients problem again?

If I backpropagate along these connections I get gradients that are not one. Peephole connections essentially lead to the same problem.

Thanks for your help!


texts: How to convert tensors used in machine translation back to words?

texts:
I am new to machine translation and I trained a Seq2Seq model. However the model gives output as a tensor. How do I convert the tensor back to words. I used the multi30k dataset and I created the iterator this way:
rain_iterator, valid_iterator, test_iterator = BucketIterator.splits(
                                                (train_data, valid_data, test_data),

                                                batch_sizes=(BATCH_SIZE, BATCH_SIZE, BATCH_SIZE),

                                                sort_within_batch=True,

                                                sort_key=lambda x: len(x.src),

                                                device=device)


texts: Understanding math notation in infoGAN paper

texts:
I'm reading this paper about mutual information in infoGAN infoGAN_paper_link and already have the code to run it. I pretty much found code for it which is fine and dandy except for the fact that I kinda don't understand some of the code in the cost function. So, I looked at the paper to dissect it for better understanding and came across some math notation that I don't understand (pic below). The usage of the notations I'm trying to figure out are how the "~" in the Expectation subscripts, "'", and "||" symbols are used.
This is what I think the notation means.

"~" in the expectation subscript: the variable on the left of "~" can be any continuous value coming from whatever is to the right of the "~"
"'" next to the "c" in P(c'|x): I have no clue. I thought those were symbols for derivates but that makes no sense for this equation so it's def not that.
"||": I'm not sure. I only know of these symbols being used in Norms but that's obviously not the case here.

Screenshot of a formula from the paper

The actual code I was trying to figure out was this in PyTorch. It's the variational lower bound (mutual information term) in the cost (It's not the formulas in the screenshot above). However, I ran into the formulas in the screenshot first before getting to the formula calculations for the variational lower bound formula.
mutual_information_lower_bound = lambda c_true, mean, logvar: Normal(mean,logvar.exp()).log_prob(c_true).mean()

Thanks for the help!


texts:
I know that [CLS] means the start of a sentence and [SEP] makes BERT know the second sentence has begun.

However, I have a question.

If I have 2 sentences, which are s1 and s2, and our fine-tuning task is the same.

In one way, I add special tokens and the input looks like [CLS]+s1+[SEP] + s2 + [SEP].

In another, I make the input look like [CLS] + s1 + s2 + [SEP].

When I input them to BERT respectively, what is the difference between them?

Will the s1 in second one integrate more information from s2 than the s1 in first one does?
Will the token embeddings change a lot between the 2 methods?

Thanks for any help!


texts:
As far as I understood from the AlphaGo Zero system:

During the self-play part, the MCTS algorithm stores a tuple ($s$, $\pi$, $z$) where $s$ is the state, $\pi$ is the distribution probability over the actions in the state and $z$ is an integer representing the winner of the game that state is in.
The network will receive $s$ as input (a stack of matrices describing the state $s$) and will output two values: $p$ and $v$. $p$ is a distribution probability over the actions and $v$ is a value in $[-1,1]$ representing which player is likely to win the game.
For the training it will use the following loss function:

$$l = (z - v)^2 - \pi^T log(p) + c ||\theta||^2$$

Lastly, it evaluates the new network and it starts the self-play section again.

My questions

If the network receives only the state $s$ (represented as matrices) as input, how can it then calculate the loss function if the values $\pi$ and $z$ are needed?
If these values are indeed passed as input for the network, are they passed through the convolutional (and other) layers of the network? Because if this is true, there is no mention in the article (unless I missed it) of this information.


texts: CNN tagging such that each input could have multiple tags

texts:
Thanks in advance for reading my question!
I've been using CNNs to classify text using Keras and TF. My data is strings "I read the news" or "I read machine learning news" and my labels are tags: Data Science, Reporter, Child...
My issues is that each text can have multiple labels attached to it. How should I construct my target such that I can capture each of those multiple outputs?
Description, Tag
"I read the news", Child
"I read the news", Reporter
"I read machine learning news", Data Science
"I read machine learning news", Reporter


texts: Observation Operator - Data Assimilation

texts:
In data assimilation, one assumes the existence of a observation operator $\mathcal{H}$ that maps the model-state vector $\bf{x_b}$ to $
\bf{y_b}$ (the model-equivalent of the observations $\bf{y_o}$) according to a reference I'm using to develop a preliminary understanding of DA.

Can someone please elaborate on the precise meaning of:

  model-equivalent of the observations $\bf{y_o}$

and the methods one can use to estimate the operator $\mathcal{H}$.


texts: Putting a predictive model into production

texts: Even after all these years of data science from 2010 to 2018, why is there no general framework for putting a predictive model into production?


texts: What kind of approach should I apply for face validation with using deep learning?

texts:
My research task is face recognition in cars with using deep learning method. Actually, in example we set an driver randomly and then the question is: Is this person driver or not?
So i created an custom dataset that includes 47 different persons's faces when they are driving cars. Every person's have 3000 face images and I used MTCNN for detect faces from images.
But I'm confused; is the problem multi-class classification or binary? I mean the result is binary (driver or not) but we'll set driver after created this model. for that I can not divide the dataset like driver and others.
When I researched I found that one-class classification approach. Deep One-Class Classification Is this help me or not, I don't get it.
What I should do?


texts: Maximum likelihood estimation vs calculating distribution parameters "manually"

texts:
I'm sorry for asking probably elementary question, but I cannot understand how estimating probability distribution parameters using maximum likelihood estimation method differs from calculating these parameters from observed data manually.
For MLE we need to know the type of probability distribution anyway so why don't we just use the known formulas for calculating the corresponding parameters from observed data?
I believe that MLE is somehow more general method but I cannot see what is the real advantage of MLE compared to getting these parameters "manually".

Thanks for explanation.

Tomas


texts: Are there any machine learning techniques to identify points on plots/ images?

texts:
I have data for each vehicle's lateral position over time and lane number as shown in these 3 plots in the image and sample data below.

> a
   Frame.ID   xcoord Lane
1       452 27.39400    3
2       453 27.38331    3
3       454 27.42999    3
4       455 27.46512    3
5       456 27.49066    3

The lateral position varies over time because a human driver does not have perfect control over vehicle's position. The lane change maneuver starts when the lateral position changes drastically and ends when the variation becomes 'normal' again. This can not be identified from the data directly. I have to manually look at each vehicle's plot to determine the start and end points of lane change maneuver in order to estimate the duration of lane change. But I have thousands of vehicles in the data set. Could you please direct me to any relevant image analysis/ machine learning algorithm that could be trained to identify these points? I work in R. Thanks in advance.


texts: Regression vs Random Forest - Combination of features

texts:
I had a discussion with a friend and we were talking about the advantages of random forest over linear regression.

At some point, my friend said that one of the advantages of the random forest over the linear regression is that it takes automatically into account the combination of features.

By this he meant that if I have a model with

Y as a target
X, W, Z as the predictors

then the random forests tests also the combinations of the features (e.g. X+W) whereas in linear regression you have to build these manually and insert them at the model.

I am quite confused, is this true?

Also if it true then is it about any kind of combination of features (e.g. X*W, X+W+Z etc) or only for some specific ones (e.g. X+W)?


texts: Identifying outliers in an unknown distribution

texts:
I have a sorted sequence of integers, e.g.

1,2,480,1000,1100

representing volumes in some categories. The task is to separate the valid data (high volumes) from the outliers (low volumes).

In the above example I expect following obvious split

1,2   |   480,1000,1100

This is of course trivial, if there is a definition of outliers, such as less than .1% of the total volume, but I'd prefer to have an
algorithm for calculating this threshold as a result.

I don't know anything about the distribution of the values, except for the outliers have low values and the regular categories have high volumes.

Intuitively, I'd say the approach should pass all possible N-1 splits of the sequence and evaluate some criterion, but I fail to
find a meaningful one (e.g. I distinctly failed comparing the WSS of the sequence with the sum of the WSS of both splits).

I'd appreciate hints to the algorithm or to transformation of this problem in other well known unsupervised problem.


texts: How to build confusion matrix , when predicted value and actual value is in sentence?

texts:
I am building some model, which predict on basis of  highest probability from history and I am assuming this is best action.
I am comparing this with real action.

Predicted : process resumed somedate2

Actual :  process resumed at somedate1

As both are similar and assumed this is correct prediction.

I have applied cosine similarity for same and doing comparison.

Is there any better way to compare same and to check accuracy ?

Any help will be appreciated.


texts: Large values in PaviaU hyperspectral dataset

texts: I am exploring the paviaU hyperspectral dataset. I had the impression that when displaying the data  I will get pixel values. However after converting it to a DataFrame I get very large values( see image below). Could anyone please explain what do these values resemble? Are they some kind of aggregation and of what?


texts: Variable input/output length for Transformer

texts:
I was reading the paper "Attention is all you need" (https://arxiv.org/pdf/1706.03762.pdf ) and came across this site
http://jalammar.github.io/illustrated-transformer/ which provided a great breakdown of the architecture of the Transformer.

Unfortunately, I was unable to find any explanation of why it works with input/output lengths that are not equal (eg. input: “je suis étudiant” and expected output: “i am a student”).

My main confusion is this. From what I understand, when we are passing the output from the encoder to the decoder (say $3 \times 10$ in this case), we do so via a Multi-Head Attention layer, which takes in 3 inputs:

A Query (from encoder), of dimension $3 \times k_1$
A Key (from encoder), of dimension $3 \times k_1$
A Value (from decoder), of dimension $L_0 \times k_1$, where $L_0$ refers to the number of words in the (masked) output sentence.

Given that the Multi-Head Attention should take in 3 matrices which have the same number of rows (or at least this is what I have understood from its architecture), how do we deal with the problem of varying output lengths?


texts: Replace human judgements with mathematical approach/theories

texts:
I would like to give a context of what I did.

1) Let's say there are two dictionaries (dict A and dict B) each containing a list of words/terms as shown below.

2) Now my task is to find matching words for dict A in dict B

3) I use an automated tool (fuzzy-matching/similarity) to do the above task and the output looks like below

4) Once I get the output as above, you can see that there are some records with match % less than 100. It is totally possible that dict B didn't have the exact matching term. It's fine.

5) So, what I do is review terms that have match % less than 50. Meaning I take those terms (that are less that are 50% match) and check for a related term in dict B again. Doing this, I am able to update the output like below. Because we know through human experience that sore throat lozenge and strepsils are related (matching is better now when compared to earlier where it was mapped to orange (totally irrelevant)). So this problem is more of a semi-automated task rather than full-blown ML task

So, now my question is (not on NLP or ML but below)

1) But how can I prove that choosing 50% as the threshold for manual review is the right one? Because this is a subjective thing/ based on individual judgment. Meaning I could have chosen 30% or 40% as well, it could have saved my time in manually reviewing terms

2) Meaning, this 50% isn't written in stone but what I am looking for is some theory/mathematical/statistical approach through which I can arrive at this threshold value rather than based on my judgment/subjective which I cannot defend/justify?

Can you people share some views/techniques on how can this be done in a systematic approach?


texts: Terminology: what is the correct word for the quantity of "association" between x value and classes

texts:
I am looking for a word for the quantity of "association" between a variable x value and a classification class.
For example, let's imagine we have two two classes $A$ and $B$ and two inputs $\mathbf{x}_A$ and $\mathbf{x}_B$ from each class respectively. The probability of membership in each class varies roughly linearly between these two inputs. I want a word to say the "association" of the input $\mathbf{x}_C = a \mathbf{x}_A + b \mathbf{x}_B$ with $A$ is determined by $a$ and $b$.
I initially used the word "correlation" without considering it, but then I realised this is not correct.
Other usage examples:

Feature $i$ has a negative "association" with class $A$
We may add vector $\mathbf{u}$ because it is "unassociated" with the output classes.
Transformation $f$ preserves "association" with the output class.
Removing noise increases output class "association".

I could say "probability of membership", but that would make certain phrases awkward. For example "Feature $i$ has a negative contribution to the probability of membership of class $A$". I would much rather say something similar to "Feature $i$ has a negative correlation with class $A$".


texts: A3C - Turning action probabilities into intensities

texts:
I'm experimenting with using an A3C network to learn to play old Atari video games. My network outputs a set of probabilities for each possible action (e.g. left, right, shoot), and I use this information to determine which action to take in the game.

However I have gotten to thinking how would one go about playing a game with non-binary actions. For example, steering a car left or right with a wheel instead of keyboard keys. I thought about simply translating probabilities into intensities (e.g. if I have values of 1.0/0.0 for left right, then make the hardest left turn possible, but make a much more gradual turn if my values are 0.6/0.4) but I'm not sure if this makes sense mathematically, or even in practice.

Is there a standard approach to doing this?


texts:
I have to tag a dataset for NER. I came across conll2002/esp. What I understand so far, in IOB2 format if I want to tag 'Alex Larson is going to Los Angeles for a job interview with Candace Patrick' it'll be like:

Alex B-PER
Larson I-PER
is O
going O
to O
Los B-LOC
Angeles I-LOC
for O
a O
job O
interview O
with O
Candace B-PER
Patrick I-PER

Am I right? What about IOB format?


texts: I am creating a specialized network with Keras where the batchsize is an important fixed variable representing the size of inputs that get compared. Due to this that makes the update that happen after every batch apply to just 1 example (of size N). Is there a way to update after X batches rather than after every batch?


texts: Can generic data sets be suitable for specific sentiment analysis

texts:
I have used the stanford movie review dataset for creating a experimentation of sentiment analysis.

Managed to create a basic application on top of Spark using the Naive bayes classification algorithm.

Steps that I did for pre-processing from the spark ML pipeline

Tokenization
Bigrams

The provided dataset above also has a testing dataset with itself which is separate of the training set. After training it I got around 97% accuracy which I believe is pretty good for Naive bayes.

Now can I use this ML model to predict for other texts such as email/chat etc., My guess is that this dataset has a large enough collection of words to perform good predictions and certain english words regardless of the business context like "I dont like this","This does not look good" is the same across different domains such as Movies/Emails/Chats etc.

I have not done the experiment since the data that I need to get hold of belongs to the customer and due to privacy restrictions we cannot access the data.

Any help/guidance would be much appreciated.


texts: What are the techniques for anomaly detection of Unsupervised learning problem

texts:
I have sufficient and properly formatted data in millions without labels.
I have to find out the anomalies.

Heard Isolation forest, Mahalanobis distance about identifying anomalies in unsupervised learning.
Are these ok to try?

Are their any other techniques we can try?

Thanks


texts: Basis expansion for regression using neural network?

texts:
I am trying to approximate a nonlinear function using a neural network. There are 3-4 input units. The network is struggling a bit to generalize the function outside the vicinity of the training data set.
I asked someone and he suggested that basis expansion might help. Can someone please provide a reference for the same, I am not able to find any. Also, he suggested "basis expansion using kernel method".


texts: How do I write a SQL (or dwSQL) query across these two different datasets?

texts:
I'm having a little fun and I'd love to know how to write a SQL or dwSQL query across these two datasets of Trumps tweets and Facebook posts. Here are the datasets:

Trump Facebook Posts
https://data.world/dave/trumps-facebook-statuses

Trump Tweets
https://data.world/dave/trump-tweets

Ideally I would like to be able to count the distinct instances that a word or phrase (like 'crooked' or 'massive' or 'the best' appeared across both datasets.

The table name for the Facebook Post dataset is TrumpFacebook and for the Trump Tweets is trump_tweets.


texts: Images matching using bf.match and drawing using cv2.drawMatches Python

texts:
I have successfully matched 2 images using bf.Match and drawn matches as well. Now, I am trying to match multiple images from 2 different directories.

For this, I am using 2 different array to store images from 2 different directories. Everything seems to be working fine and printing the array actually shows the points as well when printed.

However, cv2.drawMatches throws an error TypeError: queryDescriptors is not a numerical tuple when looping it. I am using while loop to loop the cv2.drawMatches function and iterate the stored array values.

Any help would be more than appreciable.

Below is my code that I have so far developed to achieve this:

import matplotlib.pyplot as plt
import os

sift = cv2.xfeatures2d.SIFT_create()
path = 'path_to_directory_1'
files = []
descriptors_1 = []
keypoints_1 = []
keypoints_2 = []
descriptors_2 = []
img1 = []
img2 = []

for r, d, f in os.walk(path):
    for file in f:
        if '.PNG' in file:
            files.append(os.path.join(r, file))

for f in files:
    imgtest1 = cv2.imread(f)
    imgraw1 = cv2.cvtColor(imgtest1, cv2.COLOR_BGR2RGB)
    img1.append(imgraw1)
    descriptors_1.append(sift.detectAndCompute(imgraw1,None))
    keypoints_1.append(sift.detectAndCompute(imgraw1,None))

path = 'path_to_directory_2'
files = []
for r, d, f in os.walk(path):
    for file in f:
        if '.PNG' in file:
            files.append(os.path.join(r, file))

for f in files:
    imgtest2 = cv2.imread(f)
    imgraw2 = cv2.cvtColor(imgtest2, cv2.COLOR_BGR2RGB)
    img2.append(imgraw2)
    descriptors_2.append(sift.detectAndCompute(imgraw2,None))
    keypoints_2.append(sift.detectAndCompute(imgraw2,None))

bf = cv2.BFMatcher(cv2.NORM_L1, crossCheck=True)
size = len(descriptors_1)
count = 0
while (count <= size):
  print(descriptors_1[count])
  count = count + 1
  matches = bf.match(descriptors_1[count], descriptors_2[count])
  matches = sorted(matches, key = lambda x:x.distance)
  img3 = cv2.drawMatches(img1[count], keypoints_1[count], img2[count], keypoints_2[count], matches[:50], img2[count], flags=2)
  plt.imshow(img3)
  plt.show()


texts: WGAN is too slow what are some ways to tweak for speed ups?

texts:
I have implemented a vanilla GAN which gave good results very fast but it had a lot of mode collapse issue, because of this I learned about WGAN which suppose to fix this, in fact they claim they have never encountered mode collapse problem which is great. My main issue is how slow WGAN takes to converge to good results. With a vanilla GAN by epoch 10 I was getting good looking generations, with WGAN I am still seeing noise images on epoch 50! Besides bumping up the learning rate is there anything else I can do? The network is a modified smaller DCGAN that has 1 less output layer than the original. So instead of the extra 1024 output layer mine stops at 512. I knew WGAN was going to be slow from all the research I have done but I didn't except to be this slow. Can some one point me to the right direction on how to better optimize WGAN's speed?

I have googled around but most results are just vague comment chains or agreements that there is a speed problem with no further leads on what you should do next to counter this issue.


texts: What arguments should I pass to input_shape parameter of LSTM function in Keras?

texts:
My dataset has 2944424 rows and 6 columns. I am using an LSTM in Keras to forecast taxi demand. I am having problem with the input_shape parameter of the LSTM.

It gives the error:

ValueError: Error when checking input: expected lstm_15_input to have 3 dimension


texts: Calculating Median and Mode of numerical variables for different subgroups in R

texts:
I have customer call data and I want to get the median and mode for the call success rate for different subgroups.

My variables are: Customer ID, Employment Status (Retired, Employed, Unemployed), count of calls, count of successful calls, call success rate (successful calls/number of calls)

So far I worked in Excel to get the average call reachability. Mode and median are not easy to calculate with Excel, because it can only do it for up to 255 recods I believe. I know that the summary command in R provides you with summary statistics for each variable but what if you want to get the median and mode of each subgroup. Meaning: Median and Mode of call succcess rate for Retired, Employed and Unemployed customers separately? Why I want to calculate the Median and Mode, although I have the average already? Because both are more resistant to outliers. But I am open for discussion if you think it is not necessary :)


texts: Can I deal with a missing not at random column by creating a new column? (Feature engineering)

texts:
Task: Binary classification
Example problem:
Let's say we have two feature columns A and B. A has no nulls and is a binary column if a user completed an action (=1), 0 if they didn't. For all users that completed the action, B is the resultant score. As a result, B has nulls for those that didn't complete the action (missing not at random as the nulls are dependent in B are due to A).
To deal with this missing not at random problem, is it possible to create a new variable that is equal to 1 if the user completes the action and achieves a certain score, 0 otherwise?
The column B is valuable but I'm trying to find a good way of dealing with the nulls.

A
B

1
94

0

0

1
45


texts: Training Deep Nets on an Ordinary Laptop

texts:
Would it be possible for a an amateur who is interested in getting  some "hands-on" experience in desining and training deep neural networks, to use an ordinary laptop for that purpose (no GPU), or is it hopeless to get good results in reasonable time without a powerful computer/cluster/GPU?

To be more specific, the laptop's CPU is an Intel Core i7 5500U fith generation, with 8GB RAM.

Now, since I haven't specified what problems I would like to work on, I'll frame my questions in a different way: which deep architectures would you recommend that I try to implement with my hardware, such that the following goal is achieved: Acquiring intuition and knowledge about how and when to use techniques that were introduced in the past 10 years and were essential to the uprising of deep nets (such as understanding of initialisations, drop-out, rmsprop, just to name a few).

I read about these techniques, but of course without trying them out myself I wouldn't know exactly how and when to implement these in an effective way. On the other hand, I'm afraid that if I try using a PC which isn't strong enough, then my own learning rate will be so slow that it would be meaningless to say that I've acquired any better understanding. And if I try using these techniques on shallow nets, maybe I wouldn't be building the right intuition.

I imagine the process of (my) learning as follows: I implement a neural net, let it practice for up to several hours, see what I've got, and repeat the process. If I do this once or twice a day, I would be happy if after, say, 6 months I will have gained practical knowledge which is comparable to what a professional in the field should know.


texts: How many RNN units are needed for tasks involving sequences?

texts:
I am training an RNN on the following task:
Given a sequence of thirty words, predict the next word.

Is there a benefit to having more than 30 cells (LSTM, GRU or plain RNN) in my network?
I've seen many examples online where similar networks are trained with multiple layers that each have 100 cells, but this does not make sense to me.
How does it help to have more cells than the length of the sequence? (in my case this length is 30)
I'm confused because from my understanding, each cell takes in two inputs
1. A new element of the sequence
2. The output from the previous cell So after 30 cells, there will be no new sequence elements to input into the cell. Each cell will just be processing the output of the previous cell (receiving no new info).

I am using LSTM cells for this task (however, I'm guessing the actual type of RNN cell used is irrelevant).


texts: Having and issue plotting horizontal chart

texts:
So for some weird reason I can't manage to fix the plotting issue

Any suggestions?

from sklearn.metrics import confusion_matrix

List = []
for i in range(len(model)):
  cm = confusion_matrix(Y_test, model[i].predict(X_test))

  TN = cm[0][0]
  TP = cm[1][1]
  FN = cm[1][0]
  FP = cm[0][1]

  List.append(((TP + TN) / (TP + TN + FN + FP))*100)

  print()

import numpy as np
import matplotlib.pyplot as plt

fig, ax = plt.subplots()
bars = ('Logistic', 'K Nearest Neighbor', 'K Nearest Neighbor', 'Support Vector Machine ', 'Support Vector Machine ','Support Vector Machine ','Support Vector Machine ' )
percentage = np.array([List[0], List[1], List[2], List[3],List[4],List[5],List[6]])

new_labels = [i+'  {:.2f}%'.format(j) for i, j in zip(bars, percentage)]

plt.barh(bars, percentage, color='lightskyblue', edgecolor='blue')
plt.yticks(range(len(bars)), new_labels)
# Show graphic
plt.show()


texts: Cannot predict the prices using the date parameter

texts:
#importing the libraries
import pandas as pd
import matplotlib as plt
import numpy as np

#getting the dataset
ds = pd.read_csv("data.csv")
ds['Date'] = pd.datetime.date(ds.Date,format ='%m-%d-%Y')
X = ds.iloc[:, 0].values
Y = ds.iloc[:, 1].values

#spliting the data into train and test
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,Y,test_size=0.2,random_state=0)
X_train = X_train.reshape(-1,1)

#scaling the features of the data
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
y_train = sc.fit_transform(y_train)

from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train,y_train)

#predicting the value
y_pred = regressor.predict(y_train)

#plootting the graph
plt.scatter(y_train,color ="red")
plt.plot(regressor.predict(y_train), color ="blue")
plt.title("prediction ")
plt.xlabel("date")
plt.ylabel("stock prices")
plt.show()


texts: Convert a list of lists into a Pandas Dataframe

texts:
I am trying to convert a list of lists which looks like the following into a Pandas Dataframe

[['New York Yankees ', '"Acevedo Juan"  ', 900000, ' Pitcher\n'],
['New York Yankees ', '"Anderson Jason"', 300000, ' Pitcher\n'],
['New York Yankees ', '"Clemens Roger" ', 10100000, ' Pitcher\n'],
['New York Yankees ', '"Contreras Jose"', 5500000, ' Pitcher\n']]

I am basically trying to convert each item in the array into a pandas data frame which has four columns. What would be the best approach to this as pd.Dataframe does not quite give me what I am looking for.


texts: How to feed data to multi-output Keras model from a single TFRecords file

texts:
I know how to feed data to a multi-output Keras model using numpy arrays for the training data. However, I have all my data in a single TFRecords file comprising several feature columns: an image, which is used as input to the Keras model, plus a sequence of outputs corresponding to different classification tasks: eg. an output encodes the age of the person in the image, another output encodes the gende, and so on.

From what I have seen in examples, when the output of the model is made of various heads, the model should be fed with multiple data sources, one for the input, and one for each of the ouputs.

Is there an easy way to do that when the data is all in a single TFRecords? I mean, without having to create separate TFRecords for the input and each of the ouputs?


texts: How to improve an existing machine learning classifier in python?

texts: I have a big dataset (1million x 50) to which I want to predict a particular class. I have thought of segregating the dataset in batches of 20k. And then train a classifier (lets say random forest or a basic SVM). How do I then improve that classifier by providing it with additional dataset. In other words, how do I preserve the random forests created in iteration i and use as starting model in interation i+1 to improve the model in python?


texts: I am reading pattern Recognition and machine learning by Bishop and in the chapter about probability, "noise in the observed data" is mentioned many times. I have read on the internet that noise refers to the inaccuracy while reading data but I am not sure whether it is correct. SO, what actually is noise in observed data? And what is additive noise and Gaussian noise ?