texts: Two-class classification model with multi-type input data

texts:
As a first introduction to machine learning and keras, I just finished reading Deep Learning with R by François Chollet with J.J. Allaire. I would like to extend the book's IMDB example of two-class classification to a multi-input version using functional API.

library(keras)

## Keras IMDB example

#load data
imdb <- dataset_imdb(num_words = 10000)
train_data <- imdb$train$x
train_labels <- imdb$train$y
test_data <- imdb$test$x
test_labels <- imdb$test$y

#Encoding the integer sequences into a binary matrix
vectorize_sequences <- function(sequences, dimension = 10000) {
results <- matrix(0, nrow = length(sequences), ncol = dimension)
for (i in 1:length(sequences))
results[i, sequences[[i]]] <- 1
results
}

x_train <- vectorize_sequences(train_data)
x_test <- vectorize_sequences(test_data)
y_train <- as.numeric(train_labels)
y_test <- as.numeric(test_labels)

#Defining the model
model <- keras_model_sequential() %>%
layer_dense(units = 16, activation = "relu", input_shape = c(10000)) %>%
layer_dense(units = 16, activation = "relu") %>%
layer_dense(units = 1, activation = "sigmoid")

#Compiling the model
model %>% compile(
optimizer = "rmsprop",
loss = "binary_crossentropy",
metrics = c("accuracy")
)

#Training the model
history <- model %>% fit(
partial_x_train,
partial_y_train,
epochs = 20,
batch_size = 512,
validation_data = list(x_val, y_val)
)

#Defining the model using functional API
model <- keras_model_sequential() %>%
layer_embedding(input_dim = 10000, output_dim = 8,
input_length = maxlen) %>%
layer_flatten() %>%
layer_dense(units = 1, activation = "sigmoid")

So, I would like to create a similar model where we predict class A and B based on 1 continuous and 3 categorical input data. In this dummy example, continuous1, categorical1 and categorical2 are 1D tensors while similarly to the IMDB example, categorical3 is a 2D tensor of shape (samples, indices) with length num_index=20 and are one-hot encoded.

#Dummy data
dat <- data.frame(samples=c(rep(paste0("A_",1:9800)),rep(paste0("B_",9801:10000))))
dat$label <- c(rep("A",9800),rep("B",200))
dat$continuous1 <- c(rnorm(9800, 100, 45),rnorm(200, 0, 25)) #may    normalize to unit variance
dat$continuous1[dat$continuous1<0] <- 0
dat$categorical1 <- c(rep(1:100,98),rep(1:100,2))
dat$categorical2 <- c(rep(1:98,100),rep(99:100,100))

pool_A <- factor(1:15,levels=1:20)
pool_B <- factor(16:20,levels=1:20)
dat_categorical3 <- vector("list",10000)
names(dat_categorical3) <- c(as.character(dat[dat$label=="A",]$samples),as.character(dat[dat$label=="B",]$samples))
for(i in 1:9800){
  dat_categorical3[[i]] <- (as.numeric(table(sample(pool_A, 20, replace=T))) > 0) + 0L
}
for(i in 9801:10000){
  dat_categorical3[[i]] <- (as.numeric(table(sample(pool_B, 20, replace=T))) > 0) + 0L
}

As I've never built a Keras model before, I'm a bit confused on how to set up the layers representing the different data inputs using functional API. Any suggestions and feedback are highly appreciated, thanks!


texts:
I am running a Q-learning algorithm with a finite time horizon.  Are 'optimistic initial conditions' still preferred if there is a possibility that some states will not be visited multiple times?

Wikipedia on Initial Q Values


texts: computer science student - asking for some machine learning guiding (voice cloning)

texts:
I have chosen my synopsis topic for my second last semester.
I want to make a text-to-speech program, that speaks with the voice of a game character.
I have worked with machine learning in my class, but nothing that covers audio or text-to-speech.
I know how big of a project I have taken on my shoulders, but that is just what I need.
I don't know where to start at all, since there is way too much information out there. In other programming areas, I'm usually pretty good at finding information, but I just don't know enough about ML yet to do that here (plus the keywords used online kind of contradict each other).
I'm not sure which tools I need to look at. Where should I begin?
I have 68.871 audio files, which is about 76 hours of audio in total.
The files are of different lengths, they start at around under a second and end around 49 mins.
I have not heard all the audio myself, but I'm pretty sure all of it is completely clean.
(I know that I probably can't feed any model with 68.871 files, that could all be of different lengths, so I think I have to pick out a smaller sample size, with more equal lengths)
I would like to use these files as audio samples, so a model can learn how to speak like this character.
Any push in the right direction would be deeply appreciated.


texts:
Say I had the following training set for a Naive Bayes algorithm.

   Outlook    Person    Play Golf?
   -------    ------    ----------
   Sunny      Joe       Yes
   Sunny      Mary      Yes
   Raining    Joe       Yes
   Raining    Mary      No
   Raining    Harry     Yes

If try to predict whether Harry will play golf on a sunny day (which I have no data for).
Would it be correct to exclude the person attribute and use the remaining outlook attribute to calculate the probability of this happening? Or could that potentially cause problems with a larger data set that I'm unaware of?


texts: Use a GPU to speed up neural net training in R

texts:
I'm currently training a neural net model in R and am wanting to use a GPU to speed up this process.  I've looked into this but it appears that this is unavailable to Mac users as Apple no longer uses NVIDIA GPUs.
Can anyone tell me if this is the case, and if not how I can go about utilizing a GPU?


texts: Analyzing a stream of timestamped accelerometer data to find the start and end of certain motion patterns

texts:
I'm new to machine learning, so sorry in advance if this is not a useful question.

But I'm hoping to get a few clues to start additional research.

As part of my project, I've attached an 6-axis sensor (accelerometer/gyroscope) to the end of a golf club. That sensor is streaming data to my computer, at preset intervals (e.g., every 10ms). So I'm getting 6 series of data.

The sensor stream will stream continuously, before, during, and after a "golf swing", through multiple swings (e.g., over the span of 30 minutes).

My goal is to use machine learning to be able to analyze that stream, and identify when each swing starts and ends.

I.e., while the sensor is moving randomly and unpredictably in between swings, I suspect that the particular nature of a golf swing will create an identifiable pattern that an algorithm may be able to discern.

So, for example - if you provided the algorithm with 30 minutes of streaming data, during which time the golf club was swung 30 times, the algorithm would be able to identify when each swung started and finished, and be able to analyze each one.

Does this sound feasible?

Can anyone point me in the direction of particular topics that would be useful in a project like this?

Thanks in advance.


texts: Keras Sequence Generator leads to huge memory usage when used with fit_generator

texts:
I want to use a fit_generator to stabilize the memory usage when training with very large datasets.
Q1: From my understanding, the generator puts batches in a queue which is fetched by the fit_generator function of keras to train the model on that batch. After the raining with that batch, it should release that batch from memory?

Q2: In my example, the memory increases with every batch. No memory cleaning is seen. Furthermore the memory does switch into the swap memory. afterwards the memory stays stable but the swap memory increases until the program crashes.

Q3: I am using the cpu only for testing purposes right now. Does the fit_generator can work as intended when everything has to run on the ram of the computer and everything is computed with the cpu?

Here is my code:

class DataGenerator(Sequence):
    def __init__(self, batch_size, id_list, dim, shuffle=True):
        # batch_size: batch_size at each iteration
        self.batch_size = batch_size
        self.id_list = id_list
        self.dim = dim
        self.batch_size = batch_size
        self.shuffle = shuffle
        self.on_epoch_end()

    def __len__(self):
        # Denotes the number of batches per epoch
        return int(np.floor(len(self.id_list)/self.batch_size))

    def __getitem__(self, index):
        '''''
        MAIN function:
        - index tells the number of batches that have already been processed
        :return: X and y with shape [batch_size, window_size, features]
        '''''
        X, y = self.__data_generation(self.id_list[index])
        return X, y

    @profile
    def on_epoch_end(self):
        if self.shuffle:
            random.shuffle(self.id_list)
        gc.collect()

    def __data_generation(self, fname):
        # 1. load data from specified path
        data = self.__load_data(filename=fname)
        data_x = np.squeeze(data[0, 0])
        data_y = np.squeeze(data[1, 0])
        return data_x, data_y

        # --------------------------------------- utility functions data loading

    def __load_data(self, filename):
        data = np.load(filename, allow_pickle=True)
        # load data from the filename
        return data

The generator is than put into the fit_generator function

train_gen = DataGenerator(batch_size=1,
                             id_list=train_files,
                             dim=(256, 6),
                             shuffle=True)

model.fit_generator(generator=train_gen,
                       epochs=2,
                       verbose=1,
                       use_multiprocessing=True,
                       workers=5,
                       max_queue_size=10)

I would like to know if my concept is wrong or if I misunderstood the use of the fit_generator function.


texts: Kaggle Code Competitions - Pre-trained model rules

texts: I'm new to Kaggle and I often see the rule in code competitions that pre-trained models are allowed but GPU run-time should be less than 9 hours. Does this mean I can train my model on the competition data on my own GPU outside Kaggle for more than 9 hours and use this as a pre-trained model? Would this not be violating the 9 hour GPU run-time rule? What are the exact guidelines for using pre-trained models? Thanks!


texts: for a multi-layer neural network is stochastic gradient descent(SGD) guaranteed to reach a global optimum?


texts: Training a convoltion neural network for localization

texts:
I am new into convolutional neural networks and I am trying to build a cnn for localization of object on tensorflow. Currently I am trying to replicate a cnn similar to that of Alexnet and then want to train it to add boundary boxes around objects.

I am however, unable to figure out how to train the network for boundary boxes. For the purpose of validation do I have to feed the boundary values during the training for calculating loss?

P.S. I want to observe the effects of changing cnn structure on loss and training times etc. Therefore, I believe I cannot use a trained neural network.


texts: Does orange transfrom categorial variables into dummy variables when using hierarchical clustering?

texts: I am using Orange to cluster a large amount of data consisting of three attributes. Each attribute only contains categorical unordered variables. My question is does Orange transform categorical variables into dummy variables when using Distances and Hierarchical clustering widgets on such a data set?


texts: Cluster Analysis - Comparing Same Individuals Clustered Across Different Datasets with different features

texts:
I have an interesting problem, and I think my Google is failing me since I can't find the same problem anywhere.

I have a set of individuals. I have 4 different datasets, with (some) to (all) of those individuals in any given set.  So if I had 10 people, n=10, then maybe in DS1 I have all 10, DS2 has 7 of them, DS3 8, and DS4 5.

Each dataset DS# has it's own features.  So maybe DS1 has 100 features, DS2 has 10, DS3 has 30, and DS4 also has 30.

I want to cluster the individuals who exist in any given dataset.  So, if I did 3 clusters total, I'd have 3 clusters for DS1, 3 clusters for DS2, ... , 3 clusters for DS4.

I want a metric that captures how similar the results are.  I'm doing this part for exploratory purposes.  I'm also considering later incorporating this into a similarity metric for optimization purposes, but that's a further-out idea.

I however cannot find existing work that does something like this. Is there such work someone can point me to?

Much appreciated.


texts: Feature importance with scikit-learn Random Forest shows very high Standard Deviation

texts:
I am using scikit-learn Random Forest Classifier and I want to plot the feature importance such as in this example.

However my result is completely different, in the sense that feature importance standard deviation is almost always bigger than feature importance itself (see attached image).

Is it possible to have such kind of behaviour, or am I doing some mistakes when plotting it?

My code is the following:

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier

clf = RandomForestClassifier()
clf.fit(predictors.values, outcome.values.ravel())

importance = clf.feature_importances_
importance = pd.DataFrame(importance, index=predictors.columns,
                          columns=["Importance"])

importance["Std"] = np.std([tree.feature_importances_
                            for tree in clf.estimators_], axis=0)

x = range(importance.shape[0])
y = importance.ix[:, 0]
yerr = importance.ix[:, 1]

plt.bar(x, y, yerr=yerr, align="center")

plt.show()


texts: How to use LeakyRelu as activation function in sequence DNN in keras?When it perfoms better than Relu?

texts:
How do you use LeakyRelu as an activation function in sequence DNN in keras?
If I want to write something similar to:

 model = Sequential()
    model.add(Dense(90, activation='LeakyRelu'))

What is the solution? Put LeakyRelu similar to Relu?

Second question is: what are the best general setting for tuning the parameters of LeakyRelu? When is its performance significantly better than Relu?


texts: Best forecast model for insurance policies volumes

texts:
I am new in forecasting and I am studying a dataset from an insurance company that contains the volume on a monthly basis of new policies, renewals & cancellations. New policies of a given month are renewed in intervals (3 months, 6 months, 12 months) but could be canceled as well at any time. For instance, new policies of January with 3-months duration are renewed after 3 months in April.
I would like some help in what direction to study in order to build a forecast model that will predict the amount of renewals (R) and cancellations (C) of a given month based on the input of new policies (N) of previous months.
Thank you


texts:
I understand how to build and train a neural network like shown below, as well as those low-level features/filters. I wonder what are those high-level features: how exactly do you obtain them from a trained neural network? (Are those like the "eigenfaces"?)

Note: the image is by NVIDIA, and I don't know the specifics of the classification problem here. If needed, suppose the network is trained to distinguish human from cat.


texts: Intuition behind using the inverse of a Hessian matrix for automatically estimating the learning rate (aggression parameter) in gradient descent.

texts:
I am reviewing some course material where the lecturer suggests that instead of guessing the learning rate parameter in gradient descent implementation, one could use the inverse of the Hessian multiplied by the negative of the Jacobian, to determine the step-size.

Any help with the intuition behind using the inverse of the Hessian would be much appreciated.


texts: handling missing values for LightGBM model

texts:
I have read that LightGBM handles missing values defaultly. And there certain parameters to change the consideration of missing values like zero_as_missing etc..,
I have seen some people using negative values (-1, -999) in the place missing values.
So, my question is, which is better 1) leaving it to model to handle or 2) manually replacing the values with negative values?
Thanks in advance


texts: What does the oob decision function mean in random forest, how get class predictions from it, and calculating oob for unbalanced samples

texts:
I am interested in finding the OOB score for random forest using sklearn, when it is used for a binary classification task, and there are unbalanced samples. What does the oob decision function mean in random forest, and how get class predictions from it?
I read RandomForestClassifier OOB scoring method but am still not clear. Does the oob decision function provide class probabilities, and if so, do I get the class predictions by taking whichever number is higher (e.g. by doing something like pred_train = np.argmax(forest.oob_decision_function_,axis=1))?
Since my classes are unbalanced, would it be correct to say I can't used sklearn's default OOB score here, and I should do the above to get some kind of F1 score from the OOB predictions, to get a better estimate of my random forest's error?


texts: Understanding how convolutional layers work

texts:
After working with a CNN using Keras and the Mnist dataset for the well-know hand written digit recognition problem, I came up with some questions about how the convolutional layer work. I can understand what the convolution process is.

My first question is: What are the filters? I can understand their purpose. They are used to map edges, shapes, etc. on an image. But how are they getting initialized? Do they have random initial value or there are standard image filters that are getting used? If they are getting initialized with random value then the values should get changed on the training process of the network. If that's the case then a new question is created, how does someone backpropagate the filter of the convolutional layer? What is the algorithm behind this process?
Secondly, I have noticed that I can add an activation function to the convolutional layer in Keras. Is the entire matrix of the output getting passed through the activation function? How does the usage of an activation function changes the learning process of the convolutional layer?
Last but not least, does a convolutional layer have weight and biases like a dense layer? Do we multiply the output matrix after the convolution process with a weight matrix and add some biases before passing it through the activation function? If that's true, then do we follow the same process as we do with the dense layers to train these weights and biases?


texts: find value which occurred more times in a group from data frame column R

texts:
I have a data frame with latitude and longitude of a particular place. one place can have multiple lat, longs and those lat, long values can be same or different. I need to find the correct lat, long based on no.of occurances of lat, long for particular place.
example input data frame
df:

place    lat      long
hsy      7.343    32.849
hsy      7.343    32.122
hsy      7.567    31.567
hsy      6.934    32.122
bls      2.67     6.2
bls      3.345    5.9
bls      2.987    6.321
bls      2.987    4.56
bls      1.876    6.321
bls      2.987    6.321

expected output data frame
    place    lat      long
    hsy      7.343    32.122
    bls      2.987    6.321

expected output for lat is which ever value is occurred more times among its values for particular place and the output for 'long' column value also which ever value is occurred more times among its values for particular place.
tried below code using across, but across is not working. getting Error: 'across' is not an exported object from 'namespace:dplyr' or could not find function "across" error. any alternative for across function?
df %>%
  group_by(place) %>%
  summarise(across(c(lat, long),~names(sort(table(.x), decreasing = TRUE))[1]))


texts: What should be the value of batch_size in fit() method when using sgd (Stochastic Gradient Descent) as the optimizer?

texts:
I am confused about the batch size of this model. I have used sgd i.e., Stochastic Gradient Descent as the optimizer (see the code). I am aware that in sgd, a single random instance from the training set is used to compute the gradient at each step. So, according to it, the batch_size should be equal to 1. Now, in the tf.keras.Sequential.fit() documentation it says:

  If unspecified, batch_size will default to 32.

So, do I have to manually set the batch_size equal to 1? It is because the default value, 32 will make it a Mini-batch Gradient Descent.

    import tensorflow as tf
    from tensorflow import keras

    fashion_mnist = keras.datasets.fashion_mnist
    (X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()

    X_valid, X_train = X_train_full[:5000]/255.0, X_train_full[5000:]/255.0
    y_valid, y_train = y_train_full[:5000], y_train_full[5000:]

    model = keras.models.Sequential()

    model.add(keras.layers.InputLayer(input_shape = [28, 28]))
    model.add(keras.layers.Flatten())
    model.add(keras.layers.Dense(300, activation = "relu"))
    model.add(keras.layers.Dense(100, activation = "relu"))
    model.add(keras.layers.Dense(10, activation = "softmax"))

    model.compile(loss = "sparse_categorical_crossentropy", optimizer = "sgd", metrics = ["accuracy"])

    history = model.fit(X_train, y_train, epochs = 30, validation_data = (X_valid, y_valid))


texts: What is the difference between mongodb and tuple space storages

texts:
There are some storage space type like GigaSpaces. that inspired from tuple space.
Would you say mongodb can act like a tuple space or not?
What is the difference between mongo and tuple space?


texts: How neural style transfer work in pytorch?

texts:
I am using this pytorch script to learn and understand neural style transfer. I  understood most part of the code but having some hard time understanding some parts of the code.

In line 15 Its not clear to me how model_activations work. I made a sample style tensor of the shape style.shape -> torch.Size([3, 300, 374]) and tried this sample code first without layers dict.

x = style
x = x.unsqueeze(0)
for name,layer in model._modules.items():
    x = layer(x)
    print(x.shape)

Output:

torch.Size([1, 64, 300, 374])
torch.Size([1, 64, 300, 374])
torch.Size([1, 64, 300, 374])
torch.Size([1, 64, 300, 374])
torch.Size([1, 64, 150, 187])
torch.Size([1, 128, 150, 187])
torch.Size([1, 128, 150, 187])
torch.Size([1, 128, 150, 187])
torch.Size([1, 128, 150, 187])
torch.Size([1, 128, 75, 93])
torch.Size([1, 256, 75, 93])
torch.Size([1, 256, 75, 93])
torch.Size([1, 256, 75, 93])
torch.Size([1, 256, 75, 93])
torch.Size([1, 256, 75, 93])
torch.Size([1, 256, 75, 93])
torch.Size([1, 256, 75, 93])
torch.Size([1, 256, 75, 93])
torch.Size([1, 256, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 18, 23])
torch.Size([1, 512, 9, 11])

and then with layers

layers = {
    '0' : 'conv1_1',
    '5' : 'conv2_1',
    '10': 'conv3_1',
    '19': 'conv4_1',
    '21': 'conv4_2',
    '28': 'conv5_1'
    }

x = style
x = x.unsqueeze(0)
for name,layer in model._modules.items():
    if name in layers:
      x = layer(x)
      print(x.shape)

Output:
torch.Size([1, 64, 300, 374])
torch.Size([1, 128, 300, 374])
torch.Size([1, 256, 300, 374])
torch.Size([1, 512, 300, 374])
torch.Size([1, 512, 300, 374])
torch.Size([1, 512, 300, 374])

My question is how the second method maintained the height and widght of style tensor (300, 374)?

The second confusion is in the line 91 where optimizer was set as optimizer = torch.optim.Adam([target],lr=0.007). In most pytorch tutorials I have seen them doing some thing like this optimizer = torch.optim.Adam(model.parameters(), lr = 0.01)

Why is the optimizer initialization in neural style transfer different from other neural network tutorials?
What is reason behind this optimizer = torch.optim.Adam([target],lr=0.007) >


texts: How to count categorical values including zero occurrence?

texts:
I want to count number of code by month.
This is my example dataframe.

        id    month  code
0     sally    0  s_A
1     sally    0    s_B
2     sally    0   s_C
3     sally    0   s_D
4     sally    0    s_E
5     sally    0   s_A
6     sally    0    s_A
7     sally    0   s_B
8     sally    0   s_C
9     sally    0   s_A

I transformed to this Series using count().

df.groupby(['id', 'code', 'month']).m.count()

id      code   month  count
sally  s_A      0    12
                1    10
                2     3
                7    15

But, I want to include zero occurrence, like this.

id      code   month  count
sally  s_A      0    12
                1    10
                2     3
                3    0
                4    0
                5    0
                6    0
                7    15
                8    0
                9    0
                10   0
                11   0


texts:
I have a dataframe:

df = pd.DataFrame({'docs': ['gamma alfa beta beta epsilon', 'beta gamma eta',], 'labels': ['alfa alfa beta', 'gamma fi']})

I use count vectorizer:

import numpy as np
import pandas as pd
from itertools import chain
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer

vocab_docs = set(chain(*[i.split() for i in df['docs'].unique()]))
cv_docs = CountVectorizer(vocabulary=vocab_docs)
cv_docs_s = cv_docs.fit_transform(df['docs'])

I do TFIDF:

tfidf_docs = TfidfVectorizer(vocabulary=vocab_docs)
tfidf_docs_s = tfidf_docs.fit_transform(df['docs'])
# tfidf docs
tfidf_docs_s = tfidf_docs_s.todense()

but I see that the results are different:

test = np.multiply(cv_docs_s.todense(), tfidf_docs.idf_)

test != tfidf_docs_s

Why is the result of CountVectorizer * TfidfVectorizer.idf_ different from TfidfVectorizer.fit_transform()?


texts: Load data from multiple dataframes containing both path and labels in keras

texts:
I am aware that there exists a function in keras.preprocessing.image.ImageDataGenerator called flow_from_dataframe. But this function assumes that we have one dataframe containing all the paths to the images and labels associated with it. But what if we have multiple dataframes and we want to use these dataframes on a per-epoch basis so that in each epoch we can load a different dataframe and the labels and images associated with it?
Thanks in advance.


texts: Should I normalize data to be similar to Normal distribution?

texts:
As I understood, when training a neural network, it is preferable to have data with expectation of 0 and std. of 1.

Now if I have a feature with a Ratio distribution i.e., where median and expectation can differ much. If I apply Gaussian normalization - to subtract expectation and divide by std, I will get a distribution with again median different to expectation.

Should I be bothered by this anyway? I can easily move them to match by applying Log to it but I wanna know should I waste time on such details?


texts:
I have a large amount of Data where I have to count meassurments per one ID. What I already did was creating a Data Frame over all Files and I omited the NAs. This part works properly. I was wondering if the nrow-function is the right function to solve this but I figured out that this will not lead me to the target as it returns a single number as output.

What I am looking for is if you have entries like that:

1155 2010-05-02  2.7200    1
1156 2010-05-05  2.6000    3
1157 2010-05-08  2.6700    1
1158 2010-05-11  3.5700    2

That I get a list:

ID          Number of observations
1           2
2           1
3           1


texts: Confusion about Keras' skipgram and sampling table utilities

texts:
I'm fairly new to ML, so as a learning exercise to get familiar with Keras I'm trying to learn some word2vec style embeddings from a dataset. I'm confused about the behavior of the skipgram utility, in particular the sampling table argument that's supposed to be populated from make_sampling_table. I understand the idea behind it, but I'm seeing some strange outputs.

As a toy example, let's pretend that we have the following parameters:

vocab_size = 3
sequence = [0,1,0,1,1,0]

So a small vocabulary with 3 words in it, only 2 of which show up in the sequence. Generating the skipgrams & labels works fine without a sampling table at all:

pairs, labels = skipgrams(sequence, vocab_size)
print("pairs: " + str(pairs) + " labels: " + str(labels))

OUTPUT: pairs: [[1, 1], [1, 2], [1, 1], [1, 2], [1, 1], [1, 1], [1, 1], [1, 2], [1, 1], [1, 1], [1, 1], [1, 1]] labels: [0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1]

But when I try to pass in a sampling table so as to not oversample the common words, I get no outputs at all:

pairs, labels = skipgrams(sequence, vocab_size, sampling_table=make_sampling_table(vocab_size))
print("pairs: " + str(pairs) + " labels: " + str(labels))

OUTPUT: pairs: [] labels: []

My question is: what's going on here? Why does the act of passing a sampling table into the skipgrams method make it produce no output?

Thanks for the help!


texts: Extracting specific information from multiple unstructured website

texts:
I want to extract specific information e.g the names of all professor teaching subject xyz from multiple college faculty directory. Each website has different structure and the content is not uniform (and content not always as sentences). Is there any better approach apart from writing custom web scraper/parser?

I tired using Named Entity Recognition and Part of speech tagging approach, but not sure if it helps in this case (or is there something about NER I am missing?)


texts: Adjusting weights in an convolutional neural network

texts:
I'm trying to implement a convolutional neural network at the moment. A simple feedforward network is not the problem but I'm having some trouble with the weight adjustment in the conv layer.

Lets assume I have four layers. Input, convolution, hidden and output.

src:http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/

In the picture above we just see the input and the convolution layer.
The deltas of the convolution layer are calculated as in a normal feedforward network.
But how do I update the weights/filtermatrix between input and convolutionlayer?


texts: What information can be drawn from the area between the warping function and the diagonal path in DTW?

texts:
I'm using the R package dtw: Dynamic Time Warping Algorithms to study the similarity of two time series. This package provides also the possibility to estimate the area between the warping function and the diagonal (no-warping) path, in unit steps (function warpArea). I found an interesting application of this quantity in the following paper Latency as a region contrast: Measuring ERP latency differences with Dynamic Time Warping: a positive distance, which results from the warping path being below the diagonal, indicates that the reference time series, used for alignment, precedes the query time series, while a negative indicates that the reference time series follows the query.

What other information can be drawn from the area between the warping function and the diagonal path?


texts: Estimating and/or determining the amount of data sufficient enough to train a model

texts:
To train a nice supervised algorithm (for instance, a dependency parser, a parts-of-speech tagger or NER) data is essential, but how many samples are necessary or enough? From what kind of perspectives can we try to estimate or/and determine how much data we should annotate? Or what kind of techniques can we use to predict the amount before we get the results of the first attempt?
I know that words can be separated into two types: closed class type and open class type. Should we consider that?
If such methods do not exist, how can we know if our model (mostly high variance) suffers from data shortage (knowing that data is never enough) or other reasons (like poor structure or bad training process or bad hyperparameters, etc.) after we get the results of our first version of the model and think the results are not satisfactory?


texts:
I am trying to build a RNN in Keras.
 I am inputting an array of 300K values.
 I have 4 independent variables (W,X,Y,Z) And 1 dependent variable f(W,X,Y,Z).

The array is then split 270K for training and 30K for validation.

When I try to put my data into the network, it says "expected ndim=3, found ndim=4".

My model looks like this

model=Sequential()
model.add(LSTM(4, input_shape=(270000,4)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer ='adam')
model.fit(X_tr,Y_tr, validation_data=(X_val, Y_val),
          epochs=10, batch_size=30000, verbose=1)


texts: What is generative and discriminative model? How are they used in Natural Language Processing?

texts: This question asks about generative vs. discriminative algorithm, but can someone give an example of the difference between these forms when applied to Natural Language Processing? How are generative and discriminative models used in NLP?


texts: Feature engineering decrease my cross validation

texts:
I'm currently working on a fraud detection data set. I'm evaluating my training data with a 10-skfold roc auc and an estimator of default param LightGBM. But, the problem is every time I try to create a new column by calculating a ratio of 2 column, my CV always drop even though the new column is theoretically will highlight a difference between a fraud or not.

The other problem is when I find 2 good (increasing CV) separate new feature by feature engineering when evaluating them independently, my CV actually decrease when I combine both of the feature on the same evaluation.

Is there anything wrong with my way of feature engineering? Right now my step is:
1. Create a new column based on other column
2. Evaluate CV with 10-skfold default param
3. If the CV is increased (relative to original data) then it's good feature else no.

Any help is appreciated.


texts: Train MLP Neural Network on time series data?

texts:
Newbie question here but I was curious to ask if an MLP Neural type network can be trained on time series data?
The dataset that I have is an electricity type data set from a building power meter and I can find I can train a decent NN model with including a lot of weather data and also a lot of one hot encoding dummy variables for time-of-week. (day, hour, month number, etc.)
I am experimenting in Python with the Tensorflow Keras library and I know the default during the training process randomly shuffles the data. Is this a No-No for a time series type problem where the random shuffle will take out the seasonality from the data? (stationary/non-stationary) The results shuffling the data really aren't that bad at a glance but not-randomly shuffling the data the results for MLP NN are poor, like the model doesn't train well.
I know some other times series forecast methods can include ARIMA, LSTM, etc. but I was curious to inquire if MLP can be used for these purposes too? What I ultimately need is a short term forecast method that can incorporate hourly weather forecast (from a web API) to forecast future hourly building electricity. Any tips greatly appreciated.
Thanks


texts: Is there a rule of thumb for a sufficient number of trials for hyperparameter search

texts:
I am implementing a quite complicated Bayesian hyperparameter search in hyperopt library on a CNN.
Is there a rule of thumb for a "sufficient" number of trials? Perhaps based on the number of hyperparameters that constitute the search space?


texts: Embedding variable length "multi-hot-encoded" features

texts:
How can I implement an embedding layer in Keras that takes in an input that could have a variable length? For instance, if the vocabulary was 10-long I could have inputs like:

[1]
[2,4,5]
[7,3]

But Keras embedding does not accept variable length inputs. I guess it is possible for me to zero-pad so that it looks like:

[1,0,0]
[2,4,5]
[7,3,0]

But this would make some of the inputs quite long (in my particular case, up to length ~500 vectors in a vocab of maybe ~10000). If only one data example has length 500 it forces even ones of length 1 to be zero padded to 500.

Is there a better way to handle variable length inputs?


texts: Is GLM a statistical or machine learning model?

texts: I thought that generalized linear model (GLM) would be considered a statistical model, but a friend told me that some papers classify it as a machine learning technique. Which one is true (or more precise)? Any explanation would be appreciated.


texts:
I made Alexnet NN but it had brought out an error

input : image of 32*32*3

output : 10 classes

below is my Alexnet code

#alexnet
model_3=Sequential()
#Add the Dense layers along with activation and batch normalization
model_3.add(InputLayer(input_shape=(32,32,3)))
# 1st Convolutional Layer
model_3.add(Conv2D(filters=96, kernel_size=(11,11), strides=(4,4), padding='valid'))
model_3.add(Activation('relu'))
# Max Pooling
model_3.add(MaxPool2D(pool_size=(2,2), strides=(2,2), padding='valid'))

# 2nd Convolutional Layer
model_3.add(Conv2D(filters=256, kernel_size=(11,11), strides=(1,1), padding='valid'))
model_3.add(Activation('relu'))
# Max Pooling
model_3.add(MaxPool2D(pool_size=(2,2), strides=(2,2), padding='valid'))

# 3rd Convolutional Layer
model_3.add(Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding='valid'))
model_3.add(Activation('relu'))

# 4th Convolutional Layer
model_3.add(Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding='valid'))
model_3.add(Activation('relu'))

# 5th Convolutional Layer
model_3.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), padding='valid'))
model_3.add(Activation('relu'))
# Max Pooling
model_3.add(MaxPool2D(pool_size=(2,2), strides=(2,2), padding='valid'))

# Passing it to a Fully Connected layer
model_3.add(Flatten())
# 1st Fully Connected Layer
model_3.add(Dense(4096, input_shape=(224*224*3,)))
model_3.add(Activation('relu'))
# Add Dropout to prevent overfitting
model_3.add(Dropout(0.4))

# 2nd Fully Connected Layer
model_3.add(Dense(4096))
model_3.add(Activation('relu'))
# Add Dropout
model_3.add(Dropout(0.4))

# 3rd Fully Connected Layer
model_3.add(Dense(1000))
model_3.add(Activation('relu'))
# Add Dropout
model_3.add(Dropout(0.4))

# Output Layer
model_3.add(Dense(500))
model_3.add(Activation('relu'))
model_3.add(Dropout(0.2))
model_3.add(Dense(10))
model_3.add(Activation('softmax'))

the error message say

Negative dimension size caused by subtracting 11 from 3 for 'conv2d_22/convolution' (op: 'Conv2D') with input shapes: [?,3,3,96], [11,11,96,256].

it may concerned to layer size

what is the error of the above code and how to resolve it?


texts:
Factorization machines are a great way to model interactions between different features, in a way that is more efficient than full-blown quadratic interactions. There exist efficient implementations like libFM and Vowpal Wabbit's --lrq option.

My question is about higher order interactions: the FM paper mentions that it is "easy" to extend it to higher orders (Tensor FM?), see. Eq. (5) in the paper. Are there any good implementations of 3-way factorization machines?

[moved from cross-validated]


texts: Algorithms to find average distance between data maxima

texts:
I'm having trouble figuring out a way to analyze some simple data. When graphed, the data I have make a somewhat sinusoidal curve. What I want to do is to find the x-values of the maximum peaks of the sinusoidal curve. I then want to subtract each of these x-values from the last peak found and average these differences to obtain an average distance between peaks. Is there an easy way to do this with Excel, Mathematica, or MatLab (the programs I have available to me?).

Thanks in advance!!!


texts: How to prove that a function is sigmoidal

texts:
Given a function, how to prove that it is sigmoidal in nature. So far, my approach has been to verify if the properties of sigmoidal functions hold:

1)That it is monotonic

2)That it is constrained by a pair of horizontal asymptotes

3)That it has a first derivative that is "bell" shaped

4)That it is convex for values less than 0 and concave for values more than 0

5)That a Sigmoid function and its affine compositions can posses multiple optima.

For example this is one of the functions that I am trying to verify:

Trying to prove all the properties seems like time consuming especially in exam scenarios. I suppose there is a better approach that exists and I should follow. Please suggest me what other approach could be used.

Thank You.


texts: Survey papers on Natural Language Understanding

texts: What are some good survey or review papers on the state of the art of machine learning approaches to grammatical/syntactic structure of natural languages?


texts: Gradient boosting how can accuracy increase when we lower the depth of tree?

texts:
What I don't understand about gradient boost is, doesn't lowering height of the tree means we use fewer features in our model?

From my model I get the highest accuracy when the depth is one. Meaning there is just root node at my trees, and uses one feature. How can a model that uses one feature gives such accuracy?


texts: Use trained non linear regression model to identify variables that maximize the predicted value

texts:
I trained a non-linear regression model with 23 features. I tried to make sure the model doesn't overfit with ~0.6 r squared on validation data and with 0.75 correlation coefficient between the actual and model-predicted target values.
I would like to know if doing the following would make sense.

Use the model to identify values of the 23 features that would maximize the predicted value:

For example, consider a model to predict house price using: avg_area_price and sqfeet then identify value of avg_are_price and sqfeet that would result in the maximum house price. Note that we may not have observed this in the data but the regression model captured the input feature to output relationship so it could identify this.
I tried to look into it and found that linear programming could help doing this for linear regression. https://stats.stackexchange.com/questions/475639/can-we-use-linear-regression-to-define-the-objective-function-in-linear-programm
I did not find anything for finding the feature values that leads to the maximum predicted value for non-linear regression model.
Can anyone help with how to do this when we have non-linear regression model?


texts: Multioutput regression with MLPRegressor - Does it work?

texts:
I apologize if this has been addressed before, but I did not find a solution for this issue.

I am currently trying to solve a problem with four dependent variables and four independent variables, all non-categorial. They seem to have non-linear dependence and multicolinearity - standard multivariate regression yielded reasonable results, but I am trying to compare them with results from Random Forest and Neural Networks.

According to this link, it should be possible to get multi-output with the MLPRegressor. However, when I try to apply the .fit method, I get an error message:

fit() missing 1 required positional argument: 'y'

Here is the code that I am using:

# Data Preprocessing - Importing the Libraries
import pandas as pd

# Importing the dataset
dataset = pd.read_csv('Rock_Properties_4T.csv')

y = dataset.iloc[:, 4:8].values
X = dataset.iloc[:, 0:4].values

# ANN
from sklearn.neural_network import MLPRegressor
regressor_ANN = MLPRegressor(tol = 1e-16)
MLPRegressor.fit(X, y)

There you go... Any idea of what could be the problem?


texts: Multi layer back propagation Neural network for classification

texts: Can someone explain me, how to classify a data like MNIST with MLBP-Neural network if I make more than one output (e.g 8), I mean if I just use one output I can easily classify the data, but if I use more than one, which output should I choose ?


texts:
What Python algorithms for fitting a curve for a 3D elliptic point cloud to find about its "curvature"?

Particularly it's a point cloud of a tree and I want to measure the curvature of its top.


texts: Right choice of accuracy metric or loss function

texts:
I am developing a predictive model in sports vertical. As the game will progress, my model will predict the winning probability of each team playing.

The problem I am facing is to what metric would be appropriate to quantify accuracy for the model.

What I have in mind to quantify accuracy is as follows :

Consider n matches for which my model predicts the winning probability of Team A at time instance t1, t2 ... tn. For each match, I will quantify the loss as the hinge loss at that time. i.e. if probability greater than threshold and team_A actually won, then accurate.

As the game progresses, model gets new info and I can expect the model performance to improve, so I thought about giving weights to loss function acc. to Time of prediction also. i.e. if model predicts wrongly at later point of the game then it should be penalized heavily.

But I think it can be better. Any leads on the apt. loss function for this is highly appreciated.

Hoping to learn something great from the answers.


texts:
I read about Recursive Neural Networks that they can convert Documents to distributed word representation.

In the context of new article recommendation, I am thinking to use this model to convert all news articles to vectors and then recommend to a  particular user, news articles similar to ones he browsed.

In vector space this will boil down to finding 'similar' vectors to a given vector(user's news read).

How likely is that this model will work well in practice? Any comments and/or suggestions?


texts: How to convert regression into classification?

texts: So I have a regression problem with bunch of features X, and labels in the amount (price $). How can I convert it to classification problem? I have read about convert label from continuous to categorical possibly thesholding at some points for instance 0-50 (0 class) 51-100 (1 class) and so on till 500 maybe (here thresh-holding to 50). This approach is ineffective intuitively i.e no data lay 51-100. Is there any way to mitigate this problem or if there is any effective way to do this? Like in clustering we can use density estimation can we do that here?


texts: Sales Prediction for Fashion Retail Data

texts:
I am working on project related to prediction of sales quantity  based on previous data.
Data consists of attributes like  .MRP, Size , Discount,Color,Fabric type e.t.c
I am  having both sales quantity and Sock on hand in data .Currently I am trying to predict sales quantity using Random forest Algorithm(in R).

I did some basic feature transformations like one hotted categorical variables and performed Normalization on data and training the model.
   Here I am having some issues in data.I performed some manipulations listed below

I am having two different  datasets  one contains sales records and other has stock on hand.
we are matching and aggregating  the records week wise. In some cases stock on hand is not available but sales happened .For those records making sales quantity as stock on hand.
For unsold records we are not having any discount .To keep discount we are taking average of discounts over the same period for particular item from other store (Suggestion by client).

Some of the  records are having negative sales quantity after aggregation. we are making those negative sales to zero .
To train the model I am having sales quantity "0"(Zero) for unsold items.In data unsold items are more than sold items (like if sold items are 20  unsold are around  300)

I am not getting any considerable results .I tried the same thing with **Neural Network **  also in R.

I am beginner in predictive modelling.This is my first project .Please  correct me if anything wrong in my approach


texts: How to handle NaNs for ratio feature for binary classifier?

texts:
I'm creating a churn model and would like to create a ratio (# customers / total transaction) for each merchant. About 70% of the data are NaNs (zero/zero).

I was wondering what I should impute for the 70% of the NaNs. I have other features and I don't like to delete the 70% of the data.

But if I impute 0s, the distribution would probably become different from ground truth since lower number means large transaction volume with a few customer. If I impute mean though, it'll also be different since the element (zero / zero) has no actions fundamentally.

I was about to impute -1 to distinguish the NaNs with non-NaNs. Would that make sense for a feature of binary classifier?

Thanks!


texts: Determining extreme outliers from boxplot (by eye)

texts:
I have a normalised dataset with range 0,1. I have created boxplots for every feature in  the dataset and need to identify which features have extreme outliers (by looking at their boxplots).
However, because of the look of my boxplots I am struggling to identify which would be extreme outliers or mild (or even have any at all). You can't see clearly the whiskers etc of the boxplots (due to the range of 0-1) but can see some outliers.
I'm hoping someone could help me identify if these boxplots (1, 2 and 3) in the attached image have extreme outliers or not?
(im particularly stuck with the 3rd one - because you can still see a circle)
Thanks in advance!


texts: When would you choose certain CNN pretrained models?

texts:
I'm totally new to the field of deep learning so I'm just trying to get a sense of some of the decisions made, especially with convolutional neural networks. After reading a few blogs/articles about the different pretrained CNN architectures, I'm wondering:

When would you choose a certain architecture over another? I've seen a lot of examples using VGG-16 and I'm curious why this one is the go-to when it seems like such a large (and slow?) way to train? Are there certain examples of applications where one architecture would be better than another?


texts: Statistically Robust Distance Measure/Metric for comparing more than two network data series

texts:
I have about 30 lists of unequal length (some of which are triplicates of the data), corresponding to metrics relating to nodes of different graphs. I want to compare their similarity using a distance metric, but was unsure which method I can use given the data lists are of unequal length. I was exploring using dynamic time warping, but was wondering if there is any other more basic method.

For example, I was a considering creating histograms with same bin edges and number of bins for each list and using a distance metric on the frequency, but I am not sure how to go about this using python, or if there's a function/package that does this already. Is this even a "good" way of doing it?

I'm also interested in finding a way to measure the statistical significance of the distance measures between the different graphs.

This is a lot in one question, I am new to this and appreciate any help.  Thank you in advance!


texts:
I came from a software development background and we have separate servers of the same database (dev, test, prod). The reason for this is because we develop our apps against the dev DB, run tests against the Test DB, and prod is prod. This is so we create a clear separation and won't bring down prod trying to build our app.

Do you guys train your models the same way? Have 3 environments of the same database and as your model goes from dev to test to prod, it trains against the corresponding environment?

Example:

Data scientist plays around with a 3 different algos for classification. Creates 3 models (A, B, C) using dev env's database.
Data scientist evaluates 3 models and selects Model A after testing/validation.
Data scientist deploys the code to TESTING/STAGING env (same hyperparameters). This time, however, the model trains using TESTING/STAGING env's databases. A version of Model A is created using data from the TESTING/STAGING databases.
Data scientist deploys the code to PROD env (same hyperparameters). This time, however, the model trains using PROD env's databases. A version of Model A is created using data from the PROD databases.


texts: How can spatiotemporal population data be used for modeling migration?

texts: I have a dataset that contains the population of butterflies(5 species) for 15 years for different locations. I want to model it against the climate index collected for same time period and location. The objective is to find how early the migration species migration starts before the onset of a dry period? My idea was to understand all the spatial locations around the dry place and see if the populations have changed over time in the neighbourhood which can indicate a migration, thereby can deduce the time the butterfly realizes before the dry season onset. What type of technique can be used to predict the migration pattern? Does this come under graph theory? If so can someone guide me in a right research direction, what algorithm has to be looked at?


texts: Is the LSTM share the same model parameter in each block?

texts:
I learn the LSTM recently, and a little bit confuse about the model parameters about LSTM,

The follow is the LSTM structure

And it is equation as (I slightly ignore the bias in each equation):

$$i_t = \sigma(x_tU^i + h_{t-1}W^i)$$
$$f_t = \sigma(x_tU^f + h_{t-1}W^f)$$
$$o_t = \sigma(x_tU^o + h_{t-1}W^o)$$
$$\tilde{C}_t = \tanh(x_tU^g + h_{t-1}W^g)$$
$$C_t=\sigma(f_t*C_{t-1} + i_{t}*\tilde{C}_t)$$
$$h_t = \tanh(C_t)*o_t$$

I am confused about the parameters: $U^i, U^f, U^o , U^g$ and $W^i, W^f, W^o , W^g$.  Are those parameters trained by data(e.g., back-propagation)?

My second question is that, In the Figure, We have the three block of LSTM 1, 2, and 3 (mark by blue color). Do them share the same $U^i, U^f, U^o , U^g$ and $W^i, W^f, W^o , W^g$. ?  Or each block have their own $U^*$ and $W^*$?


texts: Use MinMaxScaler , label encoder, one hot encoder , keras prediction file for later prediction

texts: I'm new to neural networks and data science field. I have a dataset with over 90,000 rows. which Include 9 text columns & 29 Number Columns. after encoding with label encoder and one hot encoder It has over 10,000 columns.  Now I like to save those scalers, encoders and prediction files for later use.  But I have no Idea how to save and use them later for single prediction. Any help is appreciate.  Thank You


texts: Pandas: how can I create multi-level columns

texts:
I have a pandas DataFrame which has the following columns:

n_0
n_1
p_0
p_1
e_0
e_1

I want to transform it to have columns and sub-columns:

0
    n
    p
    e
1
    n
    p
    e

I've searched in the documentation, and I'm completely lost on how to implement this. Does anyone have any suggestions?


texts: What does the repeated message "No further splits with positive gain, best gain: -inf" mean?

texts:
I am training a LightGBM classifier on a binary classification problem.

From time to time I get the following message repeatedly:

[LightGBM] [Warning] No further splits with positive gain, best gain: -inf

It almost seems, that if it happens, the training is stuck in an infinite loop
Does someone know, what the reason is for this and if it is indeed an infinite loop or if for example this message is output each time the split of a tree node does not succeed and the algorithm continues with the next node letting the current one be a terminal (leaf) node. In the latter case I would maybe wait a bit longer before I terminate the process.

Unfortunately at present I cannot really say with which parameters the training was started since it is performed in a hyperopt loop, I would need to terminate it first.

The training parameters were:

{'n_estimators': 1730, 'max_depth': -1, 'num_leaves': 24579, 'min_child_samples': 450, 'reg_lambda': 0.0, 'silent': False, 'reg_alpha': 0.0}

It seems that for the unlimited max_depth in combination with the high number of num_leaves the algorithm frequently came to a point where it could not find a split with at least 450 samples (the total number of samples is less than 500'000).


texts: Are there any public datasets about mental health of patients containing physiological and psychological symptoms?

texts:
I would like to segment mental illnesses with clustering using machine learning. To do so I need training dataset which contains physiological and psychological symptoms of an subject. The closest public datasets which I found are on this page https://openpsychometrics.org/_rawdata/.
However the tests were designed to fit current classification of mental illnesses like anxiety and depression, so it is not very useful for making new classification. I am searching for dataset where would bee patients symptoms prior to diagnosis and then the final diagnosis.
What I would like to find out by this data analysis is wherever the clusters match current diagnoses like anxiety or more interestingly what are the alternative classifications of psychosomatic illnesses.

Note I am not an researcher, I just want to learn data clustering on an interesting problem.


texts:
While running SVC(), how we can hyperparameter tune C vs gamma combination?
I could see changes in C and gamma are impacting the accuracy differently. Also, what I understand about C and gamma are:

C is the cost of mis-classification which means a large C gives you a low bias and high variance, low bias because you penalize the cost of mis-classification a lot. And a small C gives you a higher bias and lower variance.

Gamma controls the shape of the "peaks" where you raise the points. A small gamma gives you a pointed bump in the higher dimensions, while a large gamma gives you a softer, broader bump. So a small gamma will give you a low bias and high variance, while a large gamma will give you a higher bias and low variance.


texts: how can i interpret kernel density plots from classification?

texts:
all,
i have a classification problem where i am predicting likelihood of client defaulting on loan. i plotted the predicted probabilities from my model, and then plotted against the label '1' for default or 0 for non-default.

it is cut out here but y axis is the density. am i right to reason that this shows an exponential distribution, or that the fact the class 1 curve has a fat tail it shows that default is an extreme / unexpected event? woud you say class 1 is following any type of distribution?
compare this to the below:

doesn't the second graph show that the model isn't that good at distinguishing between class 0 and class 1?


texts: LSTM not learning with extra nontemporal data added after LSTM layer - Keras

texts:
I have three different inputs I would like to send into my LSTM - the sequence of "words", extra temporal information, as well as extra nontemporal information.

Following Adam Sypniewski excellent answer to this question (Adding Features To Time Series Model LSTM) I came with this architecture:

word_input = Input(shape=(sequence_length,))
temporal_metadata = Input(shape=(sequence_length, 1))
nontemporal_metadata = Input(shape=(nontemporal_num_features,))

embedding = Embedding(input_dim=vocab_size, output_dim=100)(word_input)
temporal_info = concatenate(inputs=[embedding, temporal_metadata])
rnn = LSTM(50, dropout = .5, recurrent_dropout=.5)(temporal_info)
all_info = concatenate(inputs=[rnn, nontemporal_metadata])
dense_1 = Dense(10, activation='relu')(all_info)
dropout_1 = Dropout(.5)(dense_1)
dense_2 = Dense(1, activation='sigmoid')(dropout_1)

model = Model([word_input, temporal_metadata, nontemporal_metadata], dense_2)
model.compile(loss='binary_crossentropy', optimizer='adam'
model.fit(x=[my_words, my_temporal, my_nontemporal], y = my_y,
     epochs = 30,
     validation =([my_words_test, my_temporal_test, my_nontemporal_test], y_test))

This does not learn, and gets an AUC of 0.5

However if I don't add the nontemporal data and change the above code to

word_input = Input(shape=(sequence_length,))
temporal_metadata = Input(shape=(sequence_length, 1))

embedding = Embedding(input_dim=vocab_size, output_dim=100)(word_input)
temporal_info = concatenate(inputs=[embedding, temporal_metadata])
rnn = LSTM(50, dropout = .5, recurrent_dropout=.5)(temporal_info)
dense_1 = Dense(10, activation='relu')(rnn)
dropout_1 = Dropout(.5)(dense_1)
dense_2 = Dense(1, activation='sigmoid')(dropout_1)

model = Model([word_input, temporal_metadata], dense_2)
model.compile(loss='binary_crossentropy', optimizer='adam'
model.fit(x=[my_words, my_temporal], y = my_y,
     epochs = 30,
     validation =([my_words_test, my_temporal_test], y_test))

It works great. Can anyone please help me find what is going wrong in the first code sample that would make the model unable to learn. Thanks!


texts:
After I calculated the similarities matrix, how do I get the neighbors, for example, consider the matrix of similarities between users, if I did not make any mistakes, the matrix must be symmetric with diagonal 1 considering pearson.

u0: 1.0  -0.2  0.8  -0.6  0.2
u1:-0.2   1.0  0.7  -0.3  0.4
u2: 0.8   0.7  1.0   0.4  0.3
u3:-0.6  -0.3  0.4   1.0  0.8
u4: 0.2  -0.4  0.3   0.8  1.0

So if I want get 2 neighbors for u0, the result would be [u2:0.8, u4:0.2].

I didn't undertando why to do KNN and not a sort operation on its row?


texts: How does two convolution layers make computation quadratic increase in GoogLeNet?

texts:
In GoogLeNet papaer, "For example, in a deep vision network, if two convolutional layers are chained, any uniform increase in the number of their filters results in a quadratic increase of computation."

But, I don't know how a quadratic increase is possible when the number of filters increase.


texts: Explanation of inductive bias of Candidate Elimination Algorithm

texts:
The definition of inductive bias says that

The inductive bias (also known as learning bias) of a learning algorithm is the set of assumptions that the learner uses to predict outputs given inputs that it has not encountered.

The inductive bias of Candidate elimination says that

The target concept c is contained in the given hypothesis space H

My question is , how does this inductive bias help us to predict for next instance in given dataset?


texts: Can we remove features that have zero-correlation with the target/label?

texts:
So I draw a pairplot/heatmap from the feature correlations of a dataset and see a set of features that bears Zero-correlations both with:

every other feature and
also with the target/label

.Reference code snippet in python is below:

corr = df.corr()
sns.heatmap(corr) # Visually see how each feature  is correlate with other (incl. the target)

Can I drop these features to improve the accuracy of my classification problem?
Can I drop these features to improve the accuracy of my classification problem, if it is explicitly given that these features are derived features?


texts: Understanding dimensions of Keras LSTM target

texts:
I'm learning about Keras and LSTMs and came across this tutorial, but I don't understand the dimensions of the target variable. Quoting the article below:

  The training y data in this case is the input x words advanced one
  time step – in other words, at each time step the model is trying to
  predict the very next word in the sequence. However, it does this at
  every time step – hence the output layer has the same number of time
  steps as the input layer.

  To make this a bit clearer, consider the following sentence:

  “The cat sat on the mat, and ate his hat. Then he jumped up and spat”

  If num_steps is set to 5, the data consumed as the input data for a
  given sample would be “The cat sat on the”. In this case, because we
  are predicted the very next word in the sequence via our model, for
  each time step, the matching output y or target data would be “cat sat
  on the mat”.

The article than shows the following code to generate batches:

def generate(self):
    x = np.zeros((self.batch_size, self.num_steps))
    y = np.zeros((self.batch_size, self.num_steps, self.vocabulary))
    while True:
        for i in range(self.batch_size):
            if self.current_idx + self.num_steps >= len(self.data):
                # reset the index back to the start of the data set
                self.current_idx = 0
            x[i, :] = self.data[self.current_idx:self.current_idx + self.num_steps]
            temp_y = self.data[self.current_idx + 1:self.current_idx + self.num_steps + 1]
            # convert all of temp_y into a one hot representation
            y[i, :, :] = to_categorical(temp_y, num_classes=self.vocabulary)
            self.current_idx += self.skip_step
        yield x, y

I understand that the model is trying to predict the next word, and I understand that the dimensions of x are self.batch_size, self.num_steps, but I don't understand the y dimensions: self.batch_size, self.num_steps, self.vocabulary. Why not just self.batch_size, self.vocabulary? I might have misunderstood the article, but I thought each record in the batch corresponds to a different time step, or at least that's what I would have assumed for any other non-LSTM type of model. Does Keras automatically unroll the y output because it involves an LSTM?


texts: Multi-label learning, how to interpret the scores of instances

texts:
I am working on a multi-label learning classification problem. When I tune the hyperparameters of the model, the sign of each label (each line on the MLL) remains the same while the scores change.

For example, an instance is predicted 1 in the first test with 0.5 score for label one and 0.4 for label two. While on the second, it's also predicted as 1 but with a less score for label 1 than the second.

As I am interested on the value of the score, I would like to know if there is a method that can tell me which of the two tests is better. In other words, I would like to say that this label is more likely to be predicted than the other (even though that a test gives better score to other one).


texts:
My dataset has 2 columns:

1) Gender( that has 1 or 2)

2) Rating1(That has 3 ratings (1,2,3))

rating1: 1,2,3,3,2

gender: 2,1,2,2,1

I want output as:

I have tried this:

nutri.groupby(['Rating1','Gender']).nunique()

but I get an output which looks wrong.


texts: Initializing weights that are a pointwise product of multiple variables

texts:
In two-layer perceptrons that slide across words of text, such as word2vec and fastText, hidden layer heights may be a product of two random variables such as positional embeddings and word embeddings (Mikolov et al. 2017, Section 2.2): $$v_c = \sum_{p\in P} d_p \odot u_{t+p}$$ However, it's unclear to me how to best initialize the two variables.
When only word embeddings are used for the hidden layer weights, word2vec and fastText initialize them to $\mathcal{U}(-1 / \text{fan_out}; 1 / \text{fan_out})$. When the product of two random variables is used, we might:

initialize the first variable with ones and the other variable with $\mathcal{U}(-1 / \text{fan_out}; 1 / \text{fan_out})$: This would maintain the distribution of the weights, but the gradients to the second variable would be way too large.

initialize the variables with a 2-factor of $\mathcal{U}(0, 1)$:

and then rescale their product to $[-1 / \text{fan_out}; 1 / \text{fan_out}]$. This would maintain the distribution of the weights, but enlarge the gradients to both variables, since they are now both initialized to ones or close to ones.

I will appreciate any ideas and pointers to existing research in this direction.


texts: What is confidence reflexivity in association rules?

texts: I did not understand the meaning of reflexivity term in association rules. I could not find a basic explanation about it. All I found was just academical researches and they heavy for me. Can anybody explain the confidence reflexivity in association rules?


texts: Fitting input data into Gaussian distribution

texts:
I'm currently reading papers on Variational Autoencoders (VAE). According to this article (http://proceedings.mlr.press/v95/guo18a/guo18a.pdf):

By fitting the input data sample x(i) into the Gaussian distribution
with the reconstructed mean vector and the reconstructed standard
deviation vector, we can get the corresponding reconstruction
probability N (x(i)|µxˆ(i, l), σxˆ(i, l)) of the lth generated latent
vector.

Basically, I don't understand what "fitting the input data sample x(i) into the Gaussian distribution" means. I assume we first build a Multivariate Gaussian distribution with the vectors µxˆ(i, l) and σxˆ(i, l). But then, what does that mean to fit a vector into that distribution?
Also, does the result of that calculation correspond to what we call the likelihood then?
Many thanks,
Guillaume


texts:
I build a CNN model using keras on the cat vs dog dataset. Now what I want is with the image classification my model should also locate that animal on that image. I searched on web and found that YOLO is used for object localization, but when I looked up some codes on github I didn't understand how to implement it with my model.

I have a simple CNN model on keras and I want to localized the animal. If someone has other suggestion please do give them.


texts: When is it convenient dropping duplicates when performing Time Series Prediction?

texts:
I have this Dataset.

 Head(df):

          ï..date   store item  qty    unit_price item_category
 1220   2017-03-15    38  295   255      13.30            13
 1453   2017-03-15    41   43   291      10.08            13
 3181   2017-03-15    75  186   324      28.59            13
 3541   2017-03-15    42  295   133      13.30            13
 3624   2017-03-15    56  127   969      55.23            13
 4657   2017-03-15    52   75   121      27.23            13
 4702   2017-03-15    13  413    75      18.55            13

There are clearly a lot of duplicates in the date column (since it depends on the stores and the items , that was expected), and since my task is to make a prediction and return predicted values in numeric of the qty variable - the target prediction variable "qty", represents the total quantities sold at the ITEM/DAY level- i think i can drop the duplicated dates (I'll be using the ARIMA model) and keep only the date , item and qty column for my prediction .

I wanted to know if my approach was right, any suggestion would be much appreciated , Thank you.


texts:
I m new in the domain of machine learning. I m here to ask for some elucidation.
I have a data set presented as a time series( from a strain sensor coming from a wind turbine). In this time series, we can capture stop and start as u can see in the attached figure . I would like to know. which type of layer should I use? is it possible to make a model that counts up the start and stop of the signal(output dimension = 2)? Until now I just did classification using conv1D and the classes are (there is a stop, there is a start, there is nothing) but this way proceed is not convenient since sometimes we can have a rotor stop and start.
I already thanks you a lot for your help,


texts: ADAM optimizer yields sub-optimal results

texts:
I've been writing my own neural network from scratch to get a better understanding of how they work (using MATLAB initially, but plan to port it to C++ afterwards). One major problem for me has been choosing a good optimizer. I've read a lot about this, and it seems the consensus is that while it certainly depends on your data, ADAM is the best generalist approach.
However, when I train my network, it seems that ADAM underperforms regular SGD and SGD with momentum when comparing final results. It does beat them in time to reach a certain threshold, but the best result suffers.
For example, using a beta1 of 0.9 and beta2 of 0.999, my best result with ADAM is 97.6% accuracy, while with the others I get to ~98.1%. However, ADAM gets to the 97% range in like 10 epochs, while with the others it takes a bit little longer.
I've tried it with and without batch normalization, trying different settings like rate = 0.01 or rate = 0.001 and different beta values, but they all seem to yield the sameish result.
Here is my Adam implementation:
// To do: Vectorize everything using indexed matrices
        function [obj] = adam(obj, gradient, rate, epoch)
            for L=1:obj.numLayers
                obj.m(L).b = 0.9 * obj.m(L).b + 0.1 * sum(gradient(L).b, 2);
                obj.m(L).w = 0.9 * obj.m(L).w + 0.1 * gradient(L).w;
                obj.v(L).b = 0.999 * obj.v(L).b + 0.001 * (sum(gradient(L).b, 2).^2);
                obj.v(L).w = 0.999 * obj.v(L).w + 0.001 * (gradient(L).w .^2);
                obj.layer(L).b = obj.layer(L).b - rate * obj.bias_fix(epoch) .* obj.m(L).b ./ sqrt(obj.v(L).b) + eps(single(1));
                obj.layer(L).w = obj.layer(L).w - rate * obj.bias_fix(epoch) .* obj.m(L).w ./ sqrt(obj.v(L).w + eps(single(1)));
            end
        end

Have I done anything wrong here assuming everything else is correctly calculated?
I noticed that the bias fix part of the algorithm could be pre-calculated, so that's what I did and just indexed the result. A part of this I could have gotten wrong is the gradient part. Most people just divide the entire gradient by the number of samples in the batch, so that's what I did.
But when I think about it, doesn't it make more sense to divide the first cost derivative by the number of samples and let this division backpropagate?


texts: Sklearn StratifiedKFold code explanation

texts:
While going through the following blog I came across the following code snippet

from sklearn.cross_validation import StratifiedKFold
eval_size = 0.10
kf = StratifiedKFold(y,round(1./eval_size))
train_indices, valid_indices = next(iter(kf))
X_train, y_train = X[train_indices], y[train_indices]
X_valid, y_valid = X[valid_indices], y[valid_indices]

and I am unable to understand how it works. Can anyone please help me with an explanation? Thanks


texts: Why are ROC curves better for imbalanced datasets?

texts:
I have recently read this:

" AUC(Area Under Curve) is good for classification problems with a class imbalance. Suppose the task is to detect dementia from speech, and 99% of people don’t have dementia and only 1% do. Then you can submit a classifier that always outputs “no dementia”, and that would achieve 99% accuracy. It would seem like your 99% accurate classifier is pretty good, when in fact it is completely useless. Using AUC scoring, your classifier would score 0.5. "

Can someone please explain why does it reach 0.5? If 99% are negative and we output always "no", wouldn't that mean that the TruePositiveRate will be very high and the FalsePositiveRate very low, resulting in an Area Under Curve close to one?


texts: Loading a Keyword and Evaluating the Information

texts:
I am an Ex Service Veteran and need assistance with a Small Program to use with my Rats of Tobruk Project, for the purpose of evaluating Archives Information, which I normally do by Manual Means, but my List has grown to 11,663 records that need Verification.

I have put this together as a Guide to what needs to happen.
I hope someone with experience can assist me with a Program.

Automate Finding Service Records & Evaluation

I produce an Excel List from my National Rats of Tobruk Database, consisting of two (2) Columns – Service Number and NAA.

a) Service Number will contain the List of Service Numbers to be found and evaluated, in sequence.
b) NAA will initially be Blank, but will be changed to (Y) if a Condition is found “True” for the Loaded Service Number.

Open Browser (Firefox or Chrome)

a) Browse to (https://recordsearch.naa.gov.au/SearchNRetrieve/Interface/DetailsReports/SeriesDetail.aspx?series_no=B883)
b) Set Button to “List Report”
c) Click Tab = “Basic Search”

From Excel Service Number List, Enter the first “Service Number” in the Pad = “Keywords” and Press “Enter” or Click “Search”

a) If the Column “Digitised Item” is Blank – Loop Back – load the next Service Number from the Excel Service Number List and evaluate.
b) If the Column “Digitised Item” shows a "Document Icon" or Other Content, the Condition = True – Go To Excel Service Number List, and show “Y” in the NAA Column, for that Loaded Service Number.

On the Web Page, Click “New Search” and Load the next Service Number from the Excel Service Number List and Repeat the Evaluation Process for each Service Number, till the end of the Excel Service Number List is exhausted.
When the Service Number List is completed, Save the Excel File to Desktop/ROTWEB/Service Number NAA Test. (ROTWEB is a Folder on my Desktop)

I hope you are able to follow my thoughts - if not please contact me, either with a Reply Post or by my Email ( ocar23@iinet.net.au )
Best Regards
Owen Carlton
( www.ratsoftobruktribute.com )


texts:
I'm new for learning-to-rank. I'm trying to learn the Learning to rank example provided by xgboost. I found that the core code is as follows in rank.py.

train_dmatrix = DMatrix(x_train, y_train)
valid_dmatrix = DMatrix(x_valid, y_valid)
test_dmatrix = DMatrix(x_test)

train_dmatrix.set_group(group_train)
valid_dmatrix.set_group(group_valid)

params = {'objective': 'rank:pairwise', 'eta': 0.1, 'gamma': 1.0,
               'min_child_weight': 0.1, 'max_depth': 6}
xgb_model = xgb.train(params, train_dmatrix, num_boost_round=4,
                           evals=[(valid_dmatrix, 'validation')])
pred = xgb_model.predict(test_dmatrix)

Group data is used in both training and validation sets. But test set prediction does not use group data. I also looked at some explanations to introduce model output such as What is the output of XGboost using 'rank:pairwise'?.

  Actually, in Learning to Rank field, we are trying to predict the
  relative score for each document to a specific query.

My understanding is that if the test set does not have group data, no query is specified. How does the model output the relative score to the specified query?

And I've tried adding test_dmatrix.set_group(group_test). The output results of the two methods are in good agreement like:

[ 1.3535978  -2.9462705   0.86084974 ... -0.23594362  0.712791
 -1.633297  ]

So my question as follows:

Why does it not need to set test group when using 'rank:pairwise' in xgboost?
How can I get label to the specified group query based on the forecasting score results?

Can anybody explain it to me? Thanks in advance.


texts: About neural network called "Self Organizing Maps"

texts:
I have this simple neural networks ("Self Organizing Maps"):

What is the purpose of the lateral connections beetween the perceptrons (neurons) ?


texts: How to install XGBoost or LightGBM on Windows?

texts:
I'm a Windows user and would like to use those mentioned algorithms in the title with my Jupyter notebook which is a part of Anaconda installation.

I've tried in anaconda promt window:

pip install xgboost

which retuned:

  Could not find a version that satisfies the requirement xgboost (from
  versions: ) No matching distribution found for xgboost

Likewise, I've tried:

conda install lightgbm

which returned:

  Solving environment: failed

  CondaHTTPError: HTTP 000 CONNECTION FAILED for url
  https://repo.anaconda.com/pkgs/r/noarch/repodata.json.bz2 Elapsed: -

Help would be appreciated.


texts: What are the advantages vs disadvantages of cross validation types? Like k-fold, leave one out, etc.


texts: Choosing a right algorithm for template-based text generation

texts:
I am doing a text generation project -- the task is to basically represent the statistical data in a readable way.

The way I decided to go about this is template-based: each data type has a template for how sentence should be formed and what synonyms can be used.

I'm torn about whether some kind of ML techinque can bolster this template-based approach. Text should be unique -- so I need an algorithm that optimises for uniqueness.

Now, there are API solutions that can give me uniqueness score at the end of the text (and even in the middle of it) -- so my first instinct was to try out reinforcement learning with sparse rewards. Templates can be represented as a tree, which the algorithm traverses, getting rewards in the end and during its trip. Input is its current options on where to go and the output is its decision on where to go.

The problem with this approach is that after it succeeded in generating an unique text, it can't generate the same one (I mean it can, but the score would be 0), which might prove difficult for the model to learn. Also many articles on the web indicate that RL is really, really hard to tune properly.

I am now in the pre-research process, so any feedback on how I should approach this task is appreciated. Maybe there is no need for ML at all?

What do you think? My instinct tells me that such problems should have established solutions and I'm just searching the wrong way.

Thanks!


texts: Policy Gradient with continuous action space

texts:
How to apply reinforce/policy-gradient algorithms for continuous action space. I have learnt that one of the advantages of policy gradients is , it is applicable for continuous action space. One way I can think of is discretizing the action space same as the way we do it for dqn. Should we follow the same method for policy -gradient algorithms also ? Or is there any other way this is done?

Thanks


texts: Date and time extraction from Excel file in Python or Pandas

texts:
I have an Excel file with a column that contain both date and time in a single cells as follows:
2020-07-10T13:32:01+00:00
I'm wondering how to extract this cell, split date and time (that are separated by a "T") and write them into 2 new columns "Date" and "Time", and able to use them afterwards to, for example, do Time math operations.
I'd have a start from pandas:
    df = pd.read_excel('file.xlsx')
    def convert_excel_time(excel_time):
    return pd.to_datetime()

but I actually don't know if it is achievable with pandas.
If I have this:

I would like to calculate how many minutes passed (for the same date) for the same ID column (2nd column).
I guess I can use python tdelta, or is there a pandas alternative?
Thank you.


texts: Scikit learn kmeans with custom definition of inertia?

texts:
I've coded a small clustering algorithm for time signals using kmeans, which works ok (gives acceptable results).

However, kmeans uses the sum of squared differences. I would like to be able to input instead my own measure of difference, but there doesn't seem to be a way provided by the library to do that.

What would be the easiest way to achieve this? Any other python library which may provide me some way to input instead my own function to define the distance? Or I guess I could instead re-implement the algorithm myself, but I'd rather keep the sci-kit one (since they provide functionalities I want to use such as parallel processing).


texts:
I have a tabular dataset of financial transactions with a target binary variable 'isFraud' which indicates if the transaction is fraud or not.
I want to build a model that given some past trasactions, will output one or more transactions that are likely to be non-fraud. I thought of predicting some future transaction (=rows) based on the past, and the predictor will learn only from non-fraud transactions.
My question are the following:

Is there a way to predict future rows of a tabular dataset based on existing ones?
Do you find this way (of predicting non-fraud) plausible?

Thanks


texts: Clustering credit card accounts based on their balance trajectories

texts:
I am trying to cluster credit accounts based on the shape of their balance trajectories over the next 36 months, to identify the different types of shapes possible in the portfolio. Here is how I am doing it:

Standardize balance trajectories by converting them into proportion of balances (balance at each point in time divided by the initial balance at the starting point)
Do PCA and reduce from 36 to 3 dimensions. The top 3 components explain almost ~85% of the variations in the initial 36 values.
Do K-means clustering (using fastclus in SAS).

The issue is that the CCC (cubic clustering criterion) value I end up with is very high (~-500) even after I identify almost 15-20% of my data as outliers (using the strict= options in fastclus). I understand the CCC value should be around 2-3 for a statistically sound clustering. Is there any way I can improve this? Should i normalize the 3 principal components before doing k-means?

Update: It seems PCA can lead to order loss because it is not mindful of the sequence in which balances occur. Can anyone suggest better parameters to capture 36 months of balance information that I can use in K-means? I want to do K-means with lowest dimensionality possible because distance doesn't work with high dimensional data (36 in my case). I think using slope, curvature and level. I can define level with average values, slope with regression slope coefficient of balance versus time equation but how do I define curvature?

My basic goal is to cluster those accounts together which have a similar balance trajectory.


texts:
I have time series data generated from the sensor. Now I know the the actual point I need to find is at the 11:50, as you can see in the graph this point dooes not hold any statistical property which I can give to detect this point. Any idea how to find this point as process is getting completed at this point?

What library or the algorithm I should use to find this point in time series?

The data I have provided is the historical data, I need to provide this data to algorithm, which can detect such point when streaming data is given to this algortihm. I needed any ideas hoe can I build this algorithm for the point to  detected.
enter image description here


texts: Public dataset for news articles with their associated categories for multilabel data classification

texts:
I am wondering if there are any public datasets of news, like New York Times (NYT) or similar to various news categories such as politics, entertainment, lifestyle, general news, sports etc.
I want to use such a dataset for multilabel data classification of various sentences or paragraphs i.e a sentence could belong to politics, entertainment, sports, or all, I need the dataset to classify the data into more than one label. I was planning to train a classifier with such a dataset and use it for predictions. However, I couldn't find any. Are there any such known datasets available?
I want dataset something like this but for news category


texts:
I am very new at this, so I might be wrong about my choice of model, but my problem is the following. I am trying to generate music, hence the reason I am using an LSTM. I have the following sequence of length 6 and 3 features, where each of one these features is a class, [['n','16','a4'],['n', '16', 'b4'],['r', '8' ,'b4'] ...], and I want to predict the next step in this sequence which will be for e.g. ['n', '8', 'c5'].

I decided to use an LSTM  and treat it as a multivariate timeseries, but I am confused as to what the output shape could be. My current model looks like this

model = Sequential()
model.add(LSTM(50, input_shape=(network_input.shape[1], network_input.shape[2])))
model.add(Dense(3))
model.add(Activation('softmax'))
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics = ['accuracy'])

I am sure that this is wrong, especially when it comes to the output layer, because I know that when dealing with classification, the output layer must be the size of possible classes. But I don't know how to translate this here, since I have multiple classes for each feature.
Can anyone advise as to how to approach this ?


texts: How to deal with time-series imbalance classification data?

texts: I want to predict the user to buy or not a product in next month in the e-comercial site. I mainly using the past 1-year data to predict it. But I found the training data is imbalanced, and the buy(1)users is just 15% of all users, and buy(1) is just 8% in the test data. How to deal with this issue about time-series imbalance classification data.


texts: How can I improve the recall of a certain class in a multiclass-classification result

texts:
I am working on a multiclass classification which is to assign medical related queries of web search to certain departments of hospital.My classifier is based on the fastText.

I found for most conditions, the result is good enough say recall is 0.8 for Nephrology. However, for just one department, Dermatology, the recall is pretty low,like 0.5. Unfortunately, this label has most samples in the test data.

How can I improve the recall of one class while maintain the performance of other classes? Will ensembling method work?


texts: How do I use the data I have to make predictions?

texts:
I've a set of CPUs (~100). I've the results of each CPU being benchmarked against a suite of benchmarks (~8). There is a specific task that interests me, and I need to find the optimal CPU for that task. A small subset of CPUs (~10-20) has noisy measurements for how well it performs at that specific task.
Two questions:

How do I use the data I have to make predictions about how well a given CPU will perform at that specific task (when its performance wasn't directly measured on that specific task)? This presumes that the large set of benchmark suite results have captured the CPU performance aspects that are relevant to that specific task. An interpretable white-box solution would be prefered, so I can tell which benchmarks are relevant for which task (and how relevant).
How do you formulate this problem in a general way, so that I could research it?


texts:
I am working on a binary classification task using SVM. The dataset is quite large so I don't want to use k-fold CV for parameter tuning, but instead a simple train-validation-test split. I have done the following:
X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify = y, test_size=0.2, random_state=1)

X_train, X_val, y_train, y_val = train_test_split(
    X_train, y_train, stratify = y_train, test_size=0.25, random_state=1)

So I have a 60-20-20 training-validation-test split. Since my validation set is predefined I want to use Sklearn's predefinedsplit. So I need to get the indices of my training and validation samples and set the validation indices to 0 and training indices to -1 so I tried the bottom answer of this question:
split_index = [0 if x in X_val.index else -1 for x in X_train.index]
But this returns a list of only -1's. I am unsure where I am going wrong. All my X and y _train, _val, _test are dataframes so .index can be applied. I have printed out X_train.index and X_val.index and both return Int64Index arrays of different lengths.
I also tried using Hypopt as one of the answers in the above link mentioned but it seems to be broken at the moment. Any suggestions as to how I should proceed?


texts: Why is my loss increasing in gradient descent?

texts:
When the learning rate is 0.01 the loss seems to be decreasing whereas when I increase the learning rate even slightly, the loss increases. Why does this happen? Are the gradients calculated wrong?

Neural Network with 2 hidden layers ,128 neurons in the first hidden layers and 64 in the second hidden layer. The output layer consists of a single sigmoid Neuron

class FNN:
    def __init__(self):
        self.W1=None
        self.b1=None
        self.W2=None
        self.b2=None
        self.W3=None
        self.b3=None
    def sigmoid(self,x):
        return 1/(1+np.exp(-x))
    def forward_prop(self,x):
        self.Z1=np.dot(self.W1,x)+self.b1
        self.A1=np.tanh(self.Z1)
        self.Z2=np.dot(self.W2,self.A1)+self.b2
        self.A2=np.tanh(self.Z2)
        self.Z3=np.dot(self.W3,self.A2)+self.b3
        self.A3=self.sigmoid(self.Z3)
        return self.A3
    def back_prop(self,x,y):
        self.forward_prop(x)
        m=x.shape[1]
        self.dZ3=self.A3-y
        self.dW3=np.dot(self.dZ3,self.A2.T)/m
        self.db3=np.sum(self.dZ3,axis=1,keepdims=True)/m
        self.dZ2=np.dot(self.W3.T,self.dZ3)*(1-self.A2**2)
        self.dW2=np.dot(self.dZ2,self.A1.T)/m
        self.db2=np.sum(self.dZ2,axis=1,keepdims=True)/m
        self.dZ1=np.dot(self.W2.T,self.dZ2)*(1-self.A1**2)
        self.dW1=np.dot(self.dZ1,x.T)/m
        self.db1=np.sum(self.dZ1,keepdims=True)/m
    def fit(self,x,y,epochs=100,learning_rate=0.01,plot=True,disp_loss=False):
        np.random.seed(4)
        self.W1=np.random.rand(128,x.shape[0])
        self.b1=np.zeros((128,1))
        self.W2=np.random.randn(64,128)
        self.b2=np.zeros((64,1))
        self.W3=np.random.randn(1,64)
        self.b3=np.zeros((1,1))
        m=x.shape[1]
        loss=[]
        for i in range(epochs):
            self.back_prop(x,y)
            self.W1-=learning_rate*self.dW1
            self.b1-=learning_rate*self.db1
            self.W2-=learning_rate*self.dW2
            self.b2-=learning_rate*self.db2
            self.W3-=learning_rate*self.dW3
            self.b3-=learning_rate*self.db3
            logprobs=y*np.log(self.A3)+(1-y)*np.log(1-self.A3)
            cost=-(np.sum(logprobs))/m
            loss.append(cost)
        e=np.arange(1,epochs+1)
        if plot:
            plt.plot(e,loss)
            plt.title('LOSS PLOT')
            plt.xlabel('Epoch')
            plt.ylabel('Loss')
            plt.show()
        if disp_loss:
            print(loss)
    def predict(self,x):
        y=np.where(self.forward_prop(x)>=0.5,1,0)
        return y

F=FNN()
F.fit(x_train,y_train)
y_pred=F.predict(x_train)

Output

Learning Rate:0.01

Learning Rate:1


texts:
Please let me know if this is the right place to ask this (or if any of my tags are wrong) or if I need to write this any differently.

Do I use the mean vector from my training set to center my testing set when dimension reducing for classification?

I am using the principal component analysis procedure to reduce the dimensions of the training set.  I build the classifier.  Then, before I classify the feature vectors from the test set, during the centering part of the dimension reduction, do I use the same mean vector from the training set, do I take the mean vector of the testing set and subtract that from the test set, or do I take the mean vector of the union of the training and test set and subtract that from the test set?

If the third option, does that mean I was also supposed to use the union of the training and testing set to center the training set as well?  No, (for the sake of generalizing to other testing sets) right?

Also, even though I am pretty sure the answer will be the same as above, can you please let me know if the same is true for using the covariance matrix from the training set to get an eigenvector matrix and multiplying the inverse (transverse) of it times the test set to reduce it.  Or, do we use the testing set or the union of the two to get the covariance and then eigenvector matrix to multiply times the testing set?

Please let me know if any of the premises are wrong.  This is my first time.


texts: Confusion about maximal and closed itemsets in association rules

texts:
So I'm studying association rules and Apriori algorithm for my data science class. There's a section about compactness (compact representation of frequent itemsets). I'm not sure whether I'm not getting something here or I'm constantly encountering plain mistakes (this professor is known for making a lot of mistakes in his literature, and they are just slides anyway, not a book or anything).
It starts with a couple of definitions and some corollaries (I had to google English equivalents to the following terms, I hope I got them right).
Definition 1: An itemset is maximal frequent if none of its direct supersets is frequent.
Possible mistake #1: It is not explicitly mentioned that it has to be a frequent itemset (while other sources mention it).
Then a corollary is stated:
Corollary: If $X$ is a maximal frequent itemset then all of its subsets are frequent.
Possible mistake #2: Okay so not technically a mistake but rather stating the obvious, because this should apply to frequent itemsets in general and is not specific to the maximal frequent ones.
Definition 2: An itemset $X$ is closed if none of its direct supersets has the same support as $X$.
And a corollary to that definition:
Corollary: All subsets of a closed itemset have the same support.
Possible mistake #3: Reason 1: This corollary just 'sounds' very incorrect. Reason 2: Okay so let's assume that we got a set $X = \{A, B, C, D\}$ and let's say it has some support $s_X$. If we were to assume that $X$ is closed that would only tell us that every one of its direct supersets ($\{A, B, C, D, E\}, \{A, B, C, D, Z\}$, etc...) has the support that is not equal to $s_X$ (actually it would be strictly less than $s_X$ but it doesn't matter). I don't see how this tells us anything about $X$'s subsets?
At one point I thought it might just be me misunderstanding everything. But then comes an example. Since all the item names are in my language, and showing you the original would make it very unpleasant to read, I've replaced names with letters and I've also shortened the example because the mistake (Possible mistake #4) is at the very beginning. So it reads:

In the following example, the itemsets that are bolded are maximal frequent:
$$\begin{matrix} \text{itemset} & \text{support count} \\
A & 4 \\
\mathbf{B} & 5 \\
C & 3 \\
D & 4 \\
\mathbf{\{A, B\}} &  4 \\
\text{etc...}
\end{matrix}$$

Surely an overlook #1: Support threshold is not mentioned.
Possible mistake #4: Aside from that, recalling the maximal frequent itemset definition (Definition 1 here) it states that in order for an itemset to be maximal frequent none of its direct supersets can be frequent. If I'm to trust other sources (Possible mistake #1) it means that a maximal frequent itemset has to be frequent to begin with. In this example, since $\{A, B\}$ is bolded (meaning it's maximal frequent) then it is frequent. But that would mean that $B$ cannot be maximal frequent because it has a frequent direct superset. And that is in contradiction with this example.
Am I misunderstanding something here or are these all really mistakes?
Thanks.