texts: Binary classification of every time series step based on past and future values

texts:
I'm currently facing a Machine Learning problem and I've reached a point where I need some help to proceed.

I have various time series of positional (x, y, z) data tracked by sensors. I've developed some more features. For example, I rasterized the whole 3D space and calculated a cell_x, cell_y and cell_z for every time step. The time series itself have variable lengths.

My goal is to build a model which classifies every time step with the labels 0 or 1 (binary classification based on past and future values). Therefore I have a lot of training time series where the labels are already set.

One thing which could be very problematic is that there are very few 1's labels in the data (for example only 3 of 800 samples are labeled with 1).

It would be great if someone can help me in the right direction because there are too many possible problems:

Wrong hyperparameters
Incorrect model
Too few 1's labels, but I think that's not a big problem because I only need the model to suggests the right time steps. So I would only use the peaks of the output.
Bad or too less training data
Bad features

I appreciate any help and tips.


texts: Which libraries in Python are there in NLP to tokenize the Hindi sentence?

texts:
For English language there are libraries like NLTK, CoreNLP which are used for Text Normalization, Word Tokenization and Detokenization, Sentence Splitting etc.
Like English, is there any library to do above operation using Hindi Script ?


texts: What optimization algorithms are used in scikit-learn?

texts: Gradient descent is one of the well-known optimisation algorithms. However, are the regression algorithms in scikit-learn implemented with gradient descent or some other techniques?


texts: Extend forcasting steps greater than 100? in time seires

texts: So we are attempting to forecast the prices of stocks from yahoo finance for our senior project.  We have a projection up in a line graph but it will only forecast up to 100 data points (days I believe).  Is there any way to increase the number of prediction points over 100?


texts: very low variance explained after applying pca

texts:
I applied PCA on MNIST data and found that the first 64 components are able to retain 86% of variance.

Is there any problem while applying pca to a big dataset like MNIST. Because in most of the papers I have read to take the components which can explain upto 99% of variance. But it would be pointless to apply pca if such variance comes at 120 pca components.


texts: Architecture and feature extraction for keyword suggestion system?

texts:
I encountered a use case for a model that takes the title of a product from a marketplace and outputs keywords most frequently encountered in the product's description. For example, if the product title is 'skinny jeans', the output keywords may include 'measurements', 'waist', 'women', etc., based on what is most frequently encountered for other products with the same words in the title. Assuming I am able to scrape a dataset of product titles and product descriptions, I am wondering what sort of feature extraction and model architecture I can use for this?

Feature extraction. The most obvious representation is Bag of Words, but should I be using TF-IDF for extraction? It seems counter-intuitive to penalize common words across a corpus (as per IDF) when I want to promote keywords that appear most frequently for a given search term. Is it a good idea then to work purely with TF? Is there some other form of extraction I can try?

Model. A lot of text generation models use RNNs/Transformers for sequential generation but its not relevant to my use case. The first solution that comes to mind is an MLP which maps the input to output, training the output against the product description. Im not sure how well this would work, and the model may be enormous (>100M params) due to the size of the input and output vocabulary. Is there a better model for this? Is it possible to use a non-deep learning approach?

Thanks.


texts: How is the GridsearchCV Score calculated?

texts: How is the score of GridsearchCV calculated? Is the score a percentage? Does this mean higher is a better?


texts: Is an artificial neural network a kind of metaheuristic?

texts:
In a research paper i have written i am solving a problem through the use of an ANN, which learns to model a technical process through training data.

I compare that to metaheuristic approaches, which can also find relevant outputs from inputs by searching, e.g., with a genetic algorithm and an appropriate cost function.

A reviewer gave me the feedback that ANN are also a kind of metaheuristics, so i should not compare ANN vs. metaheuristics as different kinds of approaches in the paper. Is this correct? I have looked up the definition of metaheuristic and while i think that i could use a metaheuristic optimizer instead of a gradient based one, an ANN itself is not a metaheuristic.


texts: InterquartileRange takes up most instances in data set

texts:
I'm very new to this community, so please overlook my noobness.

I have a data set with 2948 instances and I tried to remove outliers using InterquartileRange filter in Weka. The issue is that the number of 'YES' instances in ExtremeValues and Outliers takes up to 2947 and 2946 respectively. In other words, all my data are considered outliers.

What does this say about my data set? Or am I not meant to perform IQR on this data, if so, is there other algorithms to identify outliers other than IQR?  And how would one perform regression on such a data set?

Thank you.


texts: How does the feval parameter influences the XGBoost training process?

texts:
In the package XGBoost, is possible to modify the feval (evaluated function) to a personalized one (as shown in the link: MAPE eval metric).

I would like to know how is the training process of the XGB with a different evaluation function.

Is it very different? It really has a big influence? In which step this function matters?


texts: Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

texts:
There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos.

My question is, what's the main benefit of using these libraries compared to, for example, sklearn.model_selection.GridSearchCV() or sklearn.model_selection.RandomizedSearchCV?


texts: XGBoost Log Loss different from GridSearchCV Log Loss

texts:
I have a classification problem where i am trying to predict if the data returns a 1 or 0. So you're classic binary classification. I have my set of data that I have split into the dependent variables (ones i'm training on) and the independent variable (my target that i'm predicting, either a 0 or 1). I am using log loss as the scoring metric for my model.

Firstly I am using the cv function in xgboost to figure out the number of estimators I need as it  stops when the log loss hasn't improved over 50 rounds. I then train my model and predict. My code is below:

def modelfit(alg, dtrain, dtarget, useTrainCV=True, cv_folds=5, early_stopping_rounds=50):

    if useTrainCV:
        # gets the xgb parameters specifically.
        xgb_param = alg.get_xgb_params()

        # this is the internal xgb dataframe that is for efficiency. We map the training data to the labels.
        xgtrain = xgb.DMatrix(dtrain.values, label=dtarget)

        # this performs cross validation on the dataset. As our data is not really time dependent we can afford to cross
        # validate. It stops when it hasnt improved for 50 rounds. This is only for determining n_estimators
        cvresult = xgb.cv(xgb_param, xgtrain, num_boost_round=alg.get_params()['n_estimators'], nfold=cv_folds,
            metrics='logloss', early_stopping_rounds=early_stopping_rounds)

        print(f'Optimal n_estimators - {cvresult.shape[0]}')

        # this sets the most optimal n_estimators parameter into the booster.
        alg.set_params(n_estimators=cvresult.shape[0])

    # fit the algorithm on the data and set evaluation metric
    alg.fit(dtrain.values, dtarget, eval_metric='logloss', eval_set=[(dtrain.values, dtarget)])

    print(alg.evals_result())

    # predict training set:
    dtrain_predictions = alg.predict(dtrain.values)
    print(dtrain_predictions)
    dtrain_predprob = alg.predict_proba(dtrain.values)[:,1]

    # print model report:
    print("\nModel Report")
    print("Log Loss Score (Train): %f" % metrics.log_loss(dtarget, dtrain_predprob))

I then run this function on this particular XGBoostClassifier:

#Choose all predictors
xgb1 = XGBClassifier(
 learning_rate =0.1,
 n_estimators=1000,
 max_depth=5,
 min_child_weight=1,
 gamma=0,
 subsample=0.8,
 colsample_bytree=0.8,
 objective= 'binary:logistic',
 scale_pos_weight=1,
 nthread=-1,
 seed=27)

modelfit(xgb1, X, y)

The logloss value that is returned is: 0.577496 and number of estimators is 65.

I then turn to GridSearchCV to tune the other parameters and I start with:

param_test1 = {
 'max_depth' : range(1,10),
 'min_child_weight' : range(1,6)
}

Note how the original max depth and min child weight are contained within these ranges that i used in xgb1 classifier.

xgb2 = XGBClassifier(
        learning_rate =0.1,
        n_estimators=65,
        max_depth=5,
        min_child_weight=1,
        gamma=0,
        subsample=0.8,
        colsample_bytree=0.8,
        objective= 'binary:logistic',
        nthread=-1,
        scale_pos_weight=1,
        seed=27
)

gsearch1 = GridSearchCV(
    estimator = xgb2,
    param_grid = param_test1, scoring='neg_log_loss', n_jobs=-1, cv=5
)

gsearch1.fit(X, y)
gsearch1.best_params_, gsearch1.best_score_

However this returns me with:

(
{'max_depth': 1, 'min_child_weight': 1}, -0.6275341839742403
)

So my question is how has the grid search said the best parameters are max_depth = 1 and min_child_weight = 1 and the log loss is 0.628 when previously before using GridSearchCV my model returned a better log loss of 0.577 with max_depth = 5 and min_child_weight = 1?

Any help would be appreciated please. Thanks!


texts: how do we perform time-series walk forward analysis

texts:
I recently came across the technique called as walk forward time-series analysis which is equivalent to cross-validation technique for non-time-series problem. In this I learnt, that we decide the min number of training samples and we test on the subsequent few test samples or 1 test sample.

We further include these test samples or test sample into next training and then test on the same number of subsequent test samples as in the previous case. This process goes on and we get multiple models with different or same training sizes(Expanding or sliding) and same test sizes. At the end, we average out the performance of these models.

This performance is considered for one-set of hyperparameters and we try this analysis for other such configurations and whichever configuration gives the best average performance, that set of hyperparameters or configuration is finally selected. Another way we say this is GridSearching the hyperparameters.

However, I have got confused with the nested approach given on the link : https://towardsdatascience.com/time-series-nested-cross-validation-76adba623eb9
It says to take out the validation subset from the training, and for every training, it chooses different hyperparamters and average out the performance, which makes less sense to me. Since, for every model, hyperparamters are different which defeats the purpose of knowing that a configuration which will give good results as we get the data in temporal order. Also, there shouldn't be the need to select the model on validation and we can directly take the hyperparamters which give good average performance on test set.

This is so because the validation set is needed to get the good generalized performance and avoid overfitting. In the above scenario, by training different models with the same random initializations and same set of hyperparamters or configuration, we do the testing multiple times and try to know if the model works well or not i.e generalizing or not. Can someone pls clarify if I am thinking right or wrong? I am new to this type of time-series analysis. Your help would make my job easier as I am currently working on some similar problem.


texts: Which Loss cross-entropy do I've to use?

texts: I'm working with this dataset https://www.kaggle.com/c/sf-crime to predict the crime incident using keras. I've encoded the category with pd.get_dummies and then use it as the validation data. At first I try to use categorical_crossentropy and adam for the loss function but result returns with very bad acc which nearly 0.3 and about 2.7 of loss. So I try to experiment with different entropy by using binary_crossentropy. The result seem to be exaggerate the acc is higher than 0.97 and the loss is less than 0.1 at first epoch. I also provide 0.3 of drop out for each layer. So which crossentropy is accurate or better use for this scenario and which kind of this classification is multi-label or multi-class?


texts: Scalable way to group users with similar titles purchased

texts:
I'm trying to figure out the best way to group customers based on checkout items in their shopping cart. I have the basket, and what's in the basket, but am at a complete loss on how to group all the similar baskets. I have a group of users I believe shouldn't be counted in my overall metrics (or at least acknowledge them). These users create a new account, place 4-5 items in their cart, and check out. Then a new account is created, and the process repeats. They seem to repeat this process for up to 12 hours and then change the items, and I'd like to group those customers vs "regular" customers.

My issue is that it's not always the same 4-5 items -- and I can't figure out how to scale up the 'search'. If I knew it was Item1-Item5, it would be an easy search, even comparing everyone's cart to that, but comparing everyone's cart to everyone else's cart seems like it won't scale at all. (Also the fact that I'd like to have the 'marginals' as well if I'm looking for 5 titles, but they got 4 of them, or all 5 and one new one, etc.)

I'm completely new to this, and even pointers to what terms I should be searching for, or what software packages I should aim to be learning a bit of would be very appreciated.


texts: How to make an lstm ensemble with different input shapes using keras

texts:
This is what I got so far for making an lstm ensemble with one model input for each of the lstm models and for the ensemble model and it works perfectly.
model_input = Input(shape=(50,2))

def firstmodel(model_input):
      hiddenA1 = LSTM(6, return_sequences=True)(model_input)
      hiddenA2 = LSTM(4, activation='relu')(hiddenA1)
      outputA = Dense(24)(hiddenA2)
      model = Model(inputs= model_input, outputs= outputA, name="firstmodel")
      return model

def secondmodel(model_input):
      hiddenB1 = LSTM(30, return_sequences=True)(model_input)
      hiddenB2 = LSTM(20, activation='relu')(hiddenB1)
      outputB = Dense(24)(hiddenB2)
      model = Model(inputs= model_input, outputs= outputB, name="secondmodel")
      return model

firstmodel = firstmodel(model_input)
secondmodel = secondmodel(model_input)

models = [firstmodel, secondmodel]

def ensemble(models, model_input):
      outputs = [model.outputs[0] for model in models]
      y = Average()(outputs)
      model = Model(inputs = model_input, outputs = y, name="ensemble")
      return model

ensemblemodel = (models, model_input)

def evaluate_rmse(model):
      pred = model.predict(X_test)
      rmse = np.sqrt(mean_squared_error(y_test, pred))
      return rmse

ensemble_rmse = evaluate_rmse(ensemblemodel)

But what happens when there are two different model inputs for the first and second model:
first_model_input = Input(shape=(50,2))
second_model_input = Input(shape=(60,2))
What will be the model_input for the ensemble model then? Also what will the shape of X_test be for the ensemble model? Because when there was only one input, like the first_model_input = Input(shape=(50,2)) then the X_test.shape was (2554, 50, 2).
But now there are two inputs, so there are two X_test shapes, one for first model input which is (2554, 50, 2) and one for the second model input which is (2544, 60, 2).


texts: What is max aggregation on a set of word embeddings?

texts:
In a paper I see:

$\mathcal{Q}$ is a set of words.

$\psi_{G^w}$ are word embeddings.

so, $\{\psi_{G^w}(w_t), \forall w_t \in \mathcal{Q}\}$ gives me a set of embeddings for all words in $\mathcal{Q}$.

For example, if I have:

Q = {'a', 'b', 'c'}
embedding_gw.shape = (1000, 8) # 1000 words in vocab, embedding size is 8

I will get:

{
  [1,2,3,4,5,6,7,8], #embedding of 'a'
  [8,7,6,5,4,3,2,1], #embedding of 'b'
  [4,5,3,6,7,8,1,2]  #embedding of 'c'
}

The result of $\max\{\psi_{G^w}(w_t), \forall w_t \in \mathcal{Q}\}$ is supposed to be a single vector.

My question is, how do I get this single $\max$ vector?

Do I sum all values in each embedding and pick the largest one?
Do I pick the $\max$ value of each $i^{th}$ position, creating a completely new vector?
Do I do something else?


texts: Which classifier should I use for sparse boolean features?

texts:
I have training data that is classified into 2 categories: X and Not X

Each piece of training or experimental data has a variable number of boolean features. Each piece of data may have ~100 features, but the total number of possible boolean features may be in the tens or hundreds of thousands.

I want to train a classifier so that given an observation (which has a set of boolean features), it will output the probability that it is X.

Could you recommend which ML algorithms could be used for a large number of sparse boolean features?

Follow-up question:

I also have a large amount of training data that would be best described as Maybe X (I haven't verified is X, but there's a good chance it is X). I could further segment those Maybe X's into Likely X, Maybe X, Probably Not X. Is there a way I can train with Maybe X or it's subclassifications? In the end, what I still only care for is the probability that an observation is X.


texts: tf.trainable_variables() returns blank list outside model_fn

texts:
In a downloaded tensorflow code, when below model_fn_builder is called, then model_fn (down below) loads the tf.trainable_variables().

I tried to move tvars = tf.trainable_variables() out of model_fn so that variables do not get loaded everytime the prediction is called (model_fn is called on every prediction generator parsing).

However tf.trainable_variables() returns blank list. Why is this so? Is tf different within model_fn and outside?

model_fn = model_fn_builder(
  bert_config=bert_config,
  num_labels=len(label_list),
  init_checkpoint=FLAGS.init_checkpoint,
  learning_rate=FLAGS.learning_rate,
  num_train_steps=num_train_steps,
  num_warmup_steps=num_warmup_steps,
  use_tpu=FLAGS.use_tpu,
  use_one_hot_embeddings=FLAGS.use_tpu)

In below code #THIS I AM TRYING TO MOVE OUT OF MODEL_FN is the comment which to be focused:

def model_fn_builder(bert_config, num_labels, init_checkpoint, learning_rate,
                     num_train_steps, num_warmup_steps, use_tpu,
                     use_one_hot_embeddings):
  """Returns `model_fn` closure for TPUEstimator."""

  global tvars , initialized_variable_names , scaffold_fn
  tvars = tf.trainable_variables()  #THIS I AM TRYING TO MOVE OUT OF MODEL_FN

  if init_checkpoint:
      (assignment_map, initialized_variable_names
       ) = modeling.get_assignment_map_from_checkpoint(tvars, init_checkpoint)
      if use_tpu:

          def tpu_scaffold():
              tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
              return tf.train.Scaffold()

          scaffold_fn = tpu_scaffold
      else:
          tf.train.init_from_checkpoint(init_checkpoint, assignment_map)

  def model_fn(features, labels, mode, params):  # pylint: disable=unused-argument
    """The `model_fn` for TPUEstimator."""

    tf.logging.info("*** Features ***")
    for name in sorted(features.keys()):
      tf.logging.info("  name = %s, shape = %s" % (name, features[name].shape))

    input_ids = features["input_ids"]
    input_mask = features["input_mask"]
    segment_ids = features["segment_ids"]
    label_ids = features["label_ids"]
    is_real_example = None
    if "is_real_example" in features:
      is_real_example = tf.cast(features["is_real_example"], dtype=tf.float32)
    else:
      is_real_example = tf.ones(tf.shape(label_ids), dtype=tf.float32)

    is_training = (mode == tf.estimator.ModeKeys.TRAIN)

    (total_loss, per_example_loss, logits, probabilities) = create_model(
        bert_config, is_training, input_ids, input_mask, segment_ids, label_ids,
        num_labels, use_one_hot_embeddings)

    # tvars = tf.trainable_variables() #THIS IS ORIGINAL PLACE WHICH WORKS
    # initialized_variable_names = {}
    # scaffold_fn = None
    # if init_checkpoint:
    #   (assignment_map, initialized_variable_names
    #   ) = modeling.get_assignment_map_from_checkpoint(tvars, init_checkpoint)
    #   if use_tpu:
    #
    #     def tpu_scaffold():
    #       tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
    #       return tf.train.Scaffold()
    #
    #     scaffold_fn = tpu_scaffold
    #   else:
    #     tf.train.init_from_checkpoint(init_checkpoint, assignment_map)

    tf.logging.info("**** Trainable Variables ****")


texts: Can elastic net l1 ratio be greater than 1?

texts:
I have multiple datasets that I trained with ElasticNetCV (sklearn), and I noticed that many of them selected l1_ratio = 1 as the best value (which is the max value tried by the CV),
So as a test I wondered if values greater than 1 will produce a better result - and surprisingly the answer is yes... in fact you can reproduce this phenomenon with this code:
    from sklearn.linear_model import ElasticNet
    from sklearn.model_selection import train_test_split

    n = 200
    features = np.random.rand(n, 5)
    target = np.random.rand(n)+features.sum(axis=1)*5

    train_feat, test_feat, train_target, test_target = train_test_split(features, target)

    cls = ElasticNet(random_state=42, l1_ratio=1, alpha=0.1)
    cls.fit(train_feat, train_target)
    print(cls.score(test_feat, test_target), cls.score(train_feat, train_target))

    cls = ElasticNet(random_state=42, l1_ratio=1.1, alpha=0.1)
    cls.fit(train_feat, train_target)
    print(cls.score(test_feat, test_target), cls.score(train_feat, train_target))

And you will find that the l1_ratio=1.1 regressor is better on both train and test.
According to the documentation, you shouldn't use l1_ratio>1, but it does technically work. However it doesn't make much sense, as it would mean that the L2 part of the loss function becomes negative - so higher L2 values of the coefs don't punish, but in fact reward (!) the loss function.
Is there any theoretical logic behind this? Is there any reason not to expand the L1 search range to $[0,2]$ instead of $[0,1]$?


texts: Is Faster RCNN the same thing as VGG-16, RESNET-50, etc... or not?

texts: My understanding is that Faster RCNN is an architecture for performing object detection. It finds objects in an image and classifies them. My understanding is also that VGG-16, RESNET-50, etc... also find objects in images and classify them. Are VGG-16, RESNET-50, etc... object detection architectures in the same way that Faster RCNN is? I often see VGG-16, RESNET-50, etc... as the "backbone" for Faster RCNN and am seriously confused by the literature. Thanks in advance!


texts: Is using cross-entropy enough to ensure the output is a distribution probability?

texts:
I am following along https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html.

In this code, the last layers of the pretrained networks are linear. The loss used in this classification problem is the cross entropy.

I understand why cross entropy would be a good idea, given that the output of the layer is a probability distribution. However, if the last layer is linear, there is no guarantee that the output can be treated as such.

Hence the question : does cross entropy loss naturally lead the output to converge towards a probability distribution during the optimization process, or is there another way to interpret cross entropy when calculated over something that's not a probability distribution ?


texts: Dataset for simultaneous object detection, 6D pose estimation and grasp detection

texts:
I am trying to find a dataset which meets any of the following requirements.

Dataset of RGB or RGBD images with object bbox annotations, 6d pose annotations and grasp annotations.
Dataset of RGB or RGBD images with object bbox annotations and grasp annotations.

I have worked with Cornell grasp dataset and linemod 6d object pose estimation dataset separately.
I am trying to build a unified model that does all these tasks together and trying to find available dataset for the same.


texts:
I used your Open IE 5 to extract the triples and got the following result,

Text Input
By the algorithmic approach known as LevenbergMarquardt backpropagation algorithm, the error is decreased repeatedly. Some ANN models employ supervisory training while others are referred to as none-supervisory or self-organizing training. However, the vast majority of ANN models use supervisory the supervisory training. The training phase may consume a lot of time. In the supervisory training, the actual output of ANN is compared with the desired output. The training set consists of presenting input and output data to the network. The network adjusts the weighting coefficients, which usually begin with random set, so that the next iteration will produce a closer match between the desired and the actual the actual output of ANN. The training method tries to minimize the current errors for all processing elements. This global error reduction is created over time by continuously modifying the

Output

0.89 Context(The training method tries,List([723, 748))):(The training method; tries to minimize; the current errors for all processing elements)
0.95 (the vast majority of ANN models; use; supervisory the supervisory training)
0.88 (others; are referred; as self - organizing training)
0.89 Context(The training method tries,List([717, 742))):(The training method; tries to minimize; the current errors for all processing elements)
0.93 Context(Some ANN models employ The training phase may consume,List([120, 340))):(the error; is decreased; T:repeatedly; T:By the algorithmic approach)
0.94 Context(The training phase may consume,List([310, 340))):(Some ANN models; employ; supervisory training; while others are referred to as self - organizing training)
0.89 Context(The training method tries,List([724, 749))):(The training method; tries to minimize; the current errors for all processing elements)
0.93 Context(The training phase may consume,List([311, 341))):(the vast majority of ANN models; use; supervisory the supervisory training)
0.93 Context(Some ANN models employ The training phase may consume,List([120, 341))):(the error; is decreased; T:repeatedly; T:By the algorithmic approach)
0.94 Context(The training phase may consume,List([311, 341))):(Some ANN models; employ; supervisory training; while others are referred to as none - supervisory training)
0.92 (This global error reduction; is created; T:over time; by continuously modifying the)

Can anyone please help me understanding,

What is  List([723, 748))):
T:over time;
In some case it has 4 entities, (the error; is decreased; T:repeatedly; T:By the algorithmic approach)


texts: Trajectory data mining and pattern recognition using ORB-SLAM and KNN-DTW

texts:
I've started working on a project about the trajectory data mining from videos - for example: snowboarding video from GoPro action camera.

This is the continuation of my previous experiment (MotionML), which was based on the accelerometer datasets and KNN-DTW classification: https://github.com/llvll/motionml

Visual odometry and monocular SLAM topics have been researched with the following algorithms:

PTAM
DTAM
LSD-SLAM
ORB-SLAM

At this stage I've selected ORB-SLAM algorithm for the initial experiments: https://github.com/raulmur/ORB_SLAM

The implementation will be done in Python using Scikit-Learn and FastDTW for the model training purposes based on the trajectory data. The data mining itself, based on OpenCV and ORB-SLAM, will execute on Apache Spark for video file processing.

The following results are expected:

Extracted trajectory data.
Discovered and classified motion patterns - similar to MotionML project above.
Video georeferencing or spatio-temporal referencing - GPS geotagging might be present for frame #1, trajectory data will extrapolate geotagging to the next frames. Clock synchronization should take place.

Two questions from my side:

DTW for trajectory classification - I'm still doubting about DTW and reviewing HMM (Hidden Markov Models) or LSTM neural nets for the same purpose. Any practical advices are really appreciated.
I'm also looking for any examples or advices about SLAM-based feature extraction for machine learning. Ideally for trajectory data mining, but other examples will work as well.

If you would like to cooperate or to participate in this project, then please write me directly: oleg.v.puzanov@gmail.com


texts: What applications does linear programming have in data science?

texts: I'm currently learning about linear programming in my degree. I'm wondering how this is relevant to anything in data science?


texts:
I'm working on an implementation of a neural network so I can really grasp how these magic boxes work. However the neural network I've written code for doesn't work and I think it's due to my implementation of backpropagation

[pseudo code]

'last layer only
for each node (i) in last layer
error(i) = outputError(i)
end for

'all hidden/inner layers
for each layer (i) in layers (not including layer 0 or last)
 for each neuron (j) in layer i
  for each neuron (k) in layer i+1
   error(i)(j) +=error(i+1)(k) * weight(i)(j,k)
  end for

  errors(i)(j) = errors(i)(j) * activation(i)(j) * (1-activation(i)(j))

  for each neuron in layer i+1
   deltaWeight(i)(j,k) = -error(i)(k)*activation(i)(k)*learningRate/NumExamples
   deltaBias(i)(k) = -error(i)(k) * bias(i)(k)*learningRate/NumExample
  end for
 end for
end for

Then I update all the weights with this:

weight(i)(j,k) = weight(i)(j,k) + deltaWeight(i)(j,k)

Now the problem I have is that the output doesn't seem to get any better. It certainly changes but it doesn't seem to minimise the cost function at all, does anyone know why?


texts: Spatial Transformer Networks and Data Augmentation

texts:
We are all familiar with the famous Deep Mind paper STN.
Upon implementation, such as here, did anyone still use input data augementation such as affine transformations?
There are used to make CNN robust to rotation, transformation etc. I wonder how these/lack of these affect the output of STNs.


texts: How to estimate total correlation KL[q(z)||Πjq(zj)] of VAE after training (useful for latents disentanglement evaluation)

texts:
FactorVAE and β-TCVAE both use total correlation (TC) batch estimation for their objectives. Where TC is:

$$
KL\bigl( q(z)||\prod\nolimits_{j} q(z_{j})\bigr)
$$

both estimates are applied to $q(z|x)$ of the encoder during training. Where $z∼q(z|x)$ is sampled from diagonal Gaussian with $μ(x)$ and $σ(x)$ given by encoder. FactorVAE TC estimate is hard to train and I failed to converge it for my dataset. β-TCVAE converges and gives meaningful latents if we consider $μ(x)$ only. But it has such high $σ(x)$ that $z∼q(z|x)$ becomes meaningless (I need disentangled latents to contain info about some polar cases of data - the info is contained in $μ(x)$ but is lost when sampling $z∼q(z|x)$ - I checked the info presence by training a simple NN classifier).

I have not idea how to fix it so I simply want to separately estimate TC of the $q(μ)$ using the whole train dataset (if it's good then it's OK for my needs). FactorVAE paper mentions a straghtforward way of TC estimation in appendix D:

$$
KL\bigl( q(z)||\prod\nolimits_{j} q(z_{j})\bigr) = _{q(z)} \left[{{\mathrm{log}} \frac{q(z)}{\prod_{j}q(z_{j})}}\right] ≈ _{q(z)} \left[{{\mathrm{log}} \frac{\hat{q}(z)}{{\prod_{j} \hat{q}(z_{j})}}}\right]\\
≈ \frac{1}{H} \sum_{h=1}^H \left[{{\mathrm{log}} \frac{1}{|{\mathcal{B}}|} \sum_{i∈{\mathcal{B}}}\prod_{j=1}^D q(z_{j}^{(h)}|x^{(i)})
- {\mathrm{log}} \prod_{j=1}^D \frac{1}{|{\mathcal{B}}|} \sum_{i∈{\mathcal{B}}} q(z_{j}^{(h)}|x^{(i)})}\right]
$$

where $z^{(h)} \overset{{iid}}{∼} q(z)$ and $q(z) ≈ \hat{q}(z) = \frac{1}{|{\mathcal{B}}|} \prod\nolimits_{i∈{\mathcal{B}}} q(z|x^{(i)})$.

But first:

I cannot sample $H$ samples as I want to estimate TC of $q(μ)$ and hence I can only use deterministic mapping $μ(x)$.

And second they also say that:

  From Figure 18, we can see that even with B as big as 10,000, we get negative values for the estimate of TC, which is a KL divergence and hence should be non-negative, hence this method of using a batch estimate for $q(z)$ does not work.

I only have ~10000 samples in my database so I might end implementing it and having negative KLD too (if I go with $H=1$).

Hence my question: what is the best way to estimate TC of some database samples (as I want to estimate $μ(x)$ that is a deterministic function of the database). Is it even possible? Are there better ways than mentioned above? What can you say about failure of the estimation method from FactorVAE paper?


texts:
I'm new to Named Entity Recognition and I'm having some trouble understanding what/how features are used for this task.

Some papers I've read so far mention features used, but don't really explain them, for example in
Introduction to the CoNLL-2003 Shared Task:Language-Independent Named Entity Recognition, the following features are mentioned:

  Main features used by the the sixteen systems that participated in the
  CoNLL-2003 shared task sorted by performance on the English test data.
  Aff: affix information (n-grams); bag: bag of words; cas: global case
  information; chu: chunk tags; doc: global document information; gaz:
  gazetteers; lex: lexical features; ort: orthographic information; pat:
  orthographic patterns (like Aa0); pos: part-of-speech tags; pre:
  previously predicted NE tags; quo: flag signing that the word is
  between quotes; tri: trigger words.

I'm a bit confused by some of these, however. For example:

isn't bag of words supposed to be a method to generate features (one for each word)? How can BOW itself be a feature? Or does this simply mean we have a feature for each word as in BOW, besides all the other features mentioned?
how can a gazetteer be a feature?
how can POS tags exactly be used as features ? Don't we have a POS tag for each word? Isn't each object/instance a "text"?
what is global document information?
what is the feature trigger words?

I think all I need here is to just to look at an example table with each of these features as columns and see their values to understand how they really work, but so far I've failed to find an easy to read dataset.

Could someone please clarify or point me to some explanation or example of these features being used?


texts: How to rotate the plot and find minimum point?

texts:
I want to rotate the below curve to 45 degree and then find the minimum point.

For this, I have tried with below code:

def rotate_vector(data, angle):
    theta = np.radians(angle)
    co = np.cos(theta)
    si = np.sin(theta)
    rotation_matrix = np.array(((co,-si), (si, co)))
    return np.matmul(rotation_matrix, data)

rotated_vector = rotate_vector(data, -45)
elbow = rotated_vector.min()

But what I get is this curve:


texts:
I'm trying to train a model to distinguish between two kinds of time signals, those with RTS noise, and those with only white noise.

I have a simple 1D CNN that works well (92% accuracy) with one training set, but turns into a complete coin flip with another. To the eye these sets are very similar. One was created using real signals, the other with simulated signals. The only real difference to me is the mean magnitude. Is there a reason the model fails so reliably with this second set? Do I need to normalize the data somehow?

from keras.models import Sequential
from keras.layers import Dense, Dropout,Activation
from keras.layers import Embedding
from keras.layers import Conv1D, GlobalAveragePooling1D, MaxPooling1D,
Flatten, LSTM
import numpy as np

x_test = np.load('C:/Users/Ben WORK ONLY/Desktop/GH repos/RTS ML detect
beta/x_test.npy')
x_train = np.load('C:/Users/Ben WORK ONLY/Desktop/GH repos/RTS ML detect
beta/x_train.npy')
y_test = np.load('C:/Users/Ben WORK ONLY/Desktop/GH repos/RTS ML detect
beta/y_test.npy')
y_train = np.load('C:/Users/Ben WORK ONLY/Desktop/GH repos/RTS ML detect
beta/y_train.npy')
X_train = np.expand_dims(x_train, axis=2)
X_test = np.expand_dims(x_test, axis=2)

model = Sequential()
model.add(Conv1D(32, 12, activation='relu', input_shape=(1500, 1)))
model.add(MaxPooling1D(3))
model.add(Conv1D(64, 12, activation='relu'))

model.add(MaxPooling1D(3))
model.add(Conv1D(128, 12, activation='relu'))
model.add(GlobalAveragePooling1D())
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
          optimizer='rmsprop',
          metrics=['accuracy'])

model.fit(X_train, y_train, batch_size=16, epochs=5)
score = model.evaluate(X_test, y_test, batch_size=16)

#model.save('C:/Users/Ben WORK ONLY/Desktop/GH repos/RTS ML detect
beta/CNNlin_model.h5')

The shapes of the two train/test sets:


texts: Comparing training and validation data set Root MSE for a best subset regression?

texts:
I’ve a model with 14 dependent variables (all of them are significant) and 678 observations. I used best subset regression and validation set (33% of data for the validation) to find which statistical model has the lowest MSE (for my curiosity). I got the following graph which surprisingly the MSE for validation data set is always lower than training data set for all the models (from 1 to 14 dependent variables).
Here is the code that I used,

library(MASS)
set.seed(1)
train=sample(seq(678),452,replace=FALSE)
train

regfit.exh=regsubsets(HPV~. -Model.Types..code.-Year..code.,data=Mydata, nvmax=NULL,force.in = NULL, force.out = NULL, method="exhaustive")

val.errors=rep(NA,14)

x.test=model.matrix(HPV~.-Model.Types..code.-Year..code.,data=Mydata[-train,])
for(i in 1:14){
  coefi=coef(regfit.exh,id=i)
  pred=x.test[,names(coefi)]%*%coefi
  val.errors[i]=mean((Mydata$HPV[-train]-pred)^2)
}

plot(sqrt(val.errors),ylab="Root MSE",ylim=c(3,12), pch=11, type="b")
points(sqrt(regfit.exh$rss[-1]/452),col="blue",pch=11,type="b")
legend("topright",legend=c("Training","Validation"),col=c("blue","black"),pch=11)

How come the validation root MSE could always beat the training?. Any feedback would be appreciated.


texts:
I am using $\chi^{2}$ kernel for non-linear SVM (using libSVM) for classifying MNIST digits. I am getting very bad performance (worse than random guessing). The $\chi^{2}$ kernel code (in MATLAB) is as follows:

function D = chi2_kernel(X,Y)
%Computes the chi2 kernel for two Matrices X and Y

 D = zeros(size(X,1),size(Y,1));

for i = 1:size(Y,1)

    d = bsxfun(@minus,X, Y(i,:));
    s = bsxfun(@plus,X,Y(i,:));
    D(:,i) = sum(d.^2 ./ (s/2),2);
end

D = exp(-0.0001.*(1-D));

end

Using this I computed the kernel matrix as follows :

% Gram matrix for chi2 kernel
 dist = chi2_kernel(X_train2,X_train2);
dist1 = chi2_kernel(X_test,X_train2);

% Gram matrix for training set to training set

KTrT = [(1:numTrain/50)', dist];

% Gram matrix for training to test set

KTT = [(1:numTest)', dist1];

model = svmtrain(Y_train2, KTrT,'-t 4');

predClass = svmpredict(Y_test, KTT,model);

Does anyone have suggestions for how to better optimize this code, or an explanation for why I'm observing such poor performance?


texts: optimizing customization: beyond a combinatorial approach?

texts:
Say I have a generic non-linear model (ANN, Random Forest, Gradient Boosting, etc) that wants, based on a set of features (price of a product, service duration, age, etc), to give me a prediction of whether a client will buy my product and clearly I can change my "offer" (say a combination of price and service duration).
I want to find the combinations of such factors, that is, the best compination per user, of my offer so that I can maximize the chance of each of my clients will buy my product/service. of course, I can do this in a "combinatorial" approach (trying each combination and see which one yields me the highest implied probability of success) but combinations could be infinite depending on how I set it up or would not scale when users increase considerably.
Since it's impossible that this problem was not encountered before - and I have neither the intention nor the need to re-invent the wheel, could someone point to some algorithms/topics that adress this issue?


texts: Are there any good solutions for putting a radial basis kernel support vector machine into production?

texts: Are there any good options for a radial basis kernel SVM where I can serialize the model to store and later deserialize and evaluate? I'm using H2O for some other things and it supports SVM but no radial basis kernel.


texts: Visualising K-Means clusters for 3D data in R

texts:
I have an excel file that contains 485k rows x 3 columns of integer values. Sample data:
cluster.x    cluster.y    cluster.z
1            3            3
3            1            4
4            2            5
...

I did a kmeans clustering with 6 clusters, but I am figuring out what is the best way to display the result. I plotted it using plot3d() as shown below, but it is hard to differentiate different clusters since the values are all integers.
Does anyone have any suggestions? If possible, I also want it to be able to visualise how many data points there are in each cluster (e.g a bubble plot where the size of the bubble depicts the number of data points).


texts: Why does it speed up gradient descent if the function is smooth?

texts:
I now read a book titled "Hands-on Machine Learning with Scikit-Learn and TensorFlow" and on the chapter 11, it has the following description on the explanation of ELU (Exponential ReLU).

  Third, the function is smooth everywhere, including around z = 0, which helps speed up Gradient Descent, since it does not bounce as much left and right of z = 0.

The z means the x-axis on the graph above. I understand the derivative is smooth since the z < 0 line has a curve and in that realm the derivative is no longer equal to 0.

However, why is it the case that if the function is "smooth everywhere, including around z=0", it speeds up Gradient Descent?


texts: How can I picture an unfolded RNN as a normal Feed Forward Network?

texts:
I am currently working on a Transformer architecture.
Trying to picture an RNN (or Encoder) as a normal Feed Forward network really confused me after looking at the following image in an article:
(Image 1)

I am usually used to seeing it like this:
(Image 2)

In Image 1, it shows that the input goes in ALL at once, whereas in Image 2, we see only a single input at a timestep.
My two questions:

Does this mean that Image 1 is a single node in Image 2?

How can I picture an RNN-architecture as Image 1?


texts: PCA graph of individuals from principal rather than PCA

texts:
I am trying to run PCA in R, I have to do it with rotation and without rotation. Then, I need to plot the individuals graph.
Rotation is available in psych::principal (documentation), but there are no functions that create an individual graph from psych::principal output.
On the other hand, I can create an individual plot with FactoMineR::PCA (documentation), but FactoMineR::PCA cannot perform rotation.
Can I have something that combines both, rotation option vs not and graph of individuals?


texts: What is the relation between the features of word-embeddings and the type of classifiers (linear or non-linear)?

texts: I am doing an analysis on the results of classifiers (such as: linear SVM, RBF, Random forest, naive bayes and logistic regression) of a text classifcation problem using word-embeddings (I am using Glove). I would like to see if the features of word-embeddings are linear and this has an influence on the performance with a linear or non-linear classifier?


texts: Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight

texts:
My questions follow the below page 4 excerpt from Hochreiter's LSTM paper:

  If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal
  value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ is constant and
  not equal to zero, then
  $|f^\prime_{l_{m}}(net_{l_{m}})w_{l_{m}l_{m-1}}|$ takes on maximal
  values where

  $w_{l_{m}l_{m-1}} = {1 \over y^{l_{m-1}}} \coth \left( {1 \over 2}net_{l_{m}} \right)$,

  goes to zero for $|w_{l_{m}l_{m-1}}| \rightarrow \infty$, and is less
  than 1.0 for $|w_{l_{m}l_{m-1}}| < 4.0$.

The derivative of the sigmoid $f_{l_{m}} =  f^\prime_{l_{m}} = \sigma$, is $\sigma(1-\sigma)$, so of course its maximum value is 0.25.

But I don't understand the following:

Where does the $\coth$ come from?
Why would $y^{l_{m-1}}$ be static? My understanding is that it is the activation of the unit, which changes each timestep.
Multiplying a non-zero positive number, which $f^\prime_{l_{m}}$ could be, by $|w_{l_{m}l_{m-1}}|$ as it approaches infinity would make it approach infinity, not zero. What am I missing here?

Thank you in advance for your help.


texts: Identifying which known groups are the most similar or most dissimilar

texts:
I have a data set of 5 groups and their associations to different interests. The data is structured as follows with 2000+ interests and 5 known groups:

      Interest     Group1     Group2     Group3     Group4     Group5
01.   Sports         10%        40%        30%       80%         65%
02.   Music          35%        60%        55%       10%         15%
03.   Fashion        80%        10%        75%        5%         25%
 .
 .
 .
1998.  Movies        70%        20%        45%       50%         65%
1999.  Books         50%        70%        35%       65%         95%
2000.  Art           90%        50%        25%       15%         30%

I would like to gain an understanding of what would be the best way to determine which groups are closest to each other and which ones are the most different from each other in terms of interests.

I am leaning towards cluster analysis, however the structure of my data is throwing me off.

Any suggestions would be greatly appreciated.


texts: When should I use normal Q learning over a DQN?

texts:
From this article here, it says that using a tabular Q function is less scalable than a deep Q network. I assume that this means that the Q table approach works for some environments, but once they become more complex, they aren't as efficient.
For example, the Frozen Lake environment used in the article states that the deep Q network is slower than the Q table. The Frozen Lake environment has a relatively simple environment with 16 states and 4 actions per state. However, in an environment such as a game of snake, there are many more states, making the Q table larger. How should I decide between a Q table and a Deep Q Network?


texts: I have some experience with data science but I wanted some insight on how to deal with a very large dataset. I understand simply downloading it to your computer is not plausible so where do you even begin? Say for example I want to download a particular dataset from some website that is over 10gb.


texts: Algorithm for clustering worm-like points in a cloud

texts:
I have a cloud of points were I can clearly identify worm-like patterns in the cloud, along the Z-axis but embedded in the noise.
My goal is to cluster the points that belong to the same "worm'. Sometimes there are 2 chains close to each other and I would also like to separate them.
Can you suggest any algorithm for this? I have tried DBSCAN and t-sne without success. I am trying to do it in MATLAB ideally.
Here is the cloud attached https://pastebin.com/wp4N4JHV.
Thanks


texts: How to find combined feature contributions in a random forest algorithm

texts: I am currently working on a pancreatic cancer dataset which has numerous features including smoking, family history, age, etc. I have used the feature_importances_ method in sklearn to find and plot the individual feature importances, but wanted to know how to find the importance of combined variables (e.g. are those who smoke and have a family history of pancreatic cancer much more likely than others?). I know it is not as simple as adding the individual feature contributions for the combined features, and wanted to know if there is a specific method/function that can be used to do that. I have also used treeinterpreter to find the feature contributions.


texts: Resize image in object detection task of computer vision

texts:
In object detection, they usually resize by keeping the ratio the same as the original image, which usually names "letterbox" resize.
My question is:

Why do we need to do that? As I see with some images too long in vertical or horizontal, we will lose a lot of features in those images.

If it is better than normal resize, why people don't apply it in the classification task?


texts: Methods to generalise NCF recommender systems to unseen users, same set of items?

texts:
I'm new to recommendation models, and am starting to build a recommender system on the MovieLens dataset using NCF-style model.
As I'm building it I'm wondering if, once trained, I can apply it to my own movie ratings - although my user ID doesn't exist and isn't in the train dataset.
More generally, if one had deployed a recommender system for movies to production and a new user (not in training) has rated a few of the movies the system knows about, how can we generate predictions for this user based on their ratings history?
The way this model works as far as I understand, the input shape must be equal to the number of users multiplied by the number of items and contain one 1 in each, otherwise 0s (something like [0, ... 0,1] and [0,1,0,...,0]). This means you can only ever predict recommendation scores for users in your train dataset, since there are only n_users distinct ways you can choose the user vector. To be clear I am only talking about a new user who already has ratings on items within the specified item list.
Is the only workaround for this to work out what users the new user (e.g. myself) is similar to in the train data, maybe via cosine similarity, then apply a weighted average recommendation score for a new movie over the "relevant" users? Is there anything more sophisticated than this?
It seems to me like a big limitation of the framework if you can only run the model on users you've already seen before - so I'm hoping someone can shed some light for me as I'm quite confused!


texts:
I tried the sample code form the link below.
Get started with TensorBoard | TensorFlow
It pretty much worked fine; just had to make a couple tweaks to get it working on my machine. I was a little surprised to find that it ran just like normal Python code. I thought TensorFlow pops up an interactive GUI, called TensorBoard, and I thought you can interact with this visual interface. When I run the code from the link above, all I have is a command line interface. There is no TensorBoard, no graphics, no GUI, nothing but Python code. Is there a special command that need to be executed to make the TensorBoard work? Is there some additional step that I need to run? I'd appreciate any/all guidance on this.


texts: how to make decision based on users reports

texts:
I have users reports about an accident, i want to know how to make sure that the number of reports is enough to take that accident as a true accident not a spam.

My idea is to consider a minimum number of reports in a specific time interval, for example 4 reports in 20 minutes are good enough to believe the existence of that accident.

My question is how can I choose the number of minimum reports and that time interval? Is there another logic to take that decision?
I will appreciate your answers .


texts: Handling multiple data types in Naive Base

texts:
I was studying about NB Classifier and it came out to me that i can use Bernoulli NB or Multinomial NB for categorical variables but Gaussian NB for numeric variables. i have a dataset in which i have a mixture of both.

annual_income is my categoric variable.

I did some research about how to handle it but still i am not very clear. Some options:

To fit categorical data with Bernoulli NB and numerical data with Gaussian NB. After that taking the probabilities using predict_proba from both models, fit another model such as Gaussian NB. But as far as i know distribution of data should be gaussian in Gaussian NB, but the probabilities from predict_proba does not take gaussian distribution.
multiply probabilities from Bernoulli NB and Gaussian NB and predict on the result set.

Which one should be appropriate to use and how they will produce different results? Can i even use these methods?


texts:
I recently read Fully Convolutional Networks for Semantic Segmentation by Jonathan Long, Evan Shelhamer, Trevor Darrell. I don't understand what "deconvolutional layers" do / how they work.

The relevant part is

  3.3. Upsampling is backwards strided convolution

  Another way to connect coarse outputs to dense pixels
  is interpolation. For instance, simple bilinear interpolation
  computes each output $y_{ij}$ from the nearest four inputs by a
  linear map that depends only on the relative positions of the
  input and output cells.
  In a sense, upsampling with factor $f$ is convolution with
  a fractional input stride of 1/f. So long as $f$ is integral, a
  natural way to upsample is therefore backwards convolution
  (sometimes called deconvolution) with an output stride of
  $f$. Such an operation is trivial to implement, since it simply
  reverses the forward and backward passes of convolution.
  Thus upsampling is performed in-network for end-to-end
  learning by backpropagation from the pixelwise loss.
  Note that the deconvolution filter in such a layer need not
  be fixed (e.g., to bilinear upsampling), but can be learned.
  A stack of deconvolution layers and activation functions can
  even learn a nonlinear upsampling.
  In our experiments, we find that in-network upsampling
  is fast and effective for learning dense prediction. Our best
  segmentation architecture uses these layers to learn to upsample
  for refined prediction in Section 4.2.

I don't think I really understood how convolutional layers are trained.

What I think I've understood is that convolutional layers with a kernel size $k$ learn filters of size $k \times k$. The output of a convolutional layer with kernel size $k$, stride $s \in \mathbb{N}$ and $n$ filters is of dimension $\frac{\text{Input dim}}{s^2} \cdot n$. However, I don't know how the learning of convolutional layers works. (I understand how simple MLPs learn with gradient descent, if that helps).

So if my understanding of convolutional layers is correct, I have no clue how this can be reversed.

Could anybody please help me to understand deconvolutional layers?


texts: Problem in creating a Numpy Dataset for Classification problem

texts:
I am trying to train a classification Model to classify Images, but my images here are Numpy arrays of 1's and 0's as shown in the Image,

For now, I have tried,

x1_data=[]
def create_array(columns=5,rows=5,randomness=.3):

    board = np.zeros([rows,columns],dtype='int64')

    for i in range(rows):
        for j in range(columns):
            if np.random.random() <= randomness:
                board[i,j] = 1
    return board

for i in range(5):
    x1_data.append(create_array())

which gives me 5 arrays `

array([[[0, 0, 1, 0, 0],
        [0, 0, 1, 0, 0],
        [1, 0, 0, 0, 0],
        [0, 0, 0, 0, 1],
        [0, 0, 0, 0, 1]],

       [[0, 0, 1, 0, 0],
        [0, 0, 0, 1, 0],
        [0, 0, 0, 1, 0],
        [0, 0, 0, 1, 0],
        [0, 1, 0, 0, 0]],

       [[1, 0, 0, 1, 0],
        [1, 0, 0, 0, 0],
        [0, 1, 0, 0, 0],
        [0, 0, 0, 0, 0],
        [0, 0, 1, 0, 0]],

       [[0, 0, 0, 0, 0],
        [1, 0, 0, 1, 1],
        [0, 0, 1, 1, 1],
        [0, 0, 0, 0, 0],
        [0, 0, 1, 0, 1]],

       [[0, 0, 0, 0, 1],
        [0, 0, 0, 0, 0],
        [0, 0, 0, 1, 0],
        [0, 1, 1, 0, 0],
        [0, 0, 0, 0, 1]],

       [[1, 0, 0, 1, 1],
        [0, 0, 1, 1, 0],
        [0, 0, 1, 0, 0],
        [1, 0, 0, 0, 0],
        [1, 1, 0, 0, 0]]]

For class A, similarly I generated 5 arrays for class B and then stacked them as a single X array.

My output y is:

y = np.random.choice([0, 1], size=(10,), p=[1./3, 2./3])
array([0, 1, 1, 1, 1, 0, 0, 0, 0, 0])

How do I preprocess this Data to train on an MLP in sklearn
I am getting this error

clf = MLPClassifier(solver='lbfgs', alpha=1e-5, hidden_layer_sizes=(5, 2), random_state=1)
clf.fit(g, h)
#Error
Found array with dim 3. Estimator expected <= 2.


texts: what does one Shot learning mean? do they only need one image to train for some new class detection?

texts:
Being new to deep learning I am somewhat struggling to grasp the idea of one shot learning.
Let us say I have a class to detect which didn't exist in training dataset such as COCO or Image NET. Can I train model for that class using only image or the training set must be large as for YOLO or RCNNs?


texts: Meltdown patch impact on data processing speeds

texts: The patch for the Meltdown vulnerability disables speculative execution, which will impact all processing activities. The degree of impact is highly dependent on the type of processing being done. Is there hard data or experience of how machine learning and data processing will be impacted in measurable terms?


texts:
I have a data set with 60000 rows and 32 columns. I want to use SVM (with some more constraints that make it more complicated)and I think 32 columns are too large. So I decided to use PCA. But when I load PCA, the first 20 component describe 85% of data which 20 variables still is too large I think, but it is better than 32.
I am wondering is that ok if I use PCA? when n>>m? if not what is a better option?


texts: Transfer learning VGG16 does not work as expected. (Detect tacos as hamburgers)

texts:
I am new in this field of machine learning, to test I wanted to do a simple project. Create a cnn capable of recognizing hamburger images. As I do not have the ability to collect more than 10,000 images of hamburgers I have used an existing model to train what I need ... I have used VGG16.

This is the notebook I am using, to do transfer learning. I am training the model in detecting cats and hamburgers.
https://github.com/EsteveSegura/detecting-food/blob/master/src/BurgerCNN.ipynb

If the notebook does not load, the entire code is also here
https://gist.github.com/EsteveSegura/2be156ba5431fc42fb8ac13eb0506c82

Train

Cat: 1024 imgs

Burger: 1097 imgs

Validation

Cat: 416 imgs

Burger: 326 imgs

When I try to predict photos of hamburgers, it usually gives a good result ... But it also detects as burger photos of tacos (99% acuraccy). Is it an overfitting problem? Do I need more images? Am I doing the transfer learning in a correct way?

My goal will be to detect if there is a hamburger in the photo or not.

One of the recommendations they have given me is to train a third class with completely random objects that do not repeat ... Can this help? In case I can help? How?

This is the log of my training


texts: Hadoop and input informations divided in splits

texts:
Hadoop divides the input to a MapReduce job into fixed-size pieces
  called input splits, or just splits. Hadoop creates one map task for
  each split, which runs the user-defined map function for each record
  in the split. Having many splits means the time taken to process each
  split is small compared to the time to process the whole input. So if
  we are processing the splits in parallel, the processing is better
  load balanced when the splits are small

Why ?


texts: Predicting three related scores with one joint model

texts:
I have multiple features and I want to predict three outcome scores.

Features:

Length in cm smallest is 40cm biggest is 209cm
Kilo: 39 till 302
Age: 19 till 111
Gender: Male, female, transgender
Diagnoses: numbers of different diseases
Medicines: numbers of different medicines
Urine level: 0 till 5

Scores:

Scale of Happy 1 till 7
Scale of depression 1 till 7
Scale of health 1 till 7

I know I can use supervised learning and create models where I predict the scores individually. I already pre-trained three different models.

Do you have some feedback?

Happiness, depression and health are  related. Are there algorithms that can deal with this? Based on historical data (the features and predictions) and the trained model I want to predict the scores for new patients based on their features and historical scores.


texts: What's the difference between data classification and clustering (from a Data point of view)

texts:
What are the differences and the similarities between data classification (using dedicated distance-based methods) and data clustering (which has certain defined methods such as k-means)
Is data classification a sub-topic of data clustering ?


texts:
I installed Orange via conda install orange3

I see the package installed in Anaconda3 / pkgs directory however there is not desktop icon nor can I find Orange via Window start.


texts: Finding parameters with extreme values (classification with scikit-learn)

texts:
I am currently working with the forest cover type prediction from Kaggle, using classification models with scikit-learn. My main purpose is learning about the different models, so I don't pretend to discuss about which one is better.

When working with logistic regression, I wonder if I need the 'penalty' parameter (where I can choose L1 or L2 regularization). Based on what I found, these regularization terms are useful to avoid over-fitting, specially when the parameter values are extreme (by extreme I understand the range of some parameter values are very large compared to other parameters, Correct me if I am wrong. In this case, wouldn't it be enough to apply log-scale or normalization to these values?).

The main questions are: as the number of parameters is large, are there visualization techniques and tools in scikit-learn which can help me to find parameters with extreme values? is there any statistical function/tool which returns how extreme the values of parameters are?


texts: I am new to Machine Learning. Recently I am trying to build a model to predict the sales of a particular automobile make and model of a dealership and to which location. The data given to me has information about the dealer name, state, zip code, owner state, zip code, vehicle make, model, year, vehicle segment, body type, odometer reading, new/used indicator. How do I build a model to do predictive analysis?


texts: Importing Orange Pickled Models in Jupyter

texts:
I am doing some testing with geoscience data and a few supervised methods in Orange. I have been saving these models as pickle models, but I would like to import them in my Jupyter notebook. How can I do this? Do I need to import a specific library?

Thanks.


texts: Can't understand this simple matrix multiplication in python

texts:
Ok guys, I might be very tired here, but I can't figure out why this matrix multiplication by a scalar gives the following result (python)

  Matrix named 'dx'
        [ 1.6,  3.6,  0.4, 14.4, 25.6],
        [10. , 10. ,  0.4, 14.4,  3.6],
        [ 0.4,  0. ,  0. ,  1.6, 10. ],
        [ 6.4,  0. ,  3.6,  1.6,  0.4],
        [14.4,  0. , 25.6,  0.4,  6.4]

  10 * dx, in python, gives
        [ -40.,  -60.,  -20., -120., -160.],
        [-100., -100.,  -20., -120.,  -60.],
        [ -20.,    0.,    0.,  -40., -100.],
        [ -80.,    0.,  -60.,  -40.,  -20.],
        [-120.,    0., -160.,  -20.,  -80.]

From what I understand, each member should be multiplicated by 10, but it's clearly not the case.

I'm using Python 3.6 & numpy

What Am I missing ?
Link to matrix multiplication by scalar , pretty basic stuff :
http://www.purplemath.com/modules/mtrxmult.htm

Thanks !


texts: BERT: it is possible to use it for topic modeling?

texts: I'm struggling to understand which are the full capabilities of BERT: it is possible to make topic modeling of text, like the one we can achieve with LDA?


texts: Multilabel Classification; which network design?

texts:
I have a hard time thinking about how I can build this network with the following problem:
I want to build a CNN to classify notes from sheet music. I have tried several models with and without transfer learning, with only one label for each image i.e. 'C quarter note' or 'B half note' (not exactly like that, but you get the idea). Now training the model is going fine, but classifying new images is still rubbish.
Now I want to try and split the network into two: one network for training the pitch (label 1) and one network for training the duration of the note (label 2). So each image has two labels. How can I do this best?

1) building and training two separate models and just combine the results.
2) build one multilabel network (you still get label1 x label2 amount of labels right?).
3) a network with two output layers, each with one label and one Softmax function. although I'm not sure how to construct this and if this is different from option 1.

My second question regarding transfer learning: the models trained on imagenet are entirely different from my own dataset, so my logic was to just freeze the first couple of layers for basic features and train the rest of the model, is this correct?

Cheers


texts:
I am trying to run LOOCV on my regression model. I tried to run it in r and encountered the following warning message:

Warning message in y - yhat:
"longer object length is not a multiple of shorter object length”

This is my model:

x=glm(x,data = full_data)
mse_loocv=cv.glm(full_data,x)
mse_loocv$delta

Variables used in glm are:

x->
target_deathrate ~ avganncount + avgdeathsperyear + incidencerate +
    medincome + popest2015 + povertypercent + studypercap + medianage +
    medianagemale + medianagefemale + percentmarried + pctnohs18_24 +
    pcths18_24 + pctbachdeg18_24 + pcths25_over + pctbachdeg25_over +
    pctunemployed16_over + pctprivatecoverage + pctempprivcoverage +
    pctpubliccoverage + pctpubliccoveragealone + pctwhite + pctblack +
    pctasian + pctotherrace + pctmarriedhouseholds + birthrate +
    Midwest + Northeast + South


texts: Over-fitting issue in a classification problem (unbalanced data)

texts:
I am working on a rare event (unbalanced target variable) classification problem using decision trees. My dataset comprises of 95% non-event and 5% minority (events) class.

I used decision tree over logistic regression because I had many categorical variables comparing to continuous variables. I get a  good performance for training data with the decision tree C5.0. However I get poor results for the new data. I use the confusion matrix as a measure of performance. Training model is over-fitting.

I did pruning to reduce the over-fitting caused by the decision tree.  I used the following code to build the model

Classifi_C5.0 <- C5.0(TARGET ~., , data = training_data_SMOTED, trails = 500,
                      control = C5.0Control(minCases = mincases_count,
                                            noGlobalPruning = FALSE))

I balanced the minority and majority class using the following code:

training_data_SMOTED <- SMOTE(TARGET ~ ., training_data,
                              perc.over = 100, k = 5, perc.under = 200)

Any sort of advice will be helpful.


texts: How to identify possible false negatives in labels?

texts:
I am working on a drilling difficulty prediction project using machine learning method. The dataset has lots of geological data as the main features. It also has total drilled depth of each hole.
Not surprisingly, the holes that are seeing drilling difficulty had significantly longer drilled depth than the ones that did not have difficulty. However, the geological difficulty should not be dependent on the drilled length:  Based on particular type of rock, difficulty is always there no matter if it is drilled or not. If we drilled the holes that didn't have difficulty longer, I am sure lots of them will see difficulty eventually.
Having said that, the labels probably have a lot of false negatives (because the difficulty hasn't been encountered yet). How do I identify these false negatives in the labels?


texts: How to explain the outcome of k-means clustering?

texts:
I am currently conducting some analysis using NTSB aviation accident database. There are cause statements for most of the aviation incidents in this dataset that describe the factors lead to such event.

One of my objectives here is to try to group the causes, and clustering seems to be a feasible way to solve this kind of problem. I performed the followings prior to the beginning of k-means clustering:

Stop-word removal, that is, to remove some common functional words in the text
Text stemming, that is, to remove a word's suffix, and if necessary, transform the term into its simplest form
Vectorised the documents into TF-IDF vector to scale up the less-common but more-informative words and scale down highly-common but less-informative words
Applied SVD to reduce the dimensionality of vector

After these steps k-means clustering is applied to the vector. By using the events that occurred from Jan 1985 to Dec 1990 I get the following result with number of clusters k = 3:

(Note: I am using Python and sklearn to work on my analysis)

... some output omitted ...
Clustering sparse data with KMeans(copy_x=True, init='k-means++', max_iter=100, n_clusters=3, n_init=1,
    n_jobs=1, precompute_distances='auto', random_state=None, tol=0.0001,
    verbose=True)
Initialization complete
Iteration  0, inertia 8449.657
Iteration  1, inertia 4640.331
Iteration  2, inertia 4590.204
Iteration  3, inertia 4562.378
Iteration  4, inertia 4554.392
Iteration  5, inertia 4548.837
Iteration  6, inertia 4541.422
Iteration  7, inertia 4538.966
Iteration  8, inertia 4538.545
Iteration  9, inertia 4538.392
Iteration 10, inertia 4538.328
Iteration 11, inertia 4538.310
Iteration 12, inertia 4538.290
Iteration 13, inertia 4538.280
Iteration 14, inertia 4538.275
Iteration 15, inertia 4538.271
Converged at iteration 15

Silhouette Coefficient: 0.037
Top terms per cluster:
**Cluster 0: fuel engin power loss undetermin exhaust reason failur pilot land**
**Cluster 1: pilot failur factor land condit improp accid flight contribute inadequ**
**Cluster 2: control maintain pilot failur direct aircraft airspe stall land adequ**

and I generated a plot graph of the data as follows:

The result doesn't seem like make sense to me. I wonder why all of the clusters contain some common terms like "pilot" and "failure".

One possibility that I can think of (but I am not sure if it is valid in this case) is the documents with these common terms are actually located at the very centre of the the plot graph, therefore they can not be efficiently clustered into a right cluster. I believe this problem cannot be addressed by increasing the number of clusters, as I have just done it and this problem persists.

I just want to know if there is any other factors that could cause the scenario that I am facing? Or more broadly, am I using the right clustering algorithm?

Thanks StackExchange and Data Science Portal.


texts: Comparing XGBR with CatBoost performance

texts:
I saw on a CatBoost site that it supposed to outperform any other boosted training model and decided to try it myself on a Kaggle's https://www.kaggle.com/c/house-prices-advanced-regression-techniques.

I created some basic kernel without any complex preprocessing, feature selection, GridSearch, stacking, etc... just to compare the XGBR and CatBoost performance. But as far as I see the XGBR always outperforms the CatBoost.
https://www.kaggle.com/markbquant/compare-catboostregressor-vs-xgbregressor

The CatBoost parameters: iterations=100, depth=3, learning_rate=0.1
The XGBR parameters: subsample=0.7, colsample_bytree=0.7, n_estimators=500, learning_rate=0.03, max_depth=5, min_child_weight=3

For example, inside the Kaggle, the XGBR received score 0.134 and the CatBoot only reached 0.197 (I tried both one_hot_max_size and cat_features). I'll be thankful if someone could point me what is wrong with the CatBoost model, may be some optimization is missing.


texts: What can i do after a PCA with the results?

texts:
After performing a PCA and studying the proceeding i ask myself what the result is good for in the next step. From the PCA i learned how to visualize the dataset by lowering the dimension, i got handy new vectors to describe the members of the population in a more efficient way and i learned which original predictors correlate and contribute more than others.

I ask myself questions like, is there more to learn from PCA, is it a good idea to use the PCA model for a learning algorithms to perform better and is it possible to respect nomial or ordinal predictors in the PCA somehow?


texts: Does the phenomenon of over-fitting of data varies with training algorithms?

texts: Suppose I have a dataset which I want to train using Neural network and SVM. Is it possible that with my dataset after training, the Neural network is overfit while the SVM is not? Like can a dataset be overfit for one training algorithms and not be overfit for another training algorithm? Is it even possible? Or if it is not overfit for one training algorithm, can be assume that it will not be overfit for other training algorithms too?


texts:
In my application, I want to have clusters whose diameters are bounded by some fixed number. Also, the number of clusters in the data is unknown and, therefore, the clusters must be discovered without a 'k' parameter.
What method should I use?


texts: Does the choice of error function impact the model parametrs?

texts:
Suppose I have trained a multi variate linear regression model on a particular training set, and the model parameters $\theta=[\theta_1,\theta_2,\ldots, \theta_n]$ were determined by minimizing a cost function, say MSE (Mean Squared Error).

Now, if for some reason I decide to change my choice of error function from MSE to MAE (Mean Absolute Error), will this change my model parameter $\theta$ or will it remain the same?


texts: How is Kernel Matrix on a distribution defined?

texts:
Consider the following words taken from the lecture notes:

The Hilbert-Schmidt Independence Criterion (HSIC) measures the dependence of the two random variables $X$ and $Y$. An empirical estimate of the HSIC is proportional to the $trace(KHLH)$, where

$K$ is a kernel matrix on $X$,
$L$ is a kernel on $Y$,
$H$ is a centering matrix with $H_{ij} = δ(i, j) − \frac{1}{n}$.

$HSIC(X, Y ) = 0$ if and only if $X$ and $Y$ are independent. The larger $HSIC(X, Y)$, the larger the dependence between $X$ and $Y$.

For me the only unclear part of this definition is the "[K] kernel matrix on a distribution [X]" sentence. Can someone explain what it means to apply the kernel on a distribution?


texts:
We have ~100 answers to a questionnaire with five questions (Q5). Independently from that, we have about 50, somewhat overlapping, features describing the people who answers the questions (F50). After having thrown an impressive amount of 'black box' regression models at trying to predict any of the 5 answers from the 50 features, we are approaching the conclusion that the features are just completely orthogonal to the topic of the questionnaire.

This is interesting, and a little surprising, and it could be fun to try to 'prove'. Does anyone know of a measure or method for which we could argue that if

'X does not produce any predictive value in Q5 when applied to F50'

then

'the causal relationship between F50 and Q5 is weaker than C'

Could some flavour of multivariate mutual information be a way forward?

I hope the question makes sense. It seems like it would be generally interesting.


texts:
I am using chunks of 100000 rows at a time from the CSV file to train the a simple LASSO model.

  How do i combine all of these models trained from these different chunks? I would like to use all these trained models for prediction

  I am familiar with DASK and other alternatives but i would like to use Pandas.

pipelines = {
    'lasso' : make_pipeline(StandardScaler(), Lasso(random_state=123))
}

for key, value in pipelines.items():
    print( key, type(value) )

# Lasso hyperparameters
lasso_hyperparameters = {
    'lasso__alpha' : [0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10]
}

hyperparameters = {
'lasso' : lasso_hyperparameters
}

# Create empty dictionary called fitted_models
fitted_models = {}

# Create cross-validation object from pipeline and hyperparameters
model = GridSearchCV(pipeline, hyperparameters[name], cv=10, n_jobs=-1)

def train(X_train, y_train):
    # Fit model on X_train, y_train
    model.fit(X_train, y_train)

    # Store model in fitted_models[name]
    fitted_models[name] = model

    # Print '{name} has been fitted'
    print(name, 'has been fitted.')
    print ("__________________________________")
    print (model.cv_results_)

for df in pd.read_csv('train_V2.csv', chunksize=100000):
    df = df.dropna()
    df = pd.get_dummies(df, columns=['matchType'])
    df_train = df.drop(['Id', 'groupId', 'matchId'], axis = 1)
    y = df_train.winPlacePerc
    X = df_train.drop('winPlacePerc', axis=1)
    X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                    test_size=0.2,
                                                    random_state=1234)
    X_train = np.asarray(X_train)
    X_test = np.asarray(X_test)
    y_train = np.asarray(y_train)
    y_test = np.asarray(y_test)

    train(X_train, y_train)


texts: How to input a 3d model into ML algorithm?

texts:
I have a machine learning model that uses csv with measured data about buildings: width, length, height etc. I use it to predict some features and it works properly.

I would like to drop csv with length, height and width, and I would like to use some kind of algorithm to parse 3d model into the ML algorithm.

The second reason is to try this approach with nonrectangular buildings, which are hard to describe in simple csv generated by humans.

I do not necessarily need the algorithm to read the width, length etc. but to be able to predict some values based on training set of 3d models acompanied by csv with results.

What should I look into? Where can I find information about this approach?


texts: Use of Random Forest algorithm in PySpark for imputation

texts: I am wondering how to use Random Forest algorithm for imputing missing values in a dataset. It is supposed to work well with missing values but I am not sure how those missing values are dealt with and how RF imputation works in PySpark.


texts: Can machine learning algorithms predict sports scores or plays?

texts:
I have a variety of NFL datasets that I think might make a good side-project, but I haven't done anything with them just yet.

Coming to this site made me think of machine learning algorithms and I wondering how good they might be at either predicting the outcome of football games or even the next play.

It seems to me that there would be some trends that could be identified - on 3rd down and 1, a team with a strong running back theoretically should have a tendency to run the ball in that situation.

Scoring might be more difficult to predict, but the winning team might be.

My question is whether these are good questions to throw at a machine learning algorithm.  It could be that a thousand people have tried it before, but the nature of sports makes it an unreliable topic.


texts: How to create a timeline graph ( Date / time series) in MATLAB?

texts:
The data file is attached https://ufile.io/ttwi8.How may I create a timeline graph where my y-axis is the number of cases(Demand and prediction as sjown in attached graph) and my x axis is increasing time of the day with the year?

  clc;
clear;

data3=readtable ('PRICE_AND_DEMAND_Jan.csv')
    Date_Time = data3(:,2);
    datecell=table2cell(Date_Time);
    load = data3(:,3);
    A = table2array(load)
    p=datenum(datecell);
    plot(datenum(datecell), A)
    datetick('x', 'dd-mmm-yyyy HH:MM:SS')
    % datetick('x', 'dd-mmm-yyyy HH:MM:SS')


texts: Method to filter out objective statements (or say facts)

texts:
I have millions of lines of statements containing both subjective (like I prefer the red skirt) and objective (Washington was born on February 22, 1732) statements or opinions. How can I separate them? Not manually.
By "objective", I mean if the object or predicative of the sentence changed, it would be in conflict with the truth. For illustration, in "She prefer red skirt.", if we change "red" or "skirt", the (new) statement would remain right, which can not apply to statements like "Washington was born on February 22, 2016".
The statements which should be pushed out:

Washington was born on February 22, 1732
Washington was born on February 21, 1731
Obama was born on February 22, 1732
Nobody A was born on February 22, 2016
Red is blue
Color is a day
I'm a robot
You are a person

Statements which should be kept:

I like coffee
Red is my favorite color
I hate him
You are awesome
You are clever
I was born years ago
You are very old
Tomorrow will be better


texts:
I performed a regression analysis with two datasets, each of which has size 50. One dataset is called Spatial % and the other Min values, and I wanted to check whether the two are correlated. I did the analysis in SPSS and the resulting scatterplot is as follows:

I am not that much experienced but it seems to me that a line is not the perfect fit for this scatterplot. Would a power line fit better? Or what else do you suggest?


texts: In SIFT feature extraction how the key points will be generated and how the features will be stored in the database. In image will the bag of visual words be images or text words?


texts: Model to detect specific semantic content without labeled data

texts:
I want to build a model that can detect sentences that discuss requests for communication - like 'email me', 'phone us', 'contact us', etc. However, I do not have any labeled data which I can use to simply train a neural network.
How can I go about solving this problem? On way I was thinking was to create an initial labeled dataset by detecting above mentioned phrases. But I am not sure how to generalize and detect sentences expressing the similar sentiment using some other phrase not in my list.
I am open to any other ideas as well. Thank you.


texts: Faster R-CNN wrapper for the number of RPNs in the layer dimensions?

texts:
When I intialize Faster R-CNN in the deployment phase, the number of samples per image (parameter from config file: TEST.RPN_POST_NMS_TOP_N) is set to 300, that's the number of predicted bounding boxes to keep after non-max suppression. However, the network is initialized with the number set to 1:

('rpn/output', (1, 512, 14, 14))
('rpn/output_rpn_relu/3x3_0_split_0', (1, 512, 14, 14))
('rpn/output_rpn_relu/3x3_0_split_1', (1, 512, 14, 14))
('rpn_cls_score', (1, 18, 14, 14))
('rpn_bbox_pred', (1, 36, 14, 14))
('rpn_cls_score_reshape', (1, 2, 126, 14))
('rpn_cls_prob', (1, 2, 126, 14))
('rpn_cls_prob_reshape', (1, 18, 14, 14))
('rois', (1, 5))
('pool5', (1, 512, 7, 7))
('fc6', (1, 4096))
('fc7', (1, 4096))
('fc7_relu7_0_split_0', (1, 4096))
('fc7_relu7_0_split_1', (1, 4096))
('cls_score', (1, 21))
('bbox_pred', (1, 84))
('cls_prob', (1, 21))

The ones I'm particularly interested are fc6/7, bbox_pred and cls_prob. After net.forward(**kwargs) is run, the first dimension of these layers is changed to 300: (300,4096), (300, 84), (300,21) to match the number of RoIs. The rois output is reshaped in the TargetLayer class, but rest are a bit of a problem:

Caffe doesn't implement this out of the box, there should be some wrapper for this, but I can't find it. Any suggestions on where to look? I want to implement something similar for my algorithm.

It is a bit confusing because all these four layer (fc6/7, bbox_pred, cls_prob) are just fully connected layers defined in the config, nothing fancy.

PS Also I don't think (100,4096) means 100 times more weights, it would cetrainly be undoable for such layer size, so the weights are shared. But how?


texts: Does save_best_only in Keras prevents overfitting?

texts:
I'm training a CNN and using:

model_checkpoint = ModelCheckpoint(os.path.join(output_artifacts,'weights.h5'), monitor='val_acc', save_best_only=True)

I trained the network for 70 epochs, but the validation accuracy flattened on good value (90%) after 20 epochs remaining pretty constant.

I would say that after epoch 20 (more or less 20:00 in the graph) the network is overfitting. My question is, does save_best_only takes this into account or just save the weights at epochs 1:00 am which is the overall best (but overfitted)?
Should I use earlystopping to stop the training or is not necessary? I don't want to stop the training too early..


texts:
I am a beginner with machine learning, and I'm trying to build a model to classify products by category according to the words present in the product name.
My goal is to predict the category of some new product, just by observing the categories for existing products.
For example, having the following products:

PRODUCT                                     CATEGORY
soap bar johnsons green leaves              bath
cookie bauducco lemon 120gr                 cookie
nesfit cookie choc and st                   cookie
strawberry soap soft                        bath
spoon hercules medium                       kitchen
soap dish plastic medium                    bath
[...]

My first thought is to group the words (tokens) present in each product, indicating the designated category and the occurrences count (to be used as a weight). So, for this sample, I have:

WORD           CATEGORY         COUNT
soap           bath             3
cookie         cookie           2
medium         bath             1
medium         kitchen          1
bar            bath             1
johnsons       bath             1

Having this, I could be able to train a model, and use it to classify a new product.

For example, having a new product hands liquid soap 120oz, it could be classified as bath, because it contains the word soap, which have a strong weight for the bath category.

In other case, the new product medium hammer could be classified as bath or kitchen , according the occurrence of the word medium in the training set.

So, my doubts are:

Am I going to the correct approach?
What is the best algorithm to be used in this case?
How can I apply this using Weka?


texts: It may be well-known that when we take statistics, we essentially need a large number of samples. Because I am taught this fact before studying the math, I have been here without exploring the reason. What is the intuition behind larger number of samples are better for statistics?


texts: Resampling for imbalaced datasets: should testing set also be resampled?

texts:
Apologies for what is probably a basic question but I have not been able to find a definitive answer either in the literature or in the Internet.

When dealing with an imbalanced dataset one possible strategy is to resample either the minority or the majority class to artificially generate a balanced training set that can be used to train a machine learning model.

My doubt stems from the fact that a testing set is supposed to be a representation of what real-world data is going to look like. Under this assumption, my understanding is that the testing set is not to be resampled, unlike the training set, since the imbalance we are trying to deal with is present in the live data in the first place.

Could someone please clarify if this intuition is right?


texts: Models that converged before aren't converging anymore in Keras

texts:
I have two models with saved data that worked well previously but won't anymore.

First, it happened with one of my Jupyter notebooks. I can even load the saved model and weights that work. When I train more with the exact same model, the performance actually drops!
For example, I get a dice coefficient of -.39 with my previous training when it worked. Now if I load the same model, weights, and data, it drops to -0.04. (Loss of -1 is perfect).

So I load one of my older notebooks with a different model and saved data that worked well. It doesn't converge to nearly as high of a performance as it did previously either.

However, I tried setting up a simple MNIST CNN classifier and it worked fine.

Is there any way for there to be persistent changes to occur so that the exact same code/data that performed well before no longer does?


texts:
Convolutional Neural Networks (CNNs) use almost always the rectified linear activation function (ReLU):

$$f(x) = max(0, x)$$

However, the derivative of this function is

$$f'(x) = \begin{cases} 0 &\text{if } x \leq 0\\ 1&\text{otherwise}\end{cases}$$

(ignoring that is not differentiable at $0$, as I think it is done in practice). For inputs > 0 this is fine, but why doesn't it matter that the gradient is 0 at every point < 0? Or does it matter? (Are there publications about this problem?)

If a neuron outputs 0 for every sample of the training data, it is basically lost, correct? Its weights will never be adjusted again?


texts: Tensorflow simple neural network has very bad performance in regression (curve fitting) problem

texts:
I'm trying to implement a very simple one layered MLP for a toy regression problem with one variable (dimension = 1) and one target (dimension = 1). It's a simple curve fitting problem with zero noise.

Matlab - Deep Learning Toolbox

Using levenberg-marquardt backpropagation on a MLP with a single hidden layer with 100 neurons and hyperbolic tangent activation I got pretty decent performance with almost zero effort:

MSE = 7.18e-08

Here's a plot of the fitting:

Edit:
This is the working matlab code. Please note that the "feedforwardnet(100)" function only produces a network object with one hidden layer with 100 neurons and tanh activation and output layer with linear activation:

net = feedforwardnet(100);
net.trainParam.min_grad = 1e-25;
net.trainParam.max_fail = 50;
net.trainParam.epochs = 500;
%net1.trainParam.showWindow = false;
net.inputs{1,1}.processFcns = {};
net.outputs{1,2}.processFcns = {};
net = train(net,Train_Vars,Train_Target);
Test_Predictions = net(Test_Vars);
Accuracy = msemetric({Test_Predictions},{Test_Target});

Python - TensorFlow - Keras

With the same network settings I used in matlab there's almost no training. No matter how hard I try to tune the training parameters or switch the optimizer.

MSE = 0.12900154

I can obtain something better using RELU activations for the hidden layer but we're still far:

MSE = 0.0582045

This is the code I used in Python:

#  IMPORT LIBRARIES
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow import keras

#  IMPORT DATASET FROM CSV FILE, SHUFFLE TRAINING SET
#  AND MAKE NUMPY ARRAY FOR TRAINING (DATA ARE ALREADY NORMALIZED)
dataset_path = "C:/Users/Rob/Desktop/Learning1.csv"
Learning_Dataset = pd.read_csv(dataset_path
                          , comment='\t',sep=","
                          ,skipinitialspace=False)
Learning_Dataset = Learning_Dataset.sample(frac = 1)  # SHUFFLING

test_dataset_path = "C:/Users/Rob/Desktop/Test1.csv"
Test_Dataset = pd.read_csv(test_dataset_path
                          , comment='\t',sep=","
                          ,skipinitialspace=False)

Learning_Target = Learning_Dataset.pop('Target')
Test_Target = Test_Dataset.pop('Target')

Learning_Dataset = np.array(Learning_Dataset,dtype = "float32")
Test_Dataset = np.array(Test_Dataset,dtype = "float32")
Learning_Target = np.array(Learning_Target,dtype = "float32")
Test_Target = np.array(Test_Target,dtype = "float32")

#  DEFINE SIMPLE MLP MODEL
inputs = tf.keras.layers.Input(shape=(1,))
x = tf.keras.layers.Dense(100, activation='relu')(inputs)
y = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs=inputs, outputs=y)

#  TRAIN MODEL
opt = tf.keras.optimizers.RMSprop(learning_rate = 0.001,
                                  rho = 0.9,
                                  momentum = 0.0,
                                  epsilon = 1e-07,
                                  centered = False)
early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=100)
model.compile(optimizer = opt,
              loss = 'mse',
              metrics = ['mse'])

model.fit(Learning_Dataset,
          Learning_Target,
          epochs=500,
          validation_split = 0.2,
          verbose=0,
          callbacks=[early_stop],
          shuffle = False,
          batch_size = 100)

#  INFERENCE AND CHECK ACCURACY
Predictions = model.predict(Test_Dataset)
Predictions = Predictions.reshape(10000)

print(np.square(np.subtract(Test_Target,Predictions)).mean()) #  MSE

plt.plot(Test_Dataset,Test_Target,'o',Test_Dataset,Predictions,'o')
plt.legend(('Target','Model Prediction'))
plt.show()

What am i doing wrong?

Thanks


texts:
I do not know if this is the right place to ask such a generic question, if not please let me know and I will remove it asap.

Well, I am a Software Engineering student (BEng hons) and this September my 3rd year starts and therefore I have to start my final year project. I want to do something in Machine Learning and more specifically in Prediction Algorithms.
I would like to do predictions for stock prices etc.

I have been reading a book provided by my university online the last days, but it doesnt make that much sense as there are a lot of algorithms and different approaches.

What algorithms would work for the project I want to do????
Do you know any specific books or online courses that you would recommend?

Any guidance would be really much appreciated!
Thank you!


texts:
I am working with a time series predicting whether web traffic will increase or decrease each day compared to the previous day for a given user.

Initially I used binary classes: labeled 1 for next day traffic increases and 0 for traffic decreases (which distributed into 60/40 split). Next I tried something conditionally: if the user has increased traffic for 3 previous days in a row and they increase tomorrow, that is labeled 1, else 0. Otherwise, if the user has decreased traffic the previous 3 days and decreases traffic tomorrow, that is labeled 1, else 0. So 1 doesn't always/necessarily correspond to traffic increases, it depends on the condition which can easily be observed when using this algorithm for real life predictions (by simply looking at the data).

With this 'conditional' dependent label encoding I have gotten much better results. The new binary classes are split 55/45 and accuracy and f1 have greatly improved for testing and training sets.

Is this kind of class labeling acceptable and/or good practice? I think it is positive as I am introducing more data without increasing dimensions but I am worried about mixing up the classes with this approach.

Thank you for your help!


texts:
While using Neural Networks (TensorFlow: Deep Neural Regressor), when increasing your training data size from a sample to the whole data (say a 10x larger dataset), what changes should you make to the model architecture (deeper/wider), learning rate and hyper parameters in general?

How much of trial and error how much of heuristic logic is involved in making these changes?


texts:
I'm working on preparing/analysing this dataset of Russian propaganda ads, in which there are columns which contain lists.

'interests_categories', for example, has a row which contains ['Unknown', 'Geographic', 'LGBTQ'].

I aim to compare classifiers and feature selection methods, create pipelines, and experiment with clustering and manifold learning. However, my understanding is that these lists will be tricky to deal with.

The dataset already contains some pre-made regex columns (including the above example), which concatenates all the values in the list with underscores. I have also found the possibility of converting these lists to tuples instead.

Is one of these approaches recommended above the other?


texts: How to determine if dataset is a suitable representation of the context?

texts: How can I determine if the data I have collected is a good enough representation of the context? For example, I am working on an object detection system and have been building an image dataset. How can I know if my dataset represents the task? For example, I need to account for instances where the object is close up and far away. But what about contexts/situations I have missed, or not even considered? Is there an art to building datasets?


texts: How to train a keras model on both original and augmented data from ImageDataGenerator?

texts:
I have a dataset that contains about 87000 images in a directory, with each class in a separate subfolder. I've tried the class ImageDataGenerator() and the function flow_from_directory() for generating the images, it worked completely fine but I have a question..
Does flow_from_directory() only yield the augmented images? and if this is the case, how can I train my model "which has overfit the training set" on both original and augmented data?
Thanks


texts: Learn the evaluation matrix for hire a job candidate

texts:
Let us assume that we have a certain number of features that are weighted with some parameters . The features could be the different skills that belongs to a job candidate  applying for a job vacancies .  The decision to give a job to the single candidate is based on a weighted sum of a different features .So we have a neural network model on which we have to learn the matrix that associate the  variable x = features  that a candidate has to y = ( hire/not hire)
I was wondering what is the name in literature of this problem and where i can find suggestion to solve it.
Many thanks


texts: How does Naive Bayes classifier work for continuous variables?

texts:
I know that for categorical features we just calculate the prior and likelihood probability assuming conditional independence between the features.

How does it work for continuous variables? How can we calculate likelihood probability for continuous variable?