Long short-term memory (LSTM) Network

Class Reference

class pykitml.LSTM(layer_sizes, output_activ_func='softmax', cost_func='cross_entropy')

This class implements LSTM Network.

__init__(layer_sizes, output_activ_func='softmax', cost_func='cross_entropy')
Parameters:

  • layer_sizes (list) – A list of integers describing the number of layers and the number of neurons in each layer. For e.g. [784, 100, 100, 10] describes a network with one input layer having 784 neurons, two hidden LSTM layers having 100 neurons each and a dense output layer with 10 neurons.

  • output_activ_function (str) – Activation function to use for dense output layer. List of available activation functions: leakyrelu, relu, softmax, tanh, sigmoid, identity.

  • cost_function (str) – List of available cost functions: mse (Mean Squared Error), cross_entropy (Cross Entropy), huber (Huber loss).

feed(input_data)

Accepts input array and feeds it to the model.

Parameters:

input_data (numpy.array) – The input to feed the model.

Raises:

ValueError – If the input data has invalid dimensions/shape.

Note

This function only feeds the input data, to get the output after calling this function use get_output() or get_output_onehot()

get_output()

Returns the output activations of the model.

Returns:

The output activations.

Return type:

numpy.array

get_output_onehot()

Returns the output layer activations of the model as a one-hot array. A one-hot array is an array of bits in which only one of the bits is high/true. In this case, the corresponding bit to the neuron/node having the highest activation will be high/true.

Returns:

The one-hot output activations array.

Return type:

numpy.array

train(training_data, targets, batch_size, epochs, optimizer, testing_data=None, testing_targets=None, testing_freq=1, decay_freq=1)

Trains the model on the training data, after training is complete, you can call plot_performance() to plot performance graphs.

Parameters:

  • training_data (numpy.array) – numpy array containing training data.

  • targets (numpy.array) – numpy array containing training targets, corresponding to the training data.

  • batch_size (int) – Number of training examples to use in one epoch, or number of training examples to use to estimate the gradient.

  • epochs (int) – Number of epochs the model should be trained for.

  • optimizer (any Optimizer object) – See Optimizers

  • testing_data (numpy.array) – numpy array containing testing data.

  • testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.

  • testing_freq (int) – How frequently the model should be tested, i.e the model will be tested after every testing_freq epochs. You may want to increase this to reduce training time.

  • decay_freq (int) – How frequently the model should decay the learning rate. The learning rate will decay after every decay_freq epochs.

Raises:

ValueError – If training_data, targets, testing_data or testing_targets has invalid dimensions/shape.

reset()

Resets the hidden state.

plot_performance()

Plots logged performance data after training. Should be called after train().

Raises:

  • AttributeError – If the model has not been trained, i.e train() has not been called before.

  • IndexError – If train() failed.

cost(testing_data, testing_targets)

Tests the average cost of the model on the testing data passed to the function.

Parameters:

  • testing_data (numpy.array) – numpy array containing testing data.

  • testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.

Returns:

cost – The average cost of the model over the testing data.

Return type:

float

Raises:

ValueError – If testing_data or testing_targets has invalid dimensions/shape.

accuracy(testing_data, testing_targets)

Tests the accuracy of the model on the testing data passed to the function. This function should be only used for classification.

Parameters:

  • testing_data (numpy.array) – numpy array containing testing data.

  • testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.

Returns:

accuracy – The accuracy of the model over the testing data i.e how many testing examples did the model predict correctly.

Return type:

float

r2score(testing_data, testing_targets)

Return R-squared or coefficient of determination value.

Parameters:

  • testing_data (numpy.array) – numpy array containing testing data.

  • testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.

Returns:

r2score – The average cost of the model over the testing data.

Return type:

float

Raises:

ValueError – If testing_data or testing_targets has invalid dimensions/shape.

confusion_matrix(test_data, test_targets, gnames=[], plot=True)

Returns and plots confusion matrix on the given test data.

Parameters:

  • test_data (numpy.array) – Numpy array containing test data

  • test_targets (numpy.array) – Numpy array containing the targets corresponding to the test data.

  • plot (bool) – If set to false, will not plot the matrix. Default is true.

  • gnames (list) – List of string names for each class/group.

Returns:

confusion_matrix – The confusion matrix.

Return type:

numpy.array

nlayers

The number of layers in the network.