Long short-term memory (LSTM) Network

Class Reference

class pykitml.LSTM(layer_sizes, output_activ_func='softmax', cost_func='cross_entropy')

This class implements LSTM Network.

__init__(layer_sizes, output_activ_func='softmax', cost_func='cross_entropy')
Parameters:
  • layer_sizes (list) – A list of integers describing the number of layers and the number of neurons in each layer. For e.g. [784, 100, 100, 10] describes a network with one input layer having 784 neurons, two hidden LSTM layers having 100 neurons each and a dense output layer with 10 neurons.
  • output_activ_function (str) – Activation function to use for dense output layer. List of available activation functions: leakyrelu, relu, softmax, tanh, sigmoid, identity.
  • cost_function (str) – List of available cost functions: mse (Mean Squared Error), cross_entropy (Cross Entropy), huber (Huber loss).
feed(input_data)

Accepts input array and feeds it to the model.

Parameters:input_data (numpy.array) – The input to feed the model.
Raises:ValueError – If the input data has invalid dimensions/shape.

Note

This function only feeds the input data, to get the output after calling this function use get_output() or get_output_onehot()

get_output()

Returns the output activations of the model.

Returns:The output activations.
Return type:numpy.array
get_output_onehot()

Returns the output layer activations of the model as a one-hot array. A one-hot array is an array of bits in which only one of the bits is high/true. In this case, the corresponding bit to the neuron/node having the highest activation will be high/true.

Returns:The one-hot output activations array.
Return type:numpy.array
train(training_data, targets, batch_size, epochs, optimizer, testing_data=None, testing_targets=None, testing_freq=1, decay_freq=1)

Trains the model on the training data, after training is complete, you can call plot_performance() to plot performance graphs.

Parameters:
  • training_data (numpy.array) – numpy array containing training data.
  • targets (numpy.array) – numpy array containing training targets, corresponding to the training data.
  • batch_size (int) – Number of training examples to use in one epoch, or number of training examples to use to estimate the gradient.
  • epochs (int) – Number of epochs the model should be trained for.
  • optimizer (any Optimizer object) – See Optimizers
  • testing_data (numpy.array) – numpy array containing testing data.
  • testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.
  • testing_freq (int) – How frequently the model should be tested, i.e the model will be tested after every testing_freq epochs. You may want to increase this to reduce training time.
  • decay_freq (int) – How frequently the model should decay the learning rate. The learning rate will decay after every decay_freq epochs.
Raises:

ValueError – If training_data, targets, testing_data or testing_targets has invalid dimensions/shape.
reset()

Resets the hidden state.

plot_performance()

Plots logged performance data after training. Should be called after train().

Raises:
  • AttributeError – If the model has not been trained, i.e train() has not been called before.
  • IndexError – If train() failed.
cost(testing_data, testing_targets)

Tests the average cost of the model on the testing data passed to the function.

Parameters:
  • testing_data (numpy.array) – numpy array containing testing data.
  • testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.
Returns:

cost – The average cost of the model over the testing data.
Return type:

float
Raises:

ValueError – If testing_data or testing_targets has invalid dimensions/shape.
accuracy(testing_data, testing_targets)

Tests the accuracy of the model on the testing data passed to the function. This function should be only used for classification.

Parameters:
  • testing_data (numpy.array) – numpy array containing testing data.
  • testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.
Returns:

accuracy – The accuracy of the model over the testing data i.e how many testing examples did the model predict correctly.
Return type:

float
r2score(testing_data, testing_targets)

Return R-squared or coefficient of determination value.

Parameters:
  • testing_data (numpy.array) – numpy array containing testing data.
  • testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.
Returns:

r2score – The average cost of the model over the testing data.
Return type:

float
Raises:

ValueError – If testing_data or testing_targets has invalid dimensions/shape.
confusion_matrix(test_data, test_targets, gnames=[], plot=True)

Returns and plots confusion matrix on the given test data.

Parameters:
  • test_data (numpy.array) – Numpy array containing test data
  • test_targets (numpy.array) – Numpy array containing the targets corresponding to the test data.
  • plot (bool) – If set to false, will not plot the matrix. Default is true.
  • gnames (list) – List of string names for each class/group.
Returns:

confusion_matrix – The confusion matrix.
Return type:

numpy.array
nlayers

The number of layers in the network.