Feed-Forward Neural Network¶
Class Reference¶
-
class
pykitml.
NeuralNetwork
(layer_sizes, reg_param=0, config='leakyrelu-softmax-cross_entropy')¶ This class implements Feed-forward Neural Network.
-
__init__
(layer_sizes, reg_param=0, config='leakyrelu-softmax-cross_entropy')¶ Parameters: - layer_sizes (list) – A list of integers describing the number of layers and the number of neurons in each
layer. For e.g.
[784, 100, 100, 10]
describes a neural network with one input layer having 784 neurons, two hidden layers having 100 neurons each and a output layer with 10 neurons. - reg_param (int) – Regularization parameter for the network, also known as ‘weight decay’.
- config (str) –
The config string describes what activation functions and cost function to use for the network. The string should contain three function names seperated with ‘-’ character and should follow the order:
'<hidden_layer_activation_func>-<output_layer_activation_func>-<cost_function>'
. For e.g.'relu-softmax-cross_entropy'
tells the class to use relu as the activation function for input and hidden layers, softmax for output layer and cross entropy for the cost function.List of available activation functions:
leakyrelu
,relu
,softmax
,tanh
,sigmoid
,identity
.List of available cost functions:
mse
(Mean Squared Error),cross_entropy
(Cross Entropy),huber
(Huber loss).
Raises: AttributeError
– If invalid config string.- layer_sizes (list) – A list of integers describing the number of layers and the number of neurons in each
layer. For e.g.
-
feed
(input_data)¶ Accepts input array and feeds it to the model.
Parameters: input_data (numpy.array) – The input to feed the model. Raises: ValueError
– If the input data has invalid dimensions/shape.Note
This function only feeds the input data, to get the output after calling this function use
get_output()
orget_output_onehot()
-
get_output
()¶ Returns the output activations of the model.
Returns: The output activations. Return type: numpy.array
-
get_output_onehot
()¶ Returns the output layer activations of the model as a one-hot array. A one-hot array is an array of bits in which only one of the bits is high/true. In this case, the corresponding bit to the neuron/node having the highest activation will be high/true.
Returns: The one-hot output activations array. Return type: numpy.array
-
train
(training_data, targets, batch_size, epochs, optimizer, testing_data=None, testing_targets=None, testing_freq=1, decay_freq=1)¶ Trains the model on the training data, after training is complete, you can call
plot_performance()
to plot performance graphs.Parameters: - training_data (numpy.array) – numpy array containing training data.
- targets (numpy.array) – numpy array containing training targets, corresponding to the training data.
- batch_size (int) – Number of training examples to use in one epoch, or number of training examples to use to estimate the gradient.
- epochs (int) – Number of epochs the model should be trained for.
- optimizer (any Optimizer object) – See Optimizers
- testing_data (numpy.array) – numpy array containing testing data.
- testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.
- testing_freq (int) – How frequently the model should be tested, i.e the model will be tested
after every
testing_freq
epochs. You may want to increase this to reduce training time. - decay_freq (int) – How frequently the model should decay the learning rate. The learning rate
will decay after every
decay_freq
epochs.
Raises: ValueError
– Iftraining_data
,targets
,testing_data
ortesting_targets
has invalid dimensions/shape.
-
plot_performance
()¶ Plots logged performance data after training. Should be called after
train()
.Raises:
-
cost
(testing_data, testing_targets)¶ Tests the average cost of the model on the testing data passed to the function.
Parameters: - testing_data (numpy.array) – numpy array containing testing data.
- testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.
Returns: cost – The average cost of the model over the testing data.
Return type: float
Raises: ValueError
– Iftesting_data
ortesting_targets
has invalid dimensions/shape.
-
accuracy
(testing_data, testing_targets)¶ Tests the accuracy of the model on the testing data passed to the function. This function should be only used for classification.
Parameters: - testing_data (numpy.array) – numpy array containing testing data.
- testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.
Returns: accuracy – The accuracy of the model over the testing data i.e how many testing examples did the model predict correctly.
Return type: float
-
r2score
(testing_data, testing_targets)¶ Return R-squared or coefficient of determination value.
Parameters: - testing_data (numpy.array) – numpy array containing testing data.
- testing_targets (numpy.array) – numpy array containing testing targets, corresponding to the testing data.
Returns: r2score – The average cost of the model over the testing data.
Return type: float
Raises: ValueError
– Iftesting_data
ortesting_targets
has invalid dimensions/shape.
-
confusion_matrix
(test_data, test_targets, gnames=[], plot=True)¶ Returns and plots confusion matrix on the given test data.
Parameters: - test_data (numpy.array) – Numpy array containing test data
- test_targets (numpy.array) – Numpy array containing the targets corresponding to the test data.
- plot (bool) – If set to false, will not plot the matrix. Default is true.
- gnames (list) – List of string names for each class/group.
Returns: confusion_matrix – The confusion matrix.
Return type: numpy.array
-
nlayers
¶ The number of layers in the network.
-
Example: Handwritten Digit Recognition (MNIST)¶
Dataset
MNIST - pykitml.datasets.mnist module
Training
import os.path
import pykitml as pk
from pykitml.datasets import mnist
# Download dataset
if not os.path.exists('mnist.pkl'):
mnist.get()
# Load dataset
training_data, training_targets, testing_data, testing_targets = mnist.load()
# Create a new neural network
digit_classifier = pk.NeuralNetwork([784, 100, 10])
# Train it
digit_classifier.train(
training_data=training_data,
targets=training_targets,
batch_size=50,
epochs=1200,
optimizer=pk.Adam(learning_rate=0.012, decay_rate=0.95),
testing_data=testing_data,
testing_targets=testing_targets,
testing_freq=30,
decay_freq=15
)
# Save it
pk.save(digit_classifier, 'digit_classifier_network.pkl')
# Show performance
accuracy = digit_classifier.accuracy(training_data, training_targets)
print('Train Accuracy:', accuracy)
accuracy = digit_classifier.accuracy(testing_data, testing_targets)
print('Test Accuracy:', accuracy)
# Plot performance graph
digit_classifier.plot_performance()
# Show confusion matrix
digit_classifier.confusion_matrix(training_data, training_targets)
Predicting
import random
import matplotlib.pyplot as plt
import pykitml as pk
from pykitml.datasets import mnist
# Load dataset
training_data, training_targets, _, _ = mnist.load()
# Load the trained network
digit_classifier = pk.load('digit_classifier_network.pkl')
# Pick a random example from testing data
index = random.randint(0, 9999)
# Show the test data and the label
plt.imshow(training_data[index].reshape(28, 28))
plt.show()
print('Label: ', training_targets[index])
# Show prediction
digit_classifier.feed(training_data[index])
model_output = digit_classifier.get_output_onehot()
print('Predicted: ', model_output)
Performance Graph
Confusion Matrix