Back to code

Neural Network

Summary

Neural Network (or Net for short) is a C++ class providing a generic interface for training and executing simple backpropagation neural networks, for performing a variety of tasks, based loosely upon code by Karsten Kutza from here.

Download

Source is available here (see below for packages):

Net.h (view html)
Net.cpp (view html)
NetLayer.cpp (view html)
backprop.cpp (view html)
Makefile (view html)

or here as a package:

How to Use

The neural network class itself is the class NeuralNetwork::Net. It depends on the class NeuralNetwork::NetLayer, both of which are defined in Net.h. The file backprop.cpp is a sample of using Net to train a system to predict sunspot patterns, and Makefile is used to build the sample application on platforms with make.

I will assume you are already familiar with neural networks in the following description of how to typically use the classes:

1. Seed your random number generator (if you wish to randomize weights).
2. Create an instance of NeuralNetwork::Net, passing in the number of layers, number of perceptrons in each layer, momentum factor, learning rate, and gain for the sigmoid function.
3. Randomize or clear weights, whichever is desirable, with Net::randomizeWeights() or Net::clearWeights().
4. Create a class (or two, or more) inheriting from NeuralNetwork::ExampleFactory and implement getExample, which returns via its parameters a new input/output pair each time it's called, and numExamples, which is an estimate of the total number of unique examples.
5. Create an instance of one of these classes for your training set (the set the network attempts to minimize error on) and an instance of one of these classes for your test set (the set used to identify overtraining). These will be passed to autotrain.
6. Use Net::autotrain() to train the network to its optimum error level on the test set. It will perform a specified number of epochs, then test if error on the test set has exceeded minimum error times the specified cut off error. If so, it will consider training done, and return with the network configured to the weights that gave best results on the test set. Otherwise, it will repeat.
7. If you will not train the network again (without saving and reloading it), you can call Net::doneTraining() to dispose of temporary storage used during training.
8. Run the neural network on new cases using Net::run(), which simply takes an input and produces an output.
9. If you wish to save your network for future sessions, use Net::save() with a binary-mode std::ostream. Load it again using the Net constructor taking a binary-mode istream.

Since you might wish to observe the progress of your network's training, you may #define the symbol NEURAL_NET_DEBUG to view the total error each time it is calculated for the test set. When this starts to go up, training is nearing completion.

To-Do

• Add support for a callback to be called each time the test error is calculated, rather than just a debug trace, so end-users can view progress.
• Add support for delta functions other than the sigmoid function.
• Add support for more direct training, when more control is needed.
• There's always more room for optimization.
• Your suggestions

Back to code

Back to main page

All text and images, but not necessarily linked material, on this page ©1998-2006 Derrick Coetzee and Moonflare and may not be reproduced or used for any purpose without prior written permission except where otherwise indicated.