CSCI 5622 Professor Grudic
Machine Learning Fall 2001
1
Assignment 4
Assigned: Thu Oct 12, 2001
Due: Thu Oct 25, 2001
In this assignment, you will write software to implement a neural network and train your neural network to
recognize hand-printed digits. Your neural net simulator must meet the following minimum requirements.
•It should implement a three layer (input, hidden, output) fully interconnected neural network. Each input
unit is connected to each hidden unit, and each hidden unit is connected to each output unit. In addition,
each unit should have a bias weight associated with it.
•The simulator can be run in either “training” or “testing” mode. In training mode, the weights are initial-
ized randomly (using one of the procedures described in class), the network is presented with training
data until the training error reaches some criterion, and then the neural network weights should be
stored in a file. In testing mode, the weights are read from a file, and the trained neural network is used
to classify data in a test set.
•The neural network should be trained using the back propagation learning algorithm. You may use
either batch or on-line training modes. On-line training—where the weights are adjusted following each
training example—will be easier to implement.
•Your simulator should allow the user to easily specify: the number of input, hidden, and output units; the
name of the training or test file, the learning rate (or a learning rate schedule), a stopping criterion
(when to stop training the network, either in epochs or in terms of an error threshold). It would be nice if
these parameters were read from a file or the command line, but you can #define them if you like.
•Your simulator should handle real-valued inputs and target outputs, and it should be able to rescale the
inputs as I’ll explain.
Data base
The National Institute of Standards and Technology, NIST, collected a large data base of hand printed dig-
its. Yann LeCun at AT&T has done some preprocessing on this data base to factor out some variabilityin
the position of the digits. See www.research.att.com/~yann/exdb/mnist for details on the original
data base and Yann’s transformations. I have further shrunk the data base—by compressing 28x28 images
to 14x14 and selecting 2500 of the 70,000 examples for training and another 2500 for testing. Each digit is
coded as a real-valued pattern. For example, here are instances of the digits 0 and 4:
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 4 7 1 . . . . . . . . . . 2 . . . 1 . . . .
. . . . . 3 7 8 3 . . . . . . . . . . 6 . . . 2 3 . . .
. . . . 1 8 8 8 8 6 . . . . . . . . 3 5 . . . 2 4 . . .
. . . . 6 8 7 2 2 8 4 . . . . . . 1 7 . . . . 5 4 . . .
. . . 1 8 5 . . . 5 8 1 . . . . . 3 5 . . . 1 7 1 . . .
. . . 1 8 . . . . 5 8 2 . . . . . 4 3 . . . 3 6 . . . .
. . . 3 8 . . . 4 8 6 . . . . . . 2 7 5 5 6 7 5 . . . .
. . . 3 8 1 4 5 8 8 1 . . . . . . . . 1 1 . 6 4 . . . .
. . . 1 8 8 8 8 7 3 . . . . . . . . . . . . 6 4 . . . .
. . . . 2 6 7 5 1 . . . . . . . . . . . . . 7 3 . . . .
. . . . . . . . . . . . . . . . . . . . . . 2 . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Each pixel has an intensity from 0 (white) to 9 (dark). To make the digits stand out in the above figures, I’ve
replaced the 0 intensities with periods.
