Download Pattern Recognition - Artificial Intelligence - Lecture Notes | CECS 451 and more Study notes Computer Science in PDF only on Docsity!
Lecture 9: Pattern Recognition
One area of research that is closely related to (and often overlaps with) machine learning is that of pattern recognition. The importance of pattern recognition to machine learning and intelligence in general cannot be overstated.
Pattern recognition: the science whose goal involves the theoretical and empirical un- derstanding of how to create systems and algorithms that are able to do, amoung other things,
- classify objects and data into different categories;
- find patterns in data for the purpose of knowledge discovery and prediction;
- find patterns in sets of experience (feedback) data for the purpose of learning and evolving;
- create memories of patterns which can be retrieved upon exposure to similar patterns
Applications of Pattern Recognition:
- Machine vision: finding defects in VLSI chips, blood testing, character recognition, robotics
- Voice recognition
- Data mining: learning the preferences of a shopper, gene analysis, discovery of trends, predicting weather and markets
- security: profiling, fingerprint analysis
Pattern Classification
As we have already seen, pattern classification represents one of the main problems in both machine learning and pattern recognition. The following represents the individual tasks involved with designing a pattern classifier.
Pattern-classification design example: suppose the problem involves classifying letters written on an electronic notepad.
- sensors: a 21 × 21 matrix of cells (pixels) which lie underneath the surface of the notepad. The voltage state of each cell changes (from off to on) when a metallic pen makes contact directly above the cell.
- feature generation: the cell matrix causes the formation of a 21 × 21 binary matrix to be stored in memory. This pixel data represents the raw, generated feature.
- feature selection: for the purpose of classification, more useful features can be se- lected from the raw feature. For example, the raw feature may be divided up into 9 7 × 7 submatrices, and a feature may be the presence or absence of a subpattern (e.g. a horizontal segment of four ones).
- classifier design: once the features are selected, they form what is called a feature vector, which can then be input into a classifier, which will output a classification. Examples of classifiers include decision trees, Boolean functions, and neural networks.
- evaluation: testing must be performed to see if the percentage of correct classifications is sufficiently high. Testing will inevitably cause re-evaluation of all previous stages of development.
Artificial Neurons as Classifiers
Artificial Neurons are very similar to Boolean gates (and, or, and inverter gates), in that they have inputs and an output. The main difference though is that they can receive real-valued inputs, output real values, and have their functionality modified through a training/learning process.
Artificial Neuron: a mathematical construct (thought of as a linear threshold function) which was initially intended to model the behavior of a biological neuron. A biological neuron has the tendency to synapse upon receiving a sufficient number (i.e. a number which exceeds its threshold) of synaptic impulses from neighboring neurons. Properties of an aritficial neuron:
- associated with each neuron n is a weight vector w~. Moreover, the dimension k of w~ represents the number of neurons or environmental impulses that feed into n.
- each neuron n also has a threshold T. If the strength of the neuron synapses feeding into n exceeds T , then T itself will synapse (i.e. output a value).
- the strength of the neuron synapses feeding into n is given by the dot product w~ · ~x, where ~xi represents the i th input into n, which is also the output of the i th neuron feeding into n.
- a discrete neuron will output 1 when w~ · ~x > T , and 0 otherwise.
- a smoothed neuron will output f ( w~ · ~x − T ) where f is a smoothing function, such as
f (x) =
1 + e−x^
A discrete neuron with weight vector w~ and threshold T may be used as a type of linear classifier. For example, given a feature vector ~x, the vector may be classified into one of two categories depending on whether or not w~ · ~x > T. But how to choose w~ and T for a given set of training vectors?
Perceptron Learning Example: use the perceptron learning algorithm to find a neu- ron/line that linearly separates W 1 = {(− 1 , 1), (− 2 , 3), (1, 3)} from W 2 = {(3, −1), (4, 5)}.
Artificial Neural Networks as Classifiers
An artificial neural network is simply a collection of neurons for which the outputs of some neurons serve as the inputs to other neurons, to potentially form a complex dynamical system. They represent the most important class of nonlinear classifiers. Why neural networks?
- represent an attempt to model a brain’s structure and functionality in hopes that they will be enabled with the same classificational/pattern-recognition ability that most brains possess
- experiments have shown they classify well in cases where the feature vectors
- possess errors and noise
- represent spatial and/or audio data
- computing with neural networks is parallel and distributed, and thus is ideal for parallel processing
- learning algorithms exist for neural networks (e.g. backpropagation)
Feedforward Example 1: the “EXOR” problem. Provide a two-layer neural network which correctly classifies the vectors in W 1 = {(0, 1), (1, 0)} from W 2 = {(0, 0), (1, 1)}.
Theorem: any two classes of vectors W 1 and W 2 can be correctly classified by a three-layer neural network.
Proof:
- The first layer maps real vectors to vertices of a hypercube.
- The second and third layers provide a sum-of-minterms expansion for those vertices of the hypercube which correspond to W 1.
Feedforward Example 2: for the following sets W 1 and W 2 provide a three-layer network which correctly classifies the sets.
