Download Pattern Recognition II - Artificial Intelligence - Lecture Notes | CECS 451 and more Study notes Computer Science in PDF only on Docsity!
Lecture 10: Pattern Recognition II
Recall from Lecture 9 some of the goals of the science of pattern recognition:
- classify objects and data into different categories;
- find patterns in data for the purpose of knowledge discovery and prediction;
- find patterns in sets of experience (feedback) data for the purpose of learning and evolving;
- Associative memory: the ability to recall past information/experience based on exposure to patterns similar to those which partially comprise the stored informa- tion/experience.
In the previous lecture we examined more closely the problem of data classification, and found that neural networks represent a general-purpose tool for solving classification problems. We now examine two other goals of pattern recognition.
Associative Memory
Hopfield neural networks: unlike feedforward neural networks, Hopfield networks do not have restrictions imposed in terms of which neurons a neuron may or may not connect with. Moreover, each neuron plays the role of a flip-flop digital gate, in that it persists in either an “on” or “off” state, and changes from “off” (respectively “on”) to “on” (respectively “off”) whenever its input strength exceeds (respectively falls below) its threshold value. More properties include
- the connection weights are symmetric; i.e., if wij is the weight (strength) between neuron i and neuron j then it equals wji.
- the connection weights take on real values that may either be positive or negative.
- the changing of state for each neuron may happen synchronously or asynchronously.
- the engery function of the network is defined as
E = −
∑
i<j
wij sisj +
∑
i
Θisi,
where si and Θi respectively represent the state and threshold of the i th neuron.
Hopfield network Example 1: for the following synchronous Hopfield network, show how the network evolves to a final state. There are four neurons 1, 2 , 3 , 4, and the set of connections is {(1, 2), (2, 3), (3, 4)}.
The weights are w 12 =. 75 , w 23 =. 75 , w 34 = −.875. Assume that the initial state is given by
Hopfield network Example 2: design a Hopfield network that recalls either 11111 or 00000, depending on which bit is in the majority with respect to the input pattern. For example, an input pattern of 10111 should induce a memory recall (network evolution) of
Hopfield network Example 3: design a Hopfield network that recalls either X or O, where either character is written on a 4 × 4 grid of pixels.
The Partial Derivative of a Set of Feature Vectors
Given two feature vectors ~x and ~y of same dimension, the similarity between ~x and ~y, denoted s(~x, ~y), is a vector whose i th component equals ∗ if ~xi 6 = ~yi and is equal to ~xi otherwise.
Let S be a set of feature vectors and ~x an element of S. Then the partial derivative of S with respect to ~x, denoted by ∂S∂~x is defined as the set of vectors
∂S ∂~x
= {s(~x, ~y)|~y ∈ S}.
Partial derivative Example: given the set
S = {(trim), (tree), (told), (f our), (f ive), (nine)}
of feature vectors (each of the four features is a letter of the alphabet), compute ∂S∂~x , where ~x = tree.
Theorem: let S be a set of feature vectors and ~p be an ω-frequent pattern of S. Then there exists ~x ∈ S such that p~ is an ω-frequent pattern of ∂S∂~x. Conversely, for any vector ~x ∈ S, the set of ω-frequent patterns of ∂S∂~x are ω-frequent patterns of S.
Proof:
Theorem: equality of mixed partials: given two feature vectors ~x and ~y in S,
∂^2 S ∂~x∂s(~x, ~y)
∂^2 S
∂~y∂s(~y, ~s)
Moreover, the frequent patterns of this second-order partial derivative are those frequent patterns contained in both ~x and ~y.
Proof:
Frequent pattern Example: use the PD-algorithm for finding a set of .5-frequent patterns for the following set of binary feature vectors:
S = {(1011), (0111), (1001), (1111), (1011), (0011), (1001), (1000), (1011), (0110).}
