Download Neural Network Learning: Processes and Paradigms - Prof. Y. Choe and more Study notes Computer Science in PDF only on Docsity!
Slide
Haykin Chapter 2: Learning
Processes
CPSC 636-
Instructor: Yoonsuck Choe Spring 2008
1
Introduction
- Property of primary significance in nnet: learn from its environment, and improve its performance through learning.
- Iterative adjustment of synaptic weights.
- Learning: hard to define. - One definition by Mendel and McClaren: Learning is a process by which the free parameters of a neural network are adapted through a process of stimulation by the environment in which the network is embedded. The type of learning is determined by the manner in which the parameter changes take place.
2
Learning
Sequence of events in nnet learning:
- nnet is stimulated by the environment.
- nnet undergoes changes in its free parameters as a result of this stimulation.
- nnet responds in a new way to the environment because of the changes that have occurred in its internal structure.
A prescribed set of well-defined rules for the solution of the learning problem is called a learning algorithm.
The manner in which a nnet relates to the environment dictates the learning paradigm that refers to a model of environment operated on by the nnet.
Overview
Organization of this chapter:
- Five basic learning rules error correction, Hebbian, memory-based, copetetive, and Boltzmann
- Learning paradigms credit assignment problem, supervised learning, unsupervised learning
- Learning tasks, memory, and adaptation
- Probabilistic and statistical aspects of learning
Error-Correction Learning
- Input x(n), output yk (n), and desired response or target
output dk (n).
- Error signal ek (n) = dk (n) − yk (n)
- ek (n) actuates a control mechanism that gradually adjust the synaptic weights, to miminize the cost function (or index of performance) :
E(n) =
e^2 k (n)
- When synaptic weights reach a steady state, learning is stopped. 5
Error-Correction Learning: Delta Rule
- Widrow-Hoff rule, with learning rate η:
∆wk j(n) = ηek (n)xj (n)
- With that, we can update the weights:
wkj (n + 1) = wkj (n) + ∆wk j(n)
- There is a sound theoretical reason for doing this, which we will discuss later.
6
Memory-Based Learning
- All (or most) past experiences are explicitly stored, as input-target
pairs{xi, di)}Ni=1.
- Two classes C 1 , C 2.
- Given a new input xtest, determine class based on local
neighborhood of xtest.
- Criterion used for determining the neighborhood - Learning rule applied to the neighborhood of the input, within the set of training examples.
Memory-Based Learning: Nearest Neighbor
- A set of instances observed so far:
X = {x 1 , x 2 , ..., xN }
- Nearest neighbor x′ N ∈ X of xtest:
min
i
d(xi, xtest) = d(xi, xtest)
where d(·, ·) is the Euclidean distance.
- xtest is classified as the same class as x′ N.
- Cover and Hart (1967): The bound on error is at max twice that of the optimal (Bayes probability of error), given - The classified examples are independently and identically distributed. - The sample size N is infinitely large.
Mathematical Models of Synaptic Plasticity
- General form: ∆wkj (n) = F (yk (n), xj (n))
- Hebbian learning (with learning rate η): ∆wkj (n) = ηyk (n)xj (n)
- Covariance rule:∆wkj = η(xj − ¯x)(yk − y¯)
13
Covariance Rule (Sejnowski 1977)
∆wkj = η(xj − ¯x)(yk − y¯)
- Convergence to a nontrivial state
- Prediction of both potentiation and depression.
- Observations: - Weight enhanced when both pre- and post-synaptic activities are above average. - Weight depressed when
∗ Presynaptic activity more than average , and postsynaptic
activity less than average.
∗ Presynaptic activity less than average , and postsynaptic
activity more than average.
14
Competetive Learning
- Output neurons compete with each other for a chance to become active.
- Highly suited to discover statistically salient features (that may aid in classification).
- Three basic elements: - Same type of neurons with different weight sets, so that they respond differently to a given set of inputs. - A limit imposed on the strength of each neuron. - Competition mechanism, to choose one winner : winner-takes-all neuron.
Inputs and Weights Seen as Vectors in
High-dimensional Space
x 1 x 2 x 3
xn
yk
wk wk
wk
wkn
( x 1 x 2 x 3 ... xn)
( wk1 wk2 wk3...wkn)
x
coordinate system w
- Inputs and weights can be seen as vectors: x and wk. Note that
the weight vector belongs to a certain output neuron k, and thus
the index.
Competetive Learning: Example
- Single layer, feedforward excitatory, and lateral in- hibitory connections
- Winner selection
yk =
1 if vk > vj for all j, j 6 = k 0 otherwise
P
j wkj^ = 1^ for all^ k.
∆wkj =
η(xj − wk j) if k is the winner 0 otherwise
- The synaptic weight vector wk = (wk 1 , wk 2 , ..., wkn) is moved toward the input vector. 17
Competetive Learning
x
w(n)
x−w(n)
w(n+1) η^ (x−w(n))
∆wkj =
η(xj − wk j) if k is the winner
0 otherwise
Interpreting this as a vector, we get the above plot.
- Weight vectors converge toward local input clusters: clustering. 18
Boltzmann Learning
- Stochastic learning algorithm rooted in statistical mechanics.
- Recurrent network, binary neurons (on: ‘+1’, off: ‘-1’).
- Energy function E:
E = −
X
j
X
k,k 6 =j
wkj xk xj
- Activation: - Choose a random neuron k. - Flip state with a probability (given temperature T )
P (xk → −xk ) =
1 + exp(−∆Ek /T )
where ∆Ek is the change in E due to the flip.
Boltzmann Machine
- Two types of neurons - Visible neurons: can be affected by the environment - Hidden neurons: isolated
- Two modes of operation - Clamped : visible neuron states are fixed by environmental input and held constant. - Free-running : all neurons are allowed to update their activity freely.
Learning without a Teacher
Two classes
- Reinforcement learning (RL)/Neurodynamic programming
- Unsupervised learning/Self-organization
25
Learning without a Teacher: Reinforcement Learning
- Learning input-output mapping through continued interaction with the environment.
- Actor-critic : cricit converts primary reinforce- ment signal into higher-quality, heuristic rein- forcement signal (Barto, Sutton, ...).
- Goal is to optimize the cumulative cost of actions.
- In many cases, learning is under delayed rein- forcement. Delayed RL is difficult since (1) teacher does not provide desired action at each step, and (2) must solve temporal credit-assignment problem.
- Relation to dynamic programming , in the context of optimal control theory (Bellman).
26
Learning without a Teacher: Unsupervised Learning
- Learn based on task-independent measure of the quality of representation.
- Internal representations for encoding features of the input space.
- Competetive learning rule needed, such as winner-takes-all.
Learning Tasks, Memory, and Adaptation
Learning tasks
- Pattern association
- Pattern recognition
- Function approximation
- Control
- Filtering/Beamforming Memory and adaptation
Pattern Association
- Associtive memory : brainlike distributed memory that learns association. Storage and retrieval ( recall ).
- Pattern association (xk : key pattern , yk : memorized pattern ):
xk → yk , k = 1, 2 , ..., q
- autoassociation (xk = yk ): given partial or corrupted version of stored pattern and retrieve the original. - heteroassociation (xk 6 = yk ): Learn arbitrary pattern pairs and retrieve them.
- Relevant issues: storage capacity vs. accuracy.
29
Pattern Classification
- Mapping between input pattern and a pre- scrived number of classes (categories).
- Two general types: - Feature extraction (observation space to feature space: cf. dimensionality reduc- tion), then classification (feature space to decision space). - Single step (observation space to deci- sion space).
30
Function Approximation
- Nonlinear input-output mapping: d = f (x) for an unknown f.
- Given a set of labeled examples T = {(xi, di)}Ni=1, estimate
F(·) such that
‖F(x) − f (x)‖ < , for all x
Function Apprix: System Identification and Inverse
System Modeling
- System identification: learn function of an unknown system.
d = f (x)
- Inverse system modeling: learn inverse function:
x = f −^1 (d)
Memory
- Memory : relatively enduring neural alterations induced by an organism’s interaction with the environment.
- Memory needs to be accessible by the nervous system to influence behavior.
- Activity patterns need to be stored through a learning process.
- Types of memory: short-term and long-term memory.
37
Associative Memory
xk1 yk xk2 yk
xkm ykm
x (^) kj yki wij(k)
... ... ... ...
q pattern pairs: (xk , yk ), for k = 1, 2 , ..., q.
- Input (key vector) xk = [xk 1 , xk 2 , ..., xkm]T^.
- Output (memorized vector) yk = [yk 1 , yk 2 , ..., ykm]T^.
- Weights can be represented as a weight matrix:
yk = W(k)xk , for k = 1, 2 , ..., q
yki =
X^ m
j=
wij (k)xkj , for m = 1, 2 , ..., m
38
Associative Memory (cont’d)
- Weight matrix: yk = W(k)xk , for k = 1, 2 , ..., q
yki =
X^ m j=
wij (k)xkj , for i = 1, 2 , ..., m
yki = [wi 1 (k), wi 2 (k), ..., wim(k)]
xk 1 xk 2 : xkm
, i = 1, 2 , ..., m
yk 1 yk 2 : ykm
w 11 (k), w 12 (k), ..., w 1 m(k) w 21 (k), w 22 (k), ..., w 2 m(k) ... wm 1 (k), wm 2 (k), ..., wmm(k)
xk 1 xk 2 : xkm
Associative Memory (cont’d)
- With a single W(k), we can only represent one mapping (xk to
yk ). For all pairs (xk , yk ) (k = 1, 2 , ..., q), we need q such
weight matrices.
yk = W(k)xk , for k = 1, 2 , ..., q
- One strategy is to combine all W(k) into a single memory
matrix M by simple summation:
M =
X^ q
k=
W(k)
- Will such a simple strategy work? That is, can the following be
possible with M?
yk ≈ Mxk , for k = 1, 2 , ..., q
Associative Memory: Example – Storing Multiple
Mappings
With fixed set of key vectors xk , an m × m matrix can store m arbitrary output vectors yk.
- Let xk = [0, 0 , ... 1 , ...0]T^ where only the k-th element is 1 and all the rest is 0.
- Construct a memory matrix M with each column representing the arbitrary output vectors yk :
M =
4 y^1 ,^ y^2 , ...,^ ym
- Then, yk = Mxk , for all k = 1, 2 , ..., m.
- But, we want xk to be arbitrary too!
41
Correlation Matrix Memory
- With q pairs (xk , yk ), we can construct a candidate memory matrix that stores all q mappings as:
M^ ˆ =
X^ q k=
yk xTk = y 1 xT 1 + y 2 xT 2 + ... + yq xTq ,
where yk xTk represents the outer product of vectors that results in a matrix, i.e., (yk xTk )ij = ykixkj.
- A more convenient notation is:
M^ ˆ =
h y 1 , y 2 , ..., yq
i
x 1 x 2 ... xq
= YXT^.
This can be verified easily using partitioned matrices. 42
Correlation Matrix Memory: Recall
- Will Mxˆ k give yk?
- For convenience, let’s say
M^ ˆ =
X^ q
k=
yk xTk =
X^ q
k=
W(k).
- First, consider W(k) = yk xTk only.
Check if W(k)xk = yk :
W(k)xk = yk xTk xk = yk (xTk xk ) = cyk
where c = xTk xk , a scalar value (the length of vector xk
squared). If all xk s were normalized to have length 1,
W(k)xk = yk will hold!
Correlation Matrix Memory: Recall (cont’d)
- Now, back to Mˆ: under what condition will Mxˆ j give yj for all j? Let’s begin by assuming xTk x= k 1 (key vectors are normalized).
- We can decompose Mxˆ j as follows:
Mx^ ˆ j =
X^ q k=
yk xTk xj = yj xTj xj +
X^ q k=1,k 6 =j
yk xTk xj.
- We know yj xTj xj = yj , so it now becomes:
Mx^ ˆ j = yj + X^ q k=1,k 6 =j
yk xTk xj | {z } Noise term
.
- If all keys are orthogonal (perpendicular to each other), then for an arbitrary k 6 = j, xTk xj = ‖xk ‖‖xj ‖ cos(θkj ) = 1 × 1 × 0 = 0 , so the noise term becomes 0, and hence Mxˆ j = yj + 0 = yj. The example in page 41 is one such (extreme) case!
Statistical Nature of Learning (cont’d)
- Neural network realization of the regressive model:
Y = F (X, w).
We want to map the knowledge in the training data T into the
weights w.
- We can now define the cost function:
E(w) =
X^ N
i=
(di − F (xi, w))^2
which can be written equivalently as an average over the training
set ET [·]:
E(w) =
ET
(di − F (x, T ))^2
49
Statistical Nature of Learning (cont’d)
d−F (x, T ) = d−f (x)+f (x)−F (x, T ) = +(f (x)−F (x, T )). With that,
E(w) =^1 2
ET
h (di − F (x, T ))^2
i becomes
=^1
ET
h ( + (f (x) − F (x, T ))^2
i
=^1
ET
h ^2 + 2(f (x) − F (x, T )) + (f (x) − F (x, T ))^2
i
ET [^2 ]
| {z } Intrinsic error
- ET [(f (x) − F (x, T ))] | {z } This reduces to 0
+^1
ET [(f (x) − F (x, T ))^2 ] | {z } We’re interested in this!
50
Statistical Nature of Learning: Bias/Variance Dillema
The cost function we derived
ET [(f (x) − F (x, T ))^2 ]
can be rewritten, knowing f (x) = E[D|x]:
ET [(E[D|x] − F (x, T ))^2 ] = ET [(E[D|x] − ET [F (x, T )] + ET [F (x, T )] − F (x, T ))^2 ] = (ET [F (x, T )] − E[D|x])^2 | {z } Bias
- ET [(F (x, T ) − ET [F (x, T )])^2 ]. | {z } V ariance
The last step above is obtained using ET [E[D|x]^2 ] = E[D|x]^2 , ET [ET [F (x, T )]^2 ], = ET [F (x, T )]^2 , and ET [E[D|x]F (x, T )] == E[D|x]ET [F (x, T )].
- Note: E[c] = c and E[cX] = cE[X] for constant c and random variable X.
Bias/Variance Dillema (cont’d)
- The bias indicates how much F (x, T ) differs from the true
function f (x): approximation error
- The variance indicates the variance in F (x, T ) over the entire
training set T : estimation error
- Typically, achieving smaller bias leads to higher variance, and smaller variance leads to higher bias.
Statistical Learning Theory
- Statistical learning theory addresses the fundamental issue of how to control the generalization ability of a neural network in mathematical terms.
- Certain quantities such as sample size and the Vapnik-Chevonenkis dimension (VC dimension) is closely related to the bounds on generalization error.
- The probably approximately correct (PAC) learning model is another framework to study such bounds. In this case, the the
confidence δ (probably) and tolerable error level (approximately
correct) are important quantities. Given these, and other measures such as the VC dimension, we can calculate the sample complexity (how many samples are needed to achieve
that level of correctness with that much confidence δ).
53
Appendix on VC Dimension
- The concept of Shattering
- VC dimension
54
Shattering a Set of Instances
Definition: a dichotomy of a set S is a partition of S into
two disjoint subsets.
Definition: a set of instances S is shattered by a function
class F if and only if for every dichotomy of S there exists
some function in F consistent with this dichotomy.
Three Instances Shattered
Instance space X
Each closed contour indicates one dichotomy. What kind of classifier function can shatter the instances?
