10
1
Widrow-Hoff Learning
(LMS Algorithm)
Docsity.com
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Widrow Hoff Learning - Banking - Lecture Slides, Slides of Banking and Finance
E Banking is closely associated with computer sciences. In these Lecture Slides, the lecturer has explained the following aspects of Banking : Widrow Hoff Learning, Adaline Network, Linear Neuron, Purelin, Input Neuron, Mean Square Error, Training Set, Notation, Mean Square Error, Square Error
Typology: Slides
2012/2013
1 / 26
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Widrow Hoff Learning - Banking - Lecture Slides and more Slides Banking and Finance in PDF only on Docsity!
Widrow-Hoff Learning
(LMS Algorithm)
Docsity.co
ADALINE Network
AAAAAA
a
purelin
Wp
b
Linear Neuron
p
a
n
AA
W
A
b
R (^) x (^1)
S (^) x (^) R
S (^) x (^1)
S (^) x (^1)
S (^) x (^1)
Input
R
S
a
purelin Wp
b
(
)
Wp
b
=
=
a i^
purelin n
i
purelin
w
T
i
p
b i^
w
T
i
p
b i^
i w
w i 1
,
w i 2
,
w i R ,
Docsity.co
Mean Square Error
p
1
t
1
{ , } p 2 t 2
{ , } … p Q t Q
{
,
}
,
,
,
Training Set:
p
q
t
q
Input:
Target:
x
w
1 b
z
1 p
a
w
T
p
b
=
a
x
T
z^
=
F
x
(
)
E e
2 ]^
[
=
E
t
a
(
) 2 ]^
[
E
t
x
T
z^
(
) 2 ]^
[
=
=
Notation:
Mean Square Error:
Docsity.co
Error Analysis
F
x
(
)
E e
2 ]^
[
=
E
t
a
(
) 2 ]^
[
E
t
x
T
z^
(
) 2 ]^
[
=
=
F
x
(
)
E t
2
2 t
x
T
z^
x
T
zz^
T
x^
]
[
=
F
x
(
)
E t
2 ]^
2
x
T E^
t
z
[
]
x
T E^
zz
T
[
]
x
[
F =
x
( ) c 2 x T
h^
x
T
Rx^
=
c
E t
2 ]^
[
=
h
E t
z
[
]
=
R
E
zz
T
[
]
=
F
x
c
d
T
x^
x
T
Ax^
d
2
h
=
A
2
R
=
quadratic function: The mean square error for the ADALINE Network is a
Docsity.co
Approximate Steepest Descent
F ˆ
x
(
)
t k (
)
a k (
)
( ) 2 e 2 k
(
)
=
=
Approximate mean square error (one sample):
∇ˆ F^
x
(
)
e 2 k
( )
∇
=
e 2 k
[
]
j^
e 2 k
w 1 j
,
∂^ ---------------
e k (
)
e k (
)
w 1 j
,
j
1 2
…
R
,
,
,
=
e 2
k
[
]
R
1
e 2 k
b
e k ( )
e k (
)
b
Approximate (stochastic) gradient:
Docsity.co
Approximate Gradient Calculation
e k (
)
w 1 j
,
t k (
)
a k (
)
[
]
w 1 j
,
w 1
j
,
t k (
)
w
T
1
p
k
b
[
]
e k (
)
w 1
j
,
w 1 j
,
t k (
)
w 1 i
, p i^ k
i
1
=^ ∑ R
b
e k ( )
w 1 j
,
p j^ k
e k (
)
b
∇ˆ F
x
(
)
e 2
k
( )
∇
2 e k (
)
z
k
(
)
=
=
Docsity.co
Multiple-Neuron Case
i w
k
1
(
)
i w
k
(
)
2 α e i^ k
(
)
p
k
(
)
=
b i^ k
1
(
)
b i^ k
(
)
2 α e i^ k
(
)
=
W
k
1
(
)
W
k
( )
2 α
e
k
(
)
p
T
k
(
)
=
b
k
1
(
)
b
k
( )
2 α
e
k
(
)
=
Matrix Form:
Docsity.co
Analysis of Convergence
x
k
1
x
k
2 α e k ( )
z
k
( )
=
E
x
k
1
[
]
E
x
k
[
]
2 α E e k
(
)
z
k
(
)
[
]
=
E
x
k
1
[
]
E
x
k
[
]
α
E t k
z
k
[
]
E
x
k T
z^
k
z
k
[
]
E
x
k
1
[
]
E
x
k
[
]
α
E t
k^
z
k
[
]
E
z
k
z
T
k
x
k
[
]
E
x
k
1
[
]
E
x
k
[ ] 2 α h R E x k
[
]
{
}
=
E
x
k
1
[ ] I 2 α R
[
] E
x
k
[
]
2 α
h
=
matrix must fall inside the unit circle.For stability, the eigenvalues of this
Docsity.co
Steady State Response
E
x
ss
[ ] I 2 α R
[
] E
x
ss
[
]
2 α
h
=
E
x
ss
[
]
R
1
h
x
E
x
k
1
[ ] I 2 α R
[
] E
x
k
[
]
2 α
h
=
If the system is stable, then a steady state condition will be reached.
The solution to this equation is
This is also the strong minimum of the performance index.
Docsity.co
Example
p
1
t
1
p
2
t
2
R
E
pp
T
[
]
p
1
p
1 T
p
2
p
2 T
R
λ 1
λ 2
λ 3
=
,
=
,
=
α
λ max
Banana
Apple
Docsity.co
Iteration Two
Apple
a
1
W
p
W
p
2
e =
1
(
)
t 1
(
)
a
1
( ) t 2 a 1
(
)
1
(
)
=
=
=
=
W
T
Docsity.co
Iteration Three
a
2
W
p
W
p
1
e =
2
(
)
t 2
(
)
a
2
(
)
t 1
a
2
(
)
1
(
)
=
=
=
=
W
3
(
)
W
2
( ) 2 α e 2
(
)
p
T
2
(
)
1.1040 0.
=
=
W
∞
(
)
1 0 0
=
Docsity.co
Example: Noise Cancellation
Adaptive Filter
60-Hz
Noise Source
Noise Path
Filter
EEG Signal (random)
Contaminating
Noise
Contaminated
Signal
"Error"
Restored Signal
Adaptive Filter Adjusts to Minimize Error (and in doing this removes 60-Hz noise from contaminated signal)
Adaptively Filtered Noise to Cancel Contamination
Graduate Student
v
m
s
t
a
e
Docsity.co
Noise Cancellation Adaptive Filter
a
k
n
k
SxR
Inputs
A
w
1,
AAAA
D
w
1,
ADALINE
AA
v ( k ) a ( k )
= w
1,
v
k
w
1,
v
k
Docsity.co