Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
The exponentially weighted recursive least squares (ew-rls) algorithm for parameter adaptation in multiuser communications. The theory behind the algorithm, including the cost function, recursive least squares, and matrix inversion lemma. It also includes the parameter update recursion and the advantages and disadvantages of using ew-rls.
Typology: Study notes
1 / 41
This page cannot be seen from the preview
Don't miss anything!
ECE595: Multiuser Communications
Dr. Sudharman K. Jayaweera
Assistant Professor
Department of Electrical and Computer Engineering
University of New Mexico
Lecture 12 - November
th
, Tuesday
Fall 2007
ECE595: Multiuser Communications
Recursive Least Squares Parameter Adaptive Algorithm
Outline
A New Cost Function for Adaptive Algorithm Construction
-
Minimizing Least Squares Error
ErrorMinimizing Least Squares Error Vs. Minimizing Mean Squared
Exponentially Weighted Recursive Least Squares
-
Deterministic Normal Equations
Recursive Parameter Update Equations
RLS Algorithm Initialization
LMS Vs. RLS: Pros and Cons
Sliding Window Recursive Least Squares
-
A Two-step Algorithm
ECE595: Multiuser Communications
Least Squares Error Cost Function
information aboutA perfectly valid cost function that does not require any statistical
x ( k )
and
d ( k )
is the
Least Squares Error
0 | e ( i ) | 2
Adaptive filters can be designed so that at each time instant
k
they
update the parameter vector
θθθ ( k (^) +
in order to minimize this least
squares error, in contrast to mean squared error as we did earlier
Recursive Least Squares
(RLS) algorithm performs this
minimization efficiently
ECE595: Multiuser Communications
Mean Squared Error Vs. Least Squares Error
Minimizing the
mean square error
ξ ( k ) =
e ( k ) | 2 }
produces the
same set of coefficients
θθθ ( k )
for all sequences (of
x ( k )
and
d ( k ) ) that
have the same statistics
-
their statistical averagesi.e. the coefficients do not depend on the particular data but on
The least squares approach minimizes the
least squares error
k ) =
0 (^) | e ( i ) | 2
that depend on the specific values of the
incoming data sequence
-
Filter coefficients will be optimal
only
for the given data set and
different realizations of
x ( k )
and
d ( k )
lead to different solutions
even if they all have the same statistical properties
ECE595: Multiuser Communications
and
θθθ ( k )
θ 0 ( k )
θ 1 ( k )
θ N (^) ( k )
and
x ( i ) =
x ( i )
x ( i (^) −
x ( i (^) −
Note that, the coefficients
θθθ ( k )
are held constant over the entire
observation interval
(^) k ] in computing the cost function (although
the true parameter values used at each time
i can be different from
each other)
ECE595: Multiuser Communications
Exponentially Weighted RLS: Minimizing the Least Squares Error
Coefficients that minimize the least squares error
k )
should set the
derivative of
k )
with respect to each of the
θ n ( k ) , for
n
=
equal to zero:
∂θ
n ( k ) = k
0 λ k − i e ( i ) ∂ e ( i )
∂θ
n ( k )
Using (2) and (3)
k
0 λ k − i e ( i
x ( i (^) −
(^) n
)
for
n
=
Using (3) again k
0 λ k − i [ d ( i )
N
m = 0 θ m ( k ) x ( i
(^) m
x ( i (^) −
(^) n
)
for
n
=
ECE595: Multiuser Communications
Vector Deterministic Normal Equations
In vector notation deterministic normal equations become:
x ( k ) θθθ ( k ) = r d x ( k )
where we have defined the
exponentially
weighted deterministic autocorrelation matrix R
x ( k )
of
x ( k )
as
R x ( k ) = k
0 λ k − i x ( i ) x ( i ) T
and the
deterministic cross-correlation r
d x ( k )
between data
x ( k )
and the desired output
d ( k )
is:
r d x ( k ) = k
0 λ k − i d ( i ) x ( i ) ( N +
(^) vector
and
x
( i )
is the
-vector
x
( i ) = [
x ( i ) , (^) x
( i (^) −
(^) x
( i (^) −
T
.^
ECE595: Multiuser Communications
Least Squares Optimal Coefficients Vector
minimizes the least squares error cost function:From (6) we have the exact solution to the parameter vector that
θθθ ( k ) = R x ( k ) − 1 r d x ( k )
Recall, the optimum MMSE parameter set
θθθ opt
is:
θθθ opt
x ( k ) x ( k ) T
− 1 E^
(^) {
x ( k ) d ( k ) }
Compare the similarity with the Least Squares Error Solution:
R x ( k ) = k
0
λ k − i x ( i ) x ( i ) T
and
r d x ( k ) =
k
0
λ k − i d ( i ) x ( i
ECE595: Multiuser Communications
EW-RLS: Minimum Least Squares Error (2/2)
Using (8) we may write the minimum least squared error
k ) min
as
k ) min
= ‖ d ( k ) ‖
λ 2
− r d x ( k ) T
θθθ^ ( k )
vector aswhere we have defined the weighted norm of the desired output
‖ d ( k ) ‖
λ 2
k
0 λ k − i d 2 ( i
and the desired output vector
d ( k )
is
d ( k )
d ( k )
d ( k (^) −
d
( 0
)
k (^) +
(^) vector
ECE595: Multiuser Communications
Recursive Least Squares
From (15), the optimal least squares weight vector is:
θθθ ( k ) = R x ( k ) − 1 r d x ( k )
Since both
x ( k )
and
r d x ( k )
depend on
k , solving the deterministic
normal equations
directly
, as in (15), requires these quantities to be
computed again and again at each time instant
k
Recursive Least Squares
algorithm computes these quantities
recursively thereby minimizing the computational complexity
as: It also allows the least squares solution (15) to be found recursively
θθθ
( k )
θθθ
( k
(^) −
θθθ
( k (^) −
ECE595: Multiuser Communications
Matrix Inversion Lemma
Suppose that
and
− 1
(^) DA
− 1 B
are nonsingular square
matrices. Then,
− 1 B
(^) ( C
− 1
(^) DA
− 1
If
b
and
d T
are vectors, then applying matrix inversion
lemma gives (in this case
is a scalar and we take it as
(^) bd
T (^) ) − 1 = A − 1 − A − 1
bd
T A^
− 1
(^) d
T
A^
− 1 b
ECE595: Multiuser Communications
EW-RLS: Recursion for R
x ( k ) − 1
R Applying the matrix inversion lemma (20) to (17), we may compute
x ( k ) − 1
recursively as:
R x ( k ) − 1 = λ − 1 R x ( k
) − 1 − λ − 2 R x ( k −
) − 1 x ( k ) x ( k ) T
x ( k
−
(^1)
) −
(^) λ
− 1 x ( k ) T
x ( k (^) −
) − 1 x ( k )
Define the
inverse autocorrelation matrix P
k )
as
P ( k ) = R x ( k ) − 1
and the
gain vector g
k )
as
g ( k ) = λ − 1 R x ( k
) − 1 x ( k )
(^) λ
− 1 x ( k ) T
x ( k
−
(^1)
) − 1 x ( k ) = λ − 1 P ( k
x ( k )
(^) λ
− 1 x ( k ) T
k
−
(^1)
) x ( k )
Then, recursion for
R x ( k ) − 1
becomes:
P ( k ) = λ − 1
k (^) −
(^) g
( k ) x ( k ) T
k (^) −
17
ECE595: Multiuser Communications
EW-RLS: Parameter Update Recursion (1/2)
From (15) and (21)
θθθ ( k ) = P ( k ) r d x ( k )
Substituting for
r d x ( k )
from (18)
θθθ ( k ) = λ P ( k ) r d x ( k
(^) d
( k ) P ( k ) x ( k )
k (^) −
(^) g
( k ) x ( k ) T
k (^) −
r^ d x ( k (^) −
(^) d
( k ) P ( k ) x ( k )
(using (23) for
k ) )
k
−
(^1)
) r d x ( k −
(^) g
( k ) x ( k ) T
k (^) −
) r d x ( k
(^) d
( k ) g ( k )
(using (25) for
g ( k ) )
θθθ ( k
−
(^1)
) (^) −
(^) g
( k ) x ( k ) T
θθ^ θ ( k (^) −
(^) g
( k ) d ( k )
(using (15))
θθθ ( k
−
(^1)
) +
(^) g
( k ) (^) [ d ( k ) (^) −
(^) θ θθ ( k (^) −
T x^ ( k ) ]
19
ECE595: Multiuser Communications
EW-RLS: Parameter Update Recursion (2/2)
update is of the formHence, exponentially weighted Recursive Least Squares parameter
θθθ ( k )
θθθ ( k (^) −
(^) α
( k ) g ( k )
where
a priori error
α
( k )
is the error that would occur if the filter
coefficients were not updated. i.e. if the old parameters
θθθ ( k (^) −
were
used with the new data
x ( k ) :
α ( k ) = d ( k )
(^) θθ θ
( k
(^) −
T
x^
( k )
