Download Vector Quantization: Lecture Notes by Russell M. Mersereau and more Study notes Digital Signal Processing in PDF only on Docsity!
10/3/^
ECE 6258 Russell M. Mersereau
1
ECE6258 Lecture 19
Vector Quantization
10/3/^
ECE 6258 Russell M. Mersereau
Quantization Intervals
^ Interval boundaries-∞,^ x
,^ x ,^ x 12
,^ x ,^ ∞ 34 ^ Representation(reproduction)values^ y ,^ y^1
,^ y ,^ y 23
,^ y 4 5
r
Q [ r ] xx^1 y^5 y^4^ xx^3 4^ y^2^ y^1
10/3/^
ECE 6258 Russell M. Mersereau
3
Optimum Quantizers ^ Two conditions must be true for a compressorconsisting of an encoder followed by a decoder tobe optimal^ ^ The
encoder
must be optimal for the given decoder. The^ decoder
must be optimal for the given encoder. Necessary and sufficient conditions can be found forthese two constraints separately.
10/3/^
ECE 6258 Russell M. Mersereau
The optimal encoder for a given decoder ^ i.e. given {
y , y ,…,^12
y^ } find { N
x ,…, x^^0 N
^ The best encoder maps input values into the outputreproduction level having the minimum distortionwith respect to the input.^ Î
encoder uses a nearest neighbor rule ^ For both squared and absolute error distortions
10/3/^
ECE 6258 Russell M. Mersereau
5
The optimal decoder for a given encoder ^ Given a nondegenerate partition {
R }, the unique i
optimal codebook for the mean squared error isgiven by
Centroid condition
10/3/^
ECE 6258 Russell M. Mersereau
The Lloyd iteration for codebook improvement 1.^ Given a codebook
C^ ={ ym
}, find the optimal i
partition into quantization cells using thenearest neighbor condition.2. Using the centroid condition, find
C^ m+
10/3/^
ECE 6258 Russell M. Mersereau
7
The Lloyd algorithm 1.^ Begin with an initial codebook,
C. Set^1
m =1.
2.^ Given codebook
C^ , perform a Lloyd m iteration to generate an improved codebook C^. m +13. Compute the average distortion for
C^. If it m +
has changed by a small enough amountsince the last iteration, stop. Else
m +1Æ
m
and go to Step 2.
10/3/^
ECE 6258 Russell M. Mersereau Vector Quantization: Motivation ^ For lossless compression, improved compression ispossible by coding symbols in groups. ^ VQ applies the same approach to quantization^ ^ Group samples into vectors^ ^
Colored pixels Blocks of pixels Sets of parameters ^ Shannon showed that optimal rate-distortionperformance is approached as vector dimensionalitybecomes large
10/3/^
ECE 6258 Russell M. Mersereau
13
The Generalized Lloyd Algorithm ^ For optimal VQ^ ^ The decision boundaries should be chosen tosatisfy the nearest neighbor rule:^ ^ The
Y should lie at the centroids of their i^ respective regions.
10/3/^
ECE 6258 Russell M. Mersereau
The Algorithm (impractical version) 1.^ Start with an initial set of reconstruction values,(0)^ Y i
. Set^ k =0,
(0)^ D =∞
. Select threshold
2.^ Find decision regions3.^ Compute the distortion4.^ If [ D
( k -1( k ))- D
( k)^ ]/^ D <
ε^ stop; else continue.
5.^ k ++; Find new reconstruction values that are thecentroids of {
( k -1)^ V }. Go to Step 2. i
10/3/^
ECE 6258 Russell M. Mersereau
15
Working with training data ^ The Generalized Lloyd algorithm, asdescribed, is quite impractical.^ ^ Assumes analytic form for densities.^ ^ Involves difficult integrals for distortion andcentroids ^ It is much more common to work with trainingdata.
10/3/^
ECE 6258 Russell M. Mersereau Generalized Lloyd Algorithm with Training Data 1.^ Start with an initial set of reconstruction values,
(0)^ Y and i
training vectors
X. Set^ n
(0) k =0, D =∞. Select threshold
ε.
2.^ The quantization regions are given by3.^ Compute the average distortion
( k)^ D between the training
vectors and the reproduction values.( k -1^ 4. If [ D )-
( k )( k)D ]/^ D <^ ε^ stop; else continue.
5.^ k ++; Find new reconstruction values that are the means ofthe training vectors falling in each of the {
( k -1) V }. Go to Step i
10/3/^
ECE 6258 Russell M. Mersereau
17
Issues ^ Special actions needed for emptyquantization regions.^ ^ If a reproduction value has no vectors associatedwith it, replace it by a value from the mostpopulous cell. ^ The Generalized Lloyd Algorithm with trainingdata is not guaranteed to converge to theoptimal solution. ^ The final solution depends on the startingcodebook.
10/3/^
ECE 6258 Russell M. Mersereau
Pyramid Splitting ^ Proposed by Linde, Buzo, and Gray to avoidbad solutions^ ^ 1. Find optimal 1 vector codebook,
V.^1
^ 2. Find optimal 2 vector codebook using
V ,^ V +^11
^ 3. Find optimal 4 vector codebook using resultsfrom 2 level codebook plus small perturbations. ^ etc.
10/3/^
ECE 6258 Russell M. Mersereau
19
Pairwise Nearest Neighbors Approach ^ Start with as many clusters as trainingvectors. ^ At each stage combine the two closestvectors into a single cluster and update themean. ^ Continue until the number of clusters isreduced to
M. ^ Produces the best designs, but iscomputationally expensive.
10/3/^
ECE 6258 Russell M. Mersereau Complexity ^ L =: dimension of the vector ^ r =: rate (bits/sample) ^ N =
rL =: size of codebook ^ Encoding:^ ^ Each vector needs to be compared with everyvector in the codebook.^ ^ # searches
rL ≈ N = ^ Grows exponentially with rate and dimension.
