Download Lecture Slides on Advanced Video Coding | ECE 6258 and more Study notes Digital Signal Processing in PDF only on Docsity!
11/17/
ECE 6258 Russell M. Mersereau
1
ECE6258 Lecture 35
Advanced Video Coding
11/17/
ECE 6258 Russell M. Mersereau
What is fixed and what is free ^
All of the image and video compressionstandards define the syntax of the bit-streamand the operation of the decoder.
^
The detailed operation of the encoder andoperation of some parts of the decoder areleft up to the manufacturer.
^
What can be optimized?
11/17/
ECE 6258 Russell M. Mersereau
3
Decoder options ^
Post-processing ^
Blocking artifact removal
There are many possible reconstructed images that are consistentwith the bit stream. Find the one that is the smoothest. ^
Error concealment
If a macroblock or a slice is lost in transmission, try to estimate itusing neighboring blocks, or blocks borrowed from earlier frames. ^
Image Enhancement
11/17/
ECE 6258 Russell M. Mersereau
Encoder Options ^
Distribution of I, P, and B frames
I-frames require the most bits, B-frames thefewest
^
Method for motion estimation
Accuracy versus bit-rate versus complexity
^
Prediction mode for individual macroblocksfor P and B frames ^
Intracoded, forward prediction, backwardprediction, bidirectional prediction ^
1 or 4 motion vectors per macroblock
^
Slice size
11/17/
ECE 6258 Russell M. Mersereau
5
Encoder Options (2) ^
Rate-Control ^
Needed to prevent buffer over/underflow ^
Allocation of bits to I, P, and B frames ^
Adjustment of quantization step size.
^
Selection of quantization table
^
Error resilience features (MPEG4) ^
Reversible Huffman codes ^
Resynchronization markers ^
Error-control codes ^
Motion vectors on I-frames
11/17/
ECE 6258 Russell M. Mersereau
ITU H.264 ^
This is the newest, and most powerful of thecoding standards (2003). ^
~2x coding gain over MPEG-
^
Optimal implementations still under study.
^
Other names for the same standard: ^
MPEG-4 Part 10 ^
MPEG-4 Advanced Video Coding (AVC) ^
H.26L ^
JVT
11/17/
ECE 6258 Russell M. Mersereau
7
Why people are excited
11/17/
ECE 6258 Russell M. Mersereau
Encoder Block Diagram
Deblocking
11/17/
ECE 6258 Russell M. Mersereau
13
4x4 Block-based Spatial Prediction ^
9 prediction modes(directions) ^
Mode 0 (DC) ^
Predict all pixels from(A+B+C+D+I+J+K+L)/ ^
Mode 1 (vertical) ^
a,e,I,m predicted from A;b,f,j,n from B … ^
Mode 2 (horizontal) ^
a,b,c,d from I; e,f,g,h fromJ, … ^
Mode 3 (NW-SE) ^
A,f,k,p from (A+2Q+I+2)/4; ^
e,j,o from (Q+2I+J+2)/4… ^
^
Mode 8
p o n m L
l k j i K
h g f e J
d c b a I
D C B A Q
11/17/
ECE 6258 Russell M. Mersereau
8x8 Spatial Prediction (Chrominance)^ ^
8 x 8 chroma macroblock consists offour 4 x 4 blocks ABCD.
^
4 prediction cases depending uponwhether S0, S1, S2, or S3 are inside oroutside
^
If all inside ^
A=(S0+S2+4)/
^
B=(S1+2)/
^
C=(S3+2)/
^
D=(S1+S3+4)/8S0, S1, S2, and S3 are the sums offour neighboring samples
A^
B
C
D
SO
S
S2 S
11/17/
ECE 6258 Russell M. Mersereau
15
Temporal prediction: multiple reference frames
Currentframe
11/17/
ECE 6258 Russell M. Mersereau
Motion Estimation ^
16 x 16 Macroblock ^
Prediction block structure ^
16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x
0 0
0 0
0 0
(^0101)
1 1
1 1 2 2
3 3
MB-Modes 8x8-Modes
16x
16x
8x
8x
8x
8x
4x
4x
11/17/
ECE 6258 Russell M. Mersereau
17
Motion Vector Coding ^
Median prediction(except for 16x8 and8x16 modes)
^
Median of A,B, and C(normally)
^
A,B,C, and D can befrom different referenceframes
^
¼ pixel resolution
D^
B
A
C
E
11/17/
ECE 6258 Russell M. Mersereau
B-Frame coding: motion vector coding ^
Modes ^
Direct, forward, backward, bi-directional, intra ^
16x16, 8x16, 16x8, 8x8, 4x8, 8x4, 4x
^
Direct Mode ^
No MV data is transmitted ^
Same block structure as co-located MB intemporally subsequent picture ^
MVs are computed as scaled version ofcorresponding MV of the co-located block
11/17/
ECE 6258 Russell M. Mersereau
19
“DCT” Transform Computation (1) ^
Start with the 4x4 DCT
c
a
b
a
b
a
c
a
b
a
c
a
c a b a x x x x
x
x
x
x
x
x
x
x
x x x x c b b c
a
a
a
a
b
c
c
b
a
a
a
a
AXA
Y^
T
44 43 42 41
34 33 32 31
24 23 22 21
14 13 12 (^11) (
) (^
)^
... 65
(^3826834323). 0 (^8) / 3 cos 2 / 1
... 8
(^6532814243). 0 (^8) /
cos 2 (^2) / / 1 1
=
=
=
= =
π π
a b c
11/17/
ECE 6258 Russell M. Mersereau
“DCT” Transform Computation (2)^ ^
Manipulate
−
−
− −
− −
−
−
− −
− −
= =
b a b a d
d d
d
x x x x
x x x x
x x x x
x x x x d d
d d b a b a
B BCXC Y^
T
0 0 0
0 0 0
0 0 0
0 0 0 1 1 1
1 1
1
1 1
1
1 1 1
1 1
1 1 1 1
1
1
1 1 1 1 0 0 0
0 0 0
0 0 0
0 0 0
44 43 42 41
34 33 32 31
24 23 22 21
14 13 12 11
b c d^
/
(^
)
2
2
2
2
2
2
2
2
44 43 42 41
34 33 32 31
24 23 22 21
14 13 12 11
b ab b ab
ab a ab a
b ab b ab
ab a ab a d
d d
d
x x x x
x x x x
x x x x
x x x x d d
d d
E
CXC
Y^
T
