Download Turbo Codes: Overview, Capacity, Software, and Performance and more Slides Theories of Communication in PDF only on Docsity!
Turbo Codes
Apr. 26, 2007
Matthew Valenti
West Virginia University
Morgantown, WV 26506-
Turbo Codes
Overview
Channel capacity
Convolutional codes
Turbo codes
- Standard binary turbo codes: UMTS and cdma
- Duobinary CRSC turbo codes: DVB-RCS and 802.
Turbo Codes
Software to Accompany Tutorial
Iterative Solution’s Coded Modulation Library (CML) is a library for
simulating and analyzing coded modulation.
Available for free at the Iterative Solutions website:
- www.iterativesolutions.com
Runs in matlab, but uses c-mex for efficiency.
Supported features:
- Simulation of BICM
- Turbo, LDPC, or convolutional codes.
- PSK, QAM, FSK modulation.
- BICM-ID: Iterative demodulation and decoding.
- Generation of ergodic capacity curves (BICM/CM constraints).
- Information outage probability in block fading.
- Calculation of throughput of hybrid-ARQ.
Implemented standards:
- Binary turbo codes: UMTS/3GPP, cdma2000/3GPP2.
- Duobinary turbo codes: DVB-RCS, wimax/802.16.
- LDPC codes: DVB-S2.
Turbo Codes
Noisy Channel Coding Theorem
Claude Shannon, “A mathematical theory of communication,” Bell
Systems Technical Journal , 1948.
Every channel has associated with it a capacity C.
- Measured in bits per channel use (modulated symbol).
The channel capacity is an upper bound on information rate r.
- There exists a code of rate r < C that achieves reliable communications.
- Reliable means an arbitrarily small error probability.
Turbo Codes
Capacity of AWGN with
BPSK Constrained Input
If we only consider antipodal (BPSK) modulation, then
and the capacity is:
X = ± E s
C I X Y
I X Y
H Y H N
p y p y dy eN
p x
p x p
o
z
m ax ;
log ( ) log
l a^ fq
a f
a f b g
maximized when
two signals are equally likely
This term must be integrated numerically with
p Y ( ) y = p X ( ) y ∗ p N ( ) y = p X ( ) p N ( y − ) d
−∞
∞
z^ λ λ^ λ
Capacity of AWGN w/ 1-D Signaling
-2 -1 0 1 2 3 4 5 6 7 8 9 10
Eb/No in dB
BPSK Capacity Bound
Code Rate r
Shannon Capacity Bound
Spectral Efficiency
It is theoretically possible to operate in this region.
It is theoretically impossible to operate in this region.
Power Efficiency of Standard
Binary Channel Codes
Mariner 1969
Turbo Code 1993
Galileo:BVD Galileo:LGA 1992 1996
Pioneer 1968-
Voyager 1977
Odenwalder Convolutional Codes 1976
-2 -1 0 1 2 3 4 5 6 7 8 9 10
Eb/No in dB
BPSK Capacity Bound
Code Rate r
Shannon Capacity Bound
Uncoded BPSK
IS- 1991
Iridium 1998
5 10 − Pb =
Spectral Efficiency
arbitrarily low BER:
LDPC Code 2001 Chung, Forney, Richardson, Urbanke
Turbo Codes
Binary Convolutional Codes
A convolutional encoder comprises:
- k input streams
- We assume k=1 throughout this tutorial.
- n output streams
- m delay elements arranged in a shift register.
- Combinatorial logic (OR gates).
- Each of the n outputs depends on some modulo-2 combination of the k current inputs and the m previous inputs in storage
The constraint length is the maximum number of past and present
input bits that each output bit can depend on.
D D Constraint Length K = 3
S 0
S 3
S 2
S 1
1/
i = 0 i = 1 i = 2 i = 3 i = 4 i = 5 i = 6
Trellis Diagram
initial state
Every branch corresponds to a particular data bit and 2-bits of the code word
new state after first bit is encoded
final state m = 2 tail bits
1/10 1/
every sequence of input data bits corresponds to a unique path 1/01 through the trellis
input and output bits for time L = 4
Turbo Codes
Recursive Systematic
Convolutional (RSC) Codes
An RSC encoder is constructed from a standard convolutional encoder
by feeding back one of the outputs.
An RSC code is systematic.
- The input bits appear directly in the output.
An RSC encoder is an Infinite Impulse Response (IIR) Filter.
- An arbitrary input will cause a “good” (high weight) output with high
probability.
- Some inputs will cause “bad” (low weight) outputs.
D D
x i
ri
D D
Turbo Codes
State Diagram of RSC Code
With an RSC code, the output labels are the same.
However, input labels are changed so that each state has
an input “0” and an input “1”
Messages labeling transitions that start from S 1 and S 2 are
complemented.
S 3 = 11
S 2 = 01
S 1 = 10
S 0 = 00 S 3 = 11
S 0
S 3
S 2
S 1
0/
i = 0 i = 1 i = 2 i = 3 i = 4 i = 5 i = 6
Trellis Diagram of RSC Code
m = 2 tail bits no longer all-zeros must be calculated by the encoder
0/10 0/
Turbo Codes
Determining Message Bit Probabilities
from the Branch Probabilities
Let p i,j(t) be the probability that the encoder made a
transition from Si to Sj at time t, given the entire
received codeword.
- pi,j(t) = P( Si(t-1) Æ Sj(t) | y )
- where S (^) j(t) means that S(t)=Sj
For each t,
The probability that u(t) = 1 is
Likewise
p^ 1,
p0,
p 0,
p
2,
p^ 1,
p
3,
p 3,
p
2,
S 0
S 1
S 2
S 3
S 0
S 1
S 2
S 3
∑ (^ ( −^1 )→ ()| )=^1
Si → Sj
P Si t Sj t y
∑ (^ ) → =
= = − → : 1
( () 1 | ) ( 1 ) ()| S Su
i j i j
P ut y PS t S t y
∑ (^ ) → =
= = − → : 0
( () 0 | ) ( 1 ) ()| S Su
i j i j
P ut y PS t S t y
Turbo Codes
Determining the Branch Probabilities
Let γi,j (t) = Probability of transition from state Si
to state Sj at time t, given just the received
word y (t)
- γi,j (t) = P( Si (t-1) Æ Sj (t) | y (t) )
Let αi(t-1) = Probability of starting at state Si at
time t, given all symbols received prior to time t.
- αi (t-1) = P( Si (t-1) | y (1), y (2), …, y (t-1) )
βj = Probability of ending at state Sj at time t,
given all symbols received after time t.
- βj (t) = P( Sj (t) | y (t+1), …, y (L+m) )
Then the branch probability is:
- p (^) i,j (t) = αi(t-1) γi,j (t) βj (t)
γ1,
3,
2,
Turbo Codes
Computing α
α can be computed recursively.
Prob. of path going through Si (t-1) and
terminating at Sj(t), given y (1)… y (t) is:
Prob. of being in state Sj (t), given y (1)… y (t)
is found by adding the probabilities of the
two paths terminating at state S (^) j (t).
For example,
- α 3 (t)=α 1 (t-1) γ1,3(t) + α 3 (t-1) γ3,3(t)
The values of α can be computed for every
state in the trellis by “sweeping” through the
trellis in the forward direction.
γ1,
(t)
α 1 (t-1)
α 3 (t-1) α 3 (t)
γ3,3(t)
Turbo Codes
Computing β
Likewise, β is computed recursively.
Prob. of path going through Sj (t+1) and
terminating at Si(t), given y (t+1), …, y (L+m)
Prob. of being in state Si (t), given y (t+1), …,
y (L+m) is found by adding the probabilities
of the two paths starting at state Si (t).
For example,
- β 3 (t) = β 2 (t+1) γ1,2(t+1) + β 3 (t+1) γ3,3(t+1)
The values of β can be computed for every
state in the trellis by “sweeping” through the
trellis in the reverse direction.
3,2 (t+1)
β 3 (t)
β 2 (t+1)
β 3 (t+1)
γ3,3(t+1)
Turbo Codes
Overview of MAP algorithm
Label every branch of the trellis with γi,j(t).
Sweep through trellis in forward-direction to compute αi (t) at every
node in the trellis.
Sweep through trellis in reverse-direction to compute βj (t) at every
node in the trellis.
Compute the LLR of the message bit at each trellis section:
MAP algorithm also called the “forward-backward” algorithm (Forney).
[ ]
[ ]
∑
∑
−
−
=
=
=
( 1 ) () ()
( 1 ) () ()
log
() 0 |
() 1 | () log
S S u
i ij j
S Su
i ij j
i j
i j
t t t
t t t
Put
Put t
α γ β
α γ β
λ y
y
Turbo Codes
Log Domain Decoding
The MAP algorithm can be simplified by performing in the
log domain.
- exponential terms (e.g. used to compute γ) disappear.
- multiplications become additions.
- Addition can be approximated with maximization.
Redefine all quantities:
- γi,j (t) = log P( Si (t-1) Æ Sj(t) | y (t) )
- αi (t-1) = log P( Si (t-1) | y (1), y (2), …, y (t-1) )
- βj (t) = log P( Sj (t) | y (t+1), …, y (L+m) )
Details of the log-domain implementation will be
presented later…
Turbo Codes
Parallel Concatenated Codes
with Nonuniform Interleaving
A stronger code can be created by encoding in parallel.
A nonuniform interleaver scrambles the ordering of bits at the input
of the second encoder.
- Uses a pseudo-random interleaving pattern.
It is very unlikely that both encoders produce low weight code words.
MUX increases code rate from 1/3 to 1/2.
RSC
RSC
Nonuniform Interleaver
MUX
Input
Parity Output
Systematic Output
x i
Turbo Codes
Random Coding Interpretation
of Turbo Codes
Random codes achieve the best performance.
- Shannon showed that as n→∞, random codes achieve channel
capacity.
However, random codes are not feasible.
- The code must contain enough structure so that decoding can be
realized with actual hardware.
Coding dilemma :
- “All codes are good, except those that we can think of.”
With turbo codes:
- The nonuniform interleaver adds apparent randomness to the
code.
- Yet, they contain enough structure so that decoding is feasible.
Turbo Codes
The Turbo-Principle
Turbo codes get their name because the decoder uses
feedback, like a turbo engine.
0
Eb/No in dB
BER
1 iteration
2 iterations
6 iterations 3 iterations
10 iterations
18 iterations
Performance as a Function of
Number of Iterations
K = 5
r = 1/
L= 65,
- interleaver size
- number data bits
Log-MAP algorithm
Turbo Codes
Summary of Performance Factors
and Tradeoffs
Latency vs. performance
- Frame (interleaver) size L
Complexity vs. performance
- Decoding algorithm
- Number of iterations
- Encoder constraint length K
Spectral efficiency vs. performance
Other factors
- Interleaver design
- Puncture pattern
- Trellis termination
Tradeoff: BER Performance versus
Frame Size (Latency)
K = 5
Rate r = 1/
18 decoder iterations
AWGN Channel
0
Eb/No in dB
B E R
K=
K=
K=
K=
Turbo Codes
UMTS Interleaver:
Inserting Data into Matrix
Data is fed row-wise into a R by C matrix.
- R = 5, 10, or 20.
- 8 ≤ C ≤ 256
- If L < RC then matrix is padded with dummy characters.
X 33 X 34 X 35 X 36 X 37 X 38 X 39 X (^40)
X 25 X 26 X 27 X 28 X 29 X 30 X 31 X (^32)
X 17 X 18 X 19 X 20 X 21 X 22 X 23 X (^24)
X 9 X 10 X 11 X 12 X 13 X 14 X 15 X (^16)
X 1 X 2 X 3 X 4 X 5 X 6 X 7 X (^8)
In the CML, the UMTS interleaver is created by the function CreateUMTSInterleaver
Interleaving and Deinterleaving are implemented by Interleave and Deinterleave
Turbo Codes
UMTS Interleaver:
Intra-Row Permutations
Data is permuted within each row.
- Permutation rules are rather complicated.
- See spec for details.
X 40 X 36 X 35 X 39 X 37 X 38 X 33 X (^34)
X 26 X 28 X 27 X 31 X 29 X 30 X 25 X (^32)
X 18 X 22 X 21 X 23 X 19 X 20 X 17 X (^24)
X 10 X 12 X 11 X 15 X 13 X 14 X 9 X (^16)
X 2 X 6 X 5 X 7 X 3 X 4 X 1 X (^8)
Turbo Codes
UMTS Interleaver:
Inter-Row Permutations
Rows are permuted.
- If R = 5 or 10, the matrix is reflected about the middle row.
- For R=20 the rule is more complicated and depends on L.
X 2 X 6 X 5 X 7 X 3 X 4 X 1 X (^8)
X 10 X 12 X 11 X 15 X 13 X 14 X 9 X (^16)
X 18 X 22 X 21 X 23 X 19 X 20 X 17 X (^24)
X 26 X 28 X 27 X 31 X 29 X 30 X 25 X (^32)
X 40 X 36 X 35 X 39 X 37 X 38 X 33 X (^34)
Turbo Codes
UMTS Interleaver:
Reading Data From Matrix
Data is read from matrix column-wise.
Thus:
- X’ 1 = X 40 X’ 2 = X 26 X’ 3 = X 18 …
- X’ 38 = X 24 X’ 2 = X 16 X’ 40 = X 8
X 2 X 6 X 5 X 7 X 3 X 4 X 1 X (^8)
X 10 X 12 X 11 X 15 X 13 X 14 X 9 X (^16)
X 18 X 22 X 21 X 23 X 19 X 20 X 17 X (^24)
X 26 X 28 X 27 X 31 X 29 X 30 X 25 X (^32)
X 40 X 36 X 35 X 39 X 37 X 38 X 33 X (^34)
