Download Characteristic functions, college study notes - Introduces characteristic functions. and more Study notes Signals and Systems Theory in PDF only on Docsity!
Characteristic Functions
Nick Kingsbury
This work is produced by The Connexions Project and licensed under the Creative Commons Attribution License †
Abstract This module introduces characteristic functions. You have already encountered the Moment Generating Function of a pdf in the Part IB probability course. This function was closely related to the Laplace Transform of the pdf. Now we introduce the Characteristic Function for a random variable, which is closely related to the Fourier Transform of the pdf. In the same way that Fourier Transforms allow easy manipulation of signals when they are convolved with linear system impulse responses, Characteristic Functions allow easy manipulation of convolved pdfs when they represent sums of random processes. The Characteristic Function of a pdf is de ned as:
ΦX (u) = E
[
eiux
]
−∞ e
iuxfX (x) dx
= F {−u}
where F {u} is the Fourier Transform of the pdf. Note that whenever fX is a valid pdf, Φ (0) =
fX (x) dx = 1 Properties of Fourier Transforms apply with −u substituted for ω. In particular:
- Convolution - (sums of independent rv's)
Y =
∑^ N
i=
Xi ⇒ fY = fX 1 ∗ fX 2 ∗ · · · ∗ fXN ⇒ ΦY (u) =
∏^ N
i=
(ΦXi (u)) (2)
2 π
e−(iux)ΦX (u) du (3)
ΦX (u) =
(ix)neiuxfX (x) dx ⇒ E [Xn] =
xnfX (x) dx =
in
dn dun^
ΦX (u) |u=0 (4)
∗Version 2.3: Jun 7, 2005 4:35 pm GMT- †http://creativecommons.org/licenses/by/1.
- Scaling If Y = aX, fY (y) = fX a^ ( x)from this equation in our previous discussion of functions of random variables, then ΦY (u) =
eiuy^ fY (y) dy
eiuaxfX (x) dx = ΦX (au)
1 Characteristic Function of a Gaussian pdf
The Gaussian or normal distribution is very important, largely because of the Central Limit Theorem which we shall prove below. Because of this (and as part of the proof of this theorem) we shall show here that a Gaussian pdf has a Gaussian characteristic function too. A Gaussian distribution with mean μ and variance σ^2 has pdf:
f (x) =
2 πσ^2
e
−
„ (x−μ)^2 2 σ^2
« (6)
Its characteristic function is obtained as follows, using a trick known as completing the square of the exponent:
ΦX (u) = E
[
eiux
]
eiuxfX (x) dx
= √ 21 πσ 2
e −
“ (^) x (^2) − 2 μx+μ (^2) − 2 σ (^2) iux 2 σ^2
” dx
√ 21 πσ 2
e
x−μ+iuσ^2 )^2 2 σ^2
!
dx
e^
2 iuσ^2 μ−u^2 σ^4 2 σ^2
= eiuμe −
“ (^) u (^2) σ 2 2
”
since the integral in brackets is similar to a Gaussian pdf and integrates to unity. Thus the characteristic function of a Gaussian pdf is also Gaussian in magnitude, e −
“ (^) u (^2) σ 2 2
” , with standard deviation (^) σ^1 , and with a linear phase rotation term, eiuμ, whose rate of rotation equals the mean μ of the pdf. This coincides with standard results from Fourier analysis of Gaussian waveforms and their spectra (e.g. Fourier transform of a Gaussian waveform with time shift).
2 Summation of two or more Gaussian random variables
If two variables, X 1 and X 2 , with Gaussian pdfs are summed to produce X, their characteristic functions will be multiplied together (equivalent to convolving their pdfs) to give
ΦX (u) = ΦX 1 (u) ΦX 2 (u)
= eiu(μ^1 +μ^2 )e
− u^2 (σ 12 +σ 22 ) 2
! (8)
This is the characteristic function of a Gaussian pdf with mean ( μ 1 + μ 2 ) and variance ( σ 12 + σ 22 ). Further Gaussian variables can be added and the pdf will remain Gaussian with further terms added to the above expressions for the combined mean and variance.
These are all constants, independent of N , and dependent only on the shape of the pdfs fXi. Substituting these moments into (11) and (12) and using the series expansion, log (1 + x) = x + (terms of order x^2 or smaller), gives
log (ΦX (u)) = N log
√u N
= N log
1 − u
2 2 N σ
= N
u^2 σ^2 2 N
u^2 σ^2 2
where ** represents the terms of order N −(^
or smaller and ## represents the terms of order N −(^
or smaller. As N → ∞,
log (ΦX (u)) → −
u^2 σ^2 2
Therefore, as N → ∞
ΦX (u) → e −
“ (^) u (^2) σ 2 2
” (14) Note that, if the input pdfs are symmetric, the skewness will be zero and the error terms will decay as N −^1
rather than N −(^
1 2 ); and so convergence to a Gaussian characteristic function will be more rapid. Hence we may now infer from (6), (7) and (14) that the pdf of X as N → ∞ will be given by
fX (x) =
2 πσ^2
e−
“ (^) x 2 2 σ^2
” (15)
Thus we have proved the required central limit result. Figure 1(a) shows an example of convergence when the input pdfs are uniform, and N is gradually increased from 1 to 50. By N = 12, convergence is good, and this is how some 'Gaussian' random generator functions operate - by summing typically 12 uncorrelated random numbers with uniform pdfs. For some less smooth or more skewed pdfs, convergence can be slower, as shown for a highly skewed triangular pdf in Figure 1(b); and pdfs of discrete processes are particularly problematic in this respect, as illustrated in Figure 1(c).
(a)
(b)
