Gaussian Random Vectors
Definition: Random variables xi, i ∈Zare jointly Gaussian, if any finite
linear combination
a1xi1+a2xi2+· · · +anxin,
ak∈ <, ik∈Z, 1≤k≤n, is a Gaussian random variable.
Definition: A random vector ~x is called Gaussian, if ~x = [x1, . . . , xn]T,
where xiare jointly Gaussian random variables. We denote this as ~x ∼N(~m, K),
where E[~x] = ~m, and Cov(~x) = K.
If ~x ∼N(~m, K), then the characteristic function is
Φ~x(u) = E[exp j~uT~x] = exp −1
2~uTK~u +j~uT~m
If ~x ∼N(~m, K), and det(K) = |K| 6= 0 then,
fx(~x) = 1
(2π)n/2|K|1/2exp −1
2(~x −~m)TK−1(~x −~m)
MMSE Estimation and Gaussian Vectors:
Let ~x, and ~y , be jointly Gaussian random vectors (JGRV)s. Let e=x−ˆ
E[~x|~y],
where ˆ
E[~x|~y] is the LMMSE estimate of ~x given ~y, i.e.,
ˆ
E[~x|~y] = E[~x] + Cov(~x, ~y)Cov(~y , ~y )−1(~y −E[~y]) ,
then
~x =~e +ˆ
E[~x|~y]
Note that by construction as an LMMSE estimate, ~e is orthogonal to ~y , by
the orthogonality principle. Since ~e is a linear function of JGRVs, ~e and ~y
are also jointly Gaussian. Hence, since they are orthogonal, and ~e has zero
mean, then they are necessarily independent (recall that uncorellatedness im-
plies independence for JGRVs). (NOTE that this step does not hold for
the case where ~x and ~y are not JGRVs, and hence is why the LMMSE and
MMSE estimates do not coincide in general). Now, since ~e ∼N(0,Σe,e), where
Σe,e = Cov(~x, ~x)−Cov(~x, ~y )Cov(~y , ~y )−1Cov(~y , ~x).Then, given ~y =~u, the con-
ditional distribution of ~x is Nˆ
E[~x|~y =~u],Σe,e . Note that since ~e and ~y are
independent, the conditional distribution of ~e given ~y is the same as the uncon-
ditional distribution of ~e.
Corollary: For ~x and ~y JGRVs, E[~x|~y] = ˆ
E[~x|~y].
For a more straightforward, bruteforce proof of this result, see the text by
R. Gray, “Introduction to Statistical Signal Processing,” available online via
link from the course web page. The proof begins on page 215, and requires the
matrix inversion lemma and a bit of algebra, but is otherwise straightforward.
1