Preparing for Mid-term Exam: Probability, Convergence, MMSE Estimation, Gaussian Variables | Exams Electrical and Electronics Engineering

DetailedlistoftopicsyouneedtoknowforMid‐termExamI



Convergenceconcepts:

Definitionsofa.s.,m.s.,p.andd.convergences.Note:therandomvariablesinvolveddonotevenhave

tobedefinedoverthesameprobabilityspaceford.convergence.

Thevariousimplications:a.s.impliesp.,m.s.impliesp.andp.impliesd.(Iwon’taskyourepeatthe

proofsintheexam,buttheprooftechniquescouldbeuseful.Soyoushouldtrytounderstandthembut

youdon’thavetomemorizethem.)

Examplestoshowthat,ingeneral,a.s.doesnotimplym.s.,m.s.doesnotimplya.s.,p.doesnotimply

m.s.andp.doesnotimplya.s.

Whydowedefined.convergenceonlyatcontinuitypointsofthelimitingcdf?

d.convergenceiffcharacteristicfunctionsconvergepointwise.

Borel‐Cantellilemma(youdon’tneedtoknowtheproof)anditsapplicationtoshowthatm.s.

convergenceimpliesa.s.convergenceundersomeconditions.

Uniquenessoflimits:Ifasequenceofrandomvariablesconvergeinap.sensetotworandomvariables

XandY,thenP(X=Y)=1.(Sincea.s.,m.s.convergencealsoimplyp.convergence,theabovestatementis

alsotruefora.s.andm.s.convergence.)Theproofissimpleandyoushouldunderstandit.Ifasequence

ofrandomvariablesconvergesindistribution,thenthelimitingdistributionisunique(don’tneedto

knowtheproof).

SequencesofGaussianrandomvariablesconvergetoaGaussianrandomvariable(don’tneedtoknow

theproof).

Cauchycriteriaforrandomvariables(youdon’tneedtoknowtheproofs).

CorrelationversionoftheCauchyconvergencecriterionform.s.convergenceanditscorollaries(you

shouldunderstandtheproofs).

Thethreeformsofthelawoflargenumbersandtheirproofs.

Thecentrallimittheoremandthebasicideabehindtheproof:thekeyideaistoshowthatthe

characteristicfunctionsconverge.

ConvexfunctionsandJensen’sinequality.

TheChernoffboundanditsderivation.ThestatementofCramer’stheorem.



Partial preview of the text

Download Preparing for Mid-term Exam: Probability, Convergence, MMSE Estimation, Gaussian Variables and more Exams Electrical and Electronics Engineering in PDF only on Docsity!

Detailed list of topics you need to know for Mid‐term Exam I Convergence concepts: Definitions of a.s., m.s., p. and d. convergences. Note: the random variables involved do not even have to be defined over the same probability space for d. convergence. The various implications: a.s. implies p., m.s. implies p. and p. implies d. (I won’t ask you repeat the proofs in the exam, but the proof techniques could be useful. So you should try to understand them but you don’t have to memorize them.) Examples to show that, in general, a.s. does not imply m.s., m.s. does not imply a.s., p. does not imply m.s. and p. does not imply a.s. Why do we define d. convergence only at continuity points of the limiting cdf? d. convergence iff characteristic functions converge pointwise. Borel‐Cantelli lemma (you don’t need to know the proof) and its application to show that m.s. convergence implies a.s. convergence under some conditions. Uniqueness of limits: If a sequence of random variables converge in a p. sense to two random variables X and Y, then P(X=Y)=1. (Since a.s., m.s. convergence also imply p. convergence, the above statement is also true for a.s. and m.s. convergence.) The proof is simple and you should understand it. If a sequence of random variables converges in distribution, then the limiting distribution is unique (don’t need to know the proof). Sequences of Gaussian random variables converge to a Gaussian random variable (don’t need to know the proof). Cauchy criteria for random variables (you don’t need to know the proofs). Correlation version of the Cauchy convergence criterion for m.s. convergence and its corollaries (you should understand the proofs). The three forms of the law of large numbers and their proofs. The central limit theorem and the basic idea behind the proof: the key idea is to show that the characteristic functions converge. Convex functions and Jensen’s inequality. The Chernoff bound and its derivation. The statement of Cramer’s theorem.

MMSE Estimation: What is MMSE estimation? The orthogonality principle: An estimate is an MMSE estimate over a linear space of random variables V iff the error is orthogonal to Y for all Y in V. Expression for error variance. Conditional expectation: Suppose Y is a random variable and we are interested in estimating another random variable X as a function of Y. When joint pdf’s exist, it can be shown that the best function g(Y) which approximates X in the MMSE sense is the conditional expectation E(X|Y). The proof follows from the orthogonality principle. When the joint pdf doesn’t exist, the conditional expectation E(X|Y) is defined to be the MMSE estimate of X with V being the space of all (Borel‐measurable) functions of Y. Various facts about the MMSE estimates viewed as projections. MMSE estimation for vectors: it is really a collection of scalar MMSE estimates. In general, conditional expectation for vectors can be hard to derive, so we often restrict our estimates to be linear, called linear MMSE estimate. Derivation of optimal linear MMSE estimate: formulas for the optimal estimates and error covariance. The linear MMSE estimate depends only on mean vectors and cross‐covariance matrices. Properties of the covariance matrix of a random vector X: it is symmetric and positive semi‐definite. You should know how to check if a matrix is positive semi‐definite. You should know that a positive semi‐ definite matrix is diagonalizable using a unitary transformation. You show know how to diagonalize a positive semi‐definite matrix if such a matrix is given to you in the exam. Jointly Gaussian random variables and vectors: definitions, pdf of a Gaussian random vector, uncorrelatedness implies independence for Gaussian random variables. The conditional expectation for jointly Gaussian random vectors is the linear MMSE estimate. If X and Y are Gaussian, then the conditional expectation of X given Y=y is a Gaussian random variable with mean E(X|Y=y) and covariance Cov(e). Linear innovation sequence: recursive computation of the linear MMSE estimate using orthogonalization. There will not be any questions directly on the Kalman filter. But it may be useful to understand the Kalman filter as an application of the idea of linear innovations and the orthogonality principle.

Preparing for Mid-term Exam: Probability, Convergence, MMSE Estimation, Gaussian Variables, Exams of Electrical and Electronics Engineering

Related documents

Partial preview of the text

Download Preparing for Mid-term Exam: Probability, Convergence, MMSE Estimation, Gaussian Variables and more Exams Electrical and Electronics Engineering in PDF only on Docsity!