Docsity
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Docsity AI
Docsity AI
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Solutions to Problem Set #3 in ECE 413: Probability Theory - Prof. Dilip Sarwate, Assignments of Statistics

University of Illinois - Urbana-ChampaignStatistics
Prof. Dilip Sarwate

The solutions to problem set #3 in the ece 413: probability theory course, focusing on the law of the unconscious statistician (lotus) and the poisson distribution. Topics covered include the function being nonnegative, possible values of y, calculation of mean and variance, and exponential moments.

Typology: Assignments

Pre 2010

Uploaded on 03/16/2009

koofers-user-qmd
koofers-user-qmd 🇺🇸

9 documents

1 / 2

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
ECE 413 Solutions to Problem Set #3 Spring 2007
1. Illustration of the law of the unconscious statistician (LOTUS) with a ramp
pmf
(a) The function is nonnegative. We thus just have to check that the total mass is one, or
equivalently, that 1+2+· ··+n=n(n+1)/2.One way to verify this identity is to note that the
sum is half as much as 1+2+···+n+n+·· ·+2+1 = (1+n)+(2+n−1)+·· ·+(n+1) = n(n+1).
Or it can be shown by induction on n.
(b) The possible values of Yare 1,1/2,1/3,···,1/n. We can write
pY(u) = (2
un(n+1) (or, equivalently, 2k
n(n+1) ) if u= 1/k for some kwith 1 ≤k≤n
0 else
The pmf of Yfor the case n= 3 is:
0
1/6
2/6
3/6
u
Y
p (u)
1/3 1/2 1
(c) By definition, E[Y] = PuupY(u).There are nnonzero terms in this sum. which each have
the form u·2
un(n+1) =2
n(n+1) .Thus, all nterms are 2
n(n+1) , so we have E[Y] = n·2
n(n+1) =2
n+1 .
(d) Using LOTUS we get, more directly, E[Y] = Pkg(k)pX(k) = Pn
k=1 1
k
2k
n(n+1) =2n
n(n+1) =
2
n+1 .
2. Mean, variance, and exponential moments of the Poisson distribution
(a) P∞
i=0
e−λλi
i!=e−λP∞
i=0
λi
i!=e−λeλ= 1.
(b) Since, by convention, 0! = 1 and λ0= 1, we find P(X= 0) = e−λλ0
0! =e−λ.
(c) E[X] = P∞
i=0 ie−λλi
i!. The ith term is zero for i= 0, so the index ican be taken to run
from i= 1 to ∞. For i≥1, i
i!=1
(i−1)!..Therefore, the sum becomes E[X] = P∞
i=1
e−λλi
(i−1)|.
Letting j=i−1, and using the fact λi=λλi−1=λλj, the sum can be written as E[X] =
λP∞
j=0
e−λλj
j|=λ, where for the last equality we used the fact, verified in part (a), that the
total mass is one.
(d) Using the LOTUS, we find E[X(X−1)] = P∞
i=0 i(i−1)e−λλi
i|. The ith term is zero for
i= 0 and i= 1, so those terms can be left out of the summation. For i≥2, i(i−1)
i!=1
(i−2)!.
Therefore, the sum becomes E[X(X−1)] = P∞
i=2
e−λλi
(i−2)! Letting k=i−2, and using the fact
λi=λ2λk, the sum can be written as E[X(X−1)] = λ2P∞
k=0
e−λλk
k!=λ2,where for the last
equality we used the fact, verified in part (a), that the total mass is one.
(e) The facts E[X] = λand E[X2]−E[X] = λ2, yield that E[X2] = λ2+λ. Therefore,
Var(X) = E[X2]−E[X]2= (λ2+λ)−λ2=λ. That is, both the mean and variance of X
are λ.
(f) E[zX] = P∞
i=0 zie−λλi
i|=e−λP∞
i=0
(λz)i
i!=e−λeλz =e−λ(1−z).
1
pf2

Partial preview of the text

Download Solutions to Problem Set #3 in ECE 413: Probability Theory - Prof. Dilip Sarwate and more Assignments Statistics in PDF only on Docsity!

ECE 413 Solutions to Problem Set #3 Spring 2007

  1. Illustration of the law of the unconscious statistician (LOTUS) with a ramp pmf (a) The function is nonnegative. We thus just have to check that the total mass is one, or equivalently, that 1+2+· · ·+n = n(n+1)/ 2. One way to verify this identity is to note that the sum is half as much as 1+2+· · ·+n+n+· · ·+2+1 = (1+n)+(2+n−1)+· · ·+(n+1) = n(n+1). Or it can be shown by induction on n. (b) The possible values of Y are 1, 1 / 2 , 1 / 3 , · · · , 1 /n. We can write

pY (u) =

{ (^2) un(n+1) (or, equivalently,^

2 k n(n+1) )^ if^ u^ = 1/k^ for some^ k^ with 1^ ≤^ k^ ≤^ n 0 else The pmf of Y for the case n = 3 is:

0

1/

2/

3/

u

pY (u)

1/3 1/2 1

(c) By definition, E[Y ] =

∑ u upY^ (u).^ There are^ n^ nonzero terms in this sum. which each have the form u· (^) un(n^2 +1) = (^) n(n^2 +1). Thus, all n terms are (^) n(n^2 +1) , so we have E[Y ] = n· (^) n(n^2 +1) = (^) n+1^2. (d) Using LOTUS we get, more directly, E[Y ] =

∑ k g(k)pX (k) =^

∑n k=

1 k

2 k n(n+1) =^

2 n n(n+1) = 2 n+.

  1. Mean, variance, and exponential moments of the Poisson distribution (a)

∑∞ i=

e−λλi i! =^ e

−λ ∑∞ i=

λi i! =^ e

−λeλ (^) = 1. (b) Since, by convention, 0! = 1 and λ^0 = 1, we find P (X = 0) = e

−λλ 0 0! =^ e

−λ.

(c) E[X] =

∑∞ i=0 i e−λλi i!. The^ ith term is zero for^ i^ = 0, so the index^ i^ can be taken to run from i = 1 to ∞. For i ≥ 1, (^) ii! = (^) (i−^1 1)!.. Therefore, the sum becomes E[X] =

∑∞ i= e−λλi (i−1)|. Letting j = i − 1, and using the fact λi^ = λλi−^1 = λλj^ , the sum can be written as E[X] =

λ

∑∞ j=

e−λλj j| =^ λ,^ where for the last equality we used the fact, verified in part (a), that the total mass is one. (d) Using the LOTUS, we find E[X(X − 1)] = ∑∞ i=0 i(i^ −^ 1) e−λλi i|. The^ ith term is zero for i = 0 and i = 1, so those terms can be left out of the summation. For i ≥ 2, i(i− i! 1)= (^) (i−^1 2)!. Therefore, the sum becomes E[X(X − 1)] =

∑∞ i=

e−λλi (i−2)! Letting^ k^ =^ i^ −^ 2, and using the fact λi^ = λ^2 λk, the sum can be written as E[X(X − 1)] = λ^2

∑∞ k=

e−λλk k! =^ λ

(^2) , where for the last equality we used the fact, verified in part (a), that the total mass is one. (e) The facts E[X] = λ and E[X^2 ] − E[X] = λ^2 , yield that E[X^2 ] = λ^2 + λ. Therefore, Var(X) = E[X^2 ] − E[X]^2 = (λ^2 + λ) − λ^2 = λ. That is, both the mean and variance of X are λ. (f) E[zX^ ] =

∑∞ i=0 z i e−λλi i| =^ e

−λ ∑∞ i=

(λz)i i! =^ e

−λeλz (^) = e−λ(1−z).

  1. Estimation of the parameter of a Poisson random variable (a) Given n, we wish to select λ to maximize λ ne−λ n!. Equivalently, we select^ λ^ to maximize f (λ) = λne−λ^ over λ ≥ 0. If n = 0, this function is e−λ, which is decreasing in λ, so that λ̂ = 0 if n = 0. So now suppose n ≥ 1. In that case, f ′(λ) = (n−λ)λn−^1 e−λ. Thus, f ′(λ) > 0 for λ < n, and f ′(λ) < 0 for λ > n. Therefore, f is strictly increasing over the interval [0, n] and strictly decreasing over the interval [n, +∞). Thus, the point λ = n (where f ′(λ) = 0) must be the point at which f is maximized. Thus, ̂λ = n for n ≥ 1. Combining the cases n = 0 and n ≥ 1, we see that λ̂ = n for all values of n ≥ 0. (b) Given m, we wish to select b to maximize (b

(^2) )me−b^2 m!. This is the same as maximizing^ f^ (b

with respect to b, where f is the same as in part (a) with n replaced by m. We know from part (a), that if b is selected so that b^2 = m, then f (b^2 ) is the largest that it can be. Thus ̂ b = √m is the maximum likelihood estimate of b. (Note, if negative values of b were allowed,

−

m would also be a maximum likelihood estimate.)

  1. Parameter estimation for an increasing ramp pmf The problem is to maximize pX (k) with respect to n, for k fixed. As long as n ≥ k, this probability is strictly positive, and it is zero otherwise. So the maximizing value, n̂ , must be greater than or equal to k. Now, for n ≥ k, the probability is (^) n(n^2 k+1) , which is strictly decreasing in n. Thus, the maximizing value of n is n̂ = k.
  2. Estimating n from observation of a binomial random variable (a) Consider a trial to be a placement of one defect, and call a trial a success if the defect is placed in the region examined. Then X is the number of successes in n independent trials, where the probability of success of each trial is p = 0.25. So X is a binomial random variable with parameters n and p. More explicitly, the pmf of X is given by pX (k) =

(n k

) pk(1 − p)n−k, for 0 ≤ k ≤ n. (b) Given that X = k (for example, k = 8 in the problem statement), we seek the value ̂n of n that maximizes an =

(n k

) pk(1 − p)n−k. Examine the ratio

rn =

an an− 1

n(1 − p) n − k

for n ≥ k + 1. The ratio rn is strictly decreasing in n. We can select n̂ to be the largest n such that rn ≥ 1, or equivalently, the largest n such that n ≤ kp , yielding n̂ = bkp c. If kp is an integer, then rkp = 1, so in that case there are two values of n that maximize the likelihood: k p −^ 1 or^

k p. Thus, ̂ n =

{ bkp c if kp is not an integer k p or^

k p −^1 if^

k p is an integer For the particular problem, either n̂ = 31 or ̂n = 32 is a maximum likelihood estimate of n. (Note: If kp is an integer, the estimator bkp c is equal to kp. Another maximum likelihood estimator is dkp − 1 e, which is equal to kp − 1 if kp is an integer. The two estimators are equal if kp is not an integer.)