Download Midterm Exam 1 Solution and more Exams Probability and Statistics in PDF only on Docsity!
EECS 126 Probability and Random Processes University of California, Berkeley: Fall 2015 Kannan Ramchandran September 22, 2015
Midterm Exam 1 Solution
Last name First name SID
Name of student on your left:
Name of student on your right:
- DO NOT open the exam until instructed to do so.
- The total number of points is 110 , but a score of ≥ 100 is considered perfect.
- You have 10 minutes to read this exam without writing anything and 90 minutes to work on the problems.
- Box your final answers.
- Partial credit will not be given to answers that have no proper reasoning.
- Remember to write your name and SID on the top left corner of every sheet of paper.
- Do not write on the reverse sides of the pages.
- All eletronic devices must be turned off. Textbooks, computers, calculators, etc. are prohibited.
- No form of collaboration between students is allowed. If you are caught cheating, you may fail the course and face disciplinary consequences.
- You must include explanations to receive credit.
Problem Part Max Points Problem Part Max Points 1 (a) 8 2 20 (b) 8 3 25 (c) 8 4 25 (d) 8 (e) 8 40
Total 110
Cheat sheet
1. Discrete Random Variables
1) Geometric with parameter p ∈ [0, 1] :
P (X = n) = (1 − p)n−^1 p, n ≥ 1
E[X] = 1 p , var(X) =
1 −p p^2
2) Binomial with parameters N and p :
P (X = n) =
(N
n
pn(1 − p)N^ −n, n = 0,... , N , where
(N
n
= (N −Nn^ !)!n!
E[X] = N p , var(X) = N p(1 − p)
3) Poission with parameter λ :
P (X = n) = λ
n
n! e
−λ, n ≥ 0
E[X] = λ , var(X) = λ
2. Continuous Random Variables
1) Uniformly distributed in [a, b] , for some a < b :
fX (x) = b−^1 a 1 {a ≤ x ≤ b}
E[X] = a+ 2 b, var(X) =
(b−a)^2 12
2) Exponentially distributed with rate λ > 0 :
fX (x) = λe−λx 1 {x ≥ 0 }
E[X] = 1 λ , var(X) = λ^12
3) Gaussian, or normal, with mean μ and variance σ^2 :
fX (x) = √^1
2 πσ^2
exp {−
(x−μ)^2
2 σ^2 }
E[X] = μ , var(X) = σ^2
(b) You and your friend have created a new language in order to text message with each other. The language consists of an alphabet of 7 letters: {A, B, C, D, E, F, G}. Unfortunately, your friend has very strict parents who cap the amount of data she receives over text messaging. You decide to use Huffman coding to compress your messages as much as possible. Assume that the frequency of letters in your language is according to the following table:
P (X) X 0.15 A 0.05 B 0.2 C 0.05 D 0.05 E 0.15 F 0.35 G
Now, instead of using binary codewords, we will use ternary codewords (e.g. { 0 , 1 , 2 } ). Construct a ternary Huffman code for this alphabet and find the expected length of a codeword. Solution: The Huffman tree is shown in Figure 1.
Figure 1: Huffman tree
The codewords are shown below:
X codeword A 10 B 110 C 0 D 111 E 112 F 12 G 2
The expected length of a codeword is
2 × 0 .15 + 3 × 0 .05 + 1 × 0 .2 + 3 × 0 .05 + 3 × 0 .05 + 2 × 0 .15 + 1 × 0 .35 = 1. 6
(c) Consider a regular hexagon whose side length is 1. Five points A , B , C , D and E are chosen on the perimeter of the hexagon randomly as follows. First, we pick one of the 6 sides of the hexagon uniformly at random. Then, the position of point A is chosen uniformly at random on the picked side. We then pick one of the remaining 5 sides of the hexagon and decide the position of point B uniformly at random on the chosen side. We repeat this procedure until we locate all five points. Then, we draw edges between (A, B) , (B, C) , (C, D) , (D, E) and (E, A). The figure formed by the 5 edges can be a “star” as in Figure 2(a), or not a “star” as in Figure 2(b). What is the probability that the formed figure is a “star”?
Figure 2: Examples.
Solution: Fix the position of A , there are 4! possible permutations of the other points. Among these permutations, only 2 of them are “stars”, as shown in Figure 3. Therefore, the probability is 2 4!
Figure 3: Stars.
(e) A fair hundred-sided die (with values 0, 1 , 2 ,... , 99 ) is rolled independently 50 times, with outcomes X 1 ,... , X 50. Let Z be the empirical mean of the 50 rolls, i.e., Z = 501
i=1 Xi^ , and Y be the maximum value attained over the 50 rolls, i.e. Y = max{X 1 , X 2 ,... , X 50 }. Find E[Z|Y ] as a function of Y. Solution: We know that there must exist j (1 ≤ j ≤ 50) such that Xj = Y. For i 6 = j , conditioned on Y , Xi ’s are i.i.d. uniformly distributed in the set { 0 ,... , Y } , and E[Xi|Y ] = Y 2. Then we know
E[Z|Y ] =
[E[Xj |Y ] +
i 6 =j
E[Xi|Y ]]
[Y +
Y ]
Y.
Problem 2. (20 points) Consider two continuous random variables X and Y that are uniformly distributed with their joint probability density function (PDF) equal to A over the shaded region as shown below.
Figure 4: Joint PDF of X and Y
(a) What is A? Solution: The area of the shaded square is 2. In order that the PDF is valid, the integration of the PDF over the shaded square should be one. Therefore, A = 1/.
(b) Are X and Y independent? Are X and Y uncorrelated? Explain. Solution: X and Y are not independent. To see this, think about fX,Y (0. 75 , 0 .75) , fX (0.75) , and fY (0.75). According to the joint PDF, fX,Y (0. 75 , 0 .75) = 0. However, as for the marginal distribution, we can see that fX (0.75) > 0 and fY (0.75) > 0. Then we have fX,Y (0. 75 , 0 .75) 6 = fX (0.75)fY (0.75). Therefore, X and Y are not independent. Since the joint PDF is symmetrical with respect to the x -axis and y -axis, we know that E[XY ] = 0 , E[X] = 0 , and E[Y ] = 0. Then cov(X, Y ) = E[XY ] − E[X]E[Y ] = 0. Therefore, X and Y are uncorrelated.
Problem 3. (25 points) In an EE126 office hour, students bring either a difficult-to-answer question with probability p = 0.2 or an easy-to-answer question with probability 1 − p = 0.. A GSI takes a random amount of time to answer a question, with this time duration being exponentially distributed with rate μD = 1 (question per minute)-where D denotes “difficult”- if the problem is difficult, and μE = 2 (questions per minute)-where E denotes “easy”-if the problem is easy.
(a) Is the amount of time taken by the GSI to answer a random question exponentially dis- tributed? Justify your answer. Solution: The amount of time taken by the GSI to answer a random question is a mixture of two exponential distributions. Therefore, the time is not exponentially distributed.
One day, you visit an office hour and find the GSI answering the question of another student who came before you. You find out that the GSI has been busy with the other student’s question for T = ln 4 ' 1. 3863 (minutes) before you come. Assume that you are the only student waiting in the room.
(b) Conditioned on the fact that the GSI has been busy with the other student’s question for T minutes, let q be the conditional probability that the problem is difficult. Find the value of q. Solution: Denote by X the random amount of time to answer a question and by Z the indicator of the event that the problem being answered is difficult. Then,
P (X > t|Z = 0) = e−μE^ t P (X > t|Z = 1) = e−μD^ t
for t ≥ 0. Thus,
P (X > t) = pe−μD^ t^ + (1 − p)e−μE^ t^ = 0. 2 e−t^ + 0. 8 e−^2 t.
As the GSI has been helping the other student for T time units, we know X > T. Thus,
q = P (Z = 1|X > T ) =
P (Z = 1, X > T )
P (X > T )
pe−μD^ T pe−μD^ T^ + (1 − p)e−μE^ T
=
- 2 e−T
- 2 e−T^ + 0. 8 e−^2 T^
1 + 4e−T^
(c) Conditioned on the information above, you want to find the expected amount of time you have to wait from the time you arrive until the other student’s question is answered. Express your answer in terms of q of part (b). Solution: Using the memoryless property of exponential random variables,
E[X − T |X > T ] =P (Z = 0|X > T )E[X − T |X > T, Z = 0] + P (Z = 1|X > T )E[X − T |X > T, Z = 1]
=(1 − q)
μE
μD
1 + q 2
Problem 4. (25 points) There are N (N ≥ 2) bins and two colors of balls, black and white. In each experiment, d (1 ≤ d < N ) bins are chosen uniformly at random from the N bins. In each chosen bin, we place either a black ball or a white ball in that bin. The probability that we place a black ball is p , and the probability that we place a white ball is 1 − p. All placements are independent. This experiment is conducted K times independently. Let Bi and Wi be the number of black balls and white balls in the i th bin after the K experiments, respectively. An example of one experiment is shown in Figure 5.
To clarify, N and d are constants. In part (a), (b), and (c), K is a constant. In part (d), K is a random variable.
Figure 5: An example of one experiment with N = 7 bins and d = 3 balls.
(a) For the i th bin, what is the probability mass function (PMF) of Bi? Solution: Since there are K independent experiments and in each experiment, the prob- ability that a black ball is placed in the i th bin is dpN. Then we know that Bi ∼ Binomial(K, dpN ) , and
P (Bi = x) =
K
x
dp N )x(1 − dp N )K−x.
(b) For the i th bin, what is the joint PMF of Bi and Wi , i.e., P (Bi = x, Wi = y)? Are Bi and Wi independent? Solution: We have P (Bi = x, Wi = y) = P (Bi = x, Bi + Wi = x + y) = P (Bi + Wi = x + y)P (Bi = x|Bi + Wi = x + y).
Since Bi + Wi is the total number of balls in a bin, we have Bi + Wi ∼ Binomial(K, (^) Nd ). Conditioned on Bi + Wi = x + y , the distribution of Bi is also binomial, with parameters x + y and p. Therefore we have
P (Bi = x, Wi = y) =
K
x + y
d N
)x+y(1 − d N
)K−(x+y)
x + y x
px(1 − p)y
K!
(K − x − y)!x!y!
dp N )x( d(1 − p) N )y(1 − d N )K−(x+y).
Bi and Wi are not independent. According to part (a), Bi ∼ Binomial(K, dpN ) , and similarly, Wi ∼ Binomial(K, d(1 N− p)). Then we can see that P (Bi = x, Wi = y) 6 = P (Bi = x)P (Wi = y).
(c) We call a bin a mixed bin if the bin contains both black and white balls. Suppose after K experiments, we find that the first bin is empty. Conditioned on this, what is the probability that the second bin is a mixed bin? Solution: Let E 1 be the event that the first bin is empty, and M 2 be the event that the second bin is a mixed bin. We want to find P (M 2 |E 1 ). We have
P (M 2 |E 1 ) = P (B 2 ≥ 1 , W 2 ≥ 1 |E 1 ) = 1 − P (B 2 = 0 or W 2 = 0|E 1 ) = 1 − [P (B 2 = 0|E 1 ) + P (W 2 = 0|E 1 ) − P (B 2 = 0, W 2 = 0|E 1 )].
Conditioned on the event that the first bin is empty, we know that in each experiment, the d balls are chosen uniformly at random from all the bins except the first bin. Therefore, we have
P (B 2 = 0|E 1 ) = (1 − dp N − 1
)K
P (W 2 = 0|E 1 ) = (1 −
d(1 − p) N − 1
)K
P (B 2 = 0, W 2 = 0|E 1 ) = (1 −
d N − 1
)K
Therefore, we know that
P (M 2 |E 1 ) = 1 − (1 −
dp N − 1
)K^ − (1 −
d(1 − p) N − 1
)K^ + (1 −
d N − 1
)K^.
END OF THE EXAM.
Please check whether you have written your name and SID on every page.
