Download Probability with Engineering Applications: Problem Set 5 - Decision Theory - Prof. Dilip S and more Assignments Statistics in PDF only on Docsity!
UNIVERSITY OF ILLINOIS, URBANA-CHAMPAIGN
Department of Electrical and Computer Engineering
ECE 313: Probability with Engineering Applications-Fall 2003
Problem Set 5: An Introduction to Decision Theory
Issued: September 26, 2003 Due: October 3, 2003
Note: Class is cancelled on Friday, October 3. Homework should be deposited at Sung-Hoon Kim’s mailbox at EL 159 by 5:00 p.m. on Friday.
Reading Assignments: Ross, Chapter 3 and Notes on Decision Making (excerpted from notes of Prof. D.V. Sarwate).
Problems NOT to be turned in:
Problems Theoretical Exercises Chapter 3 14, 29, 55 1, 6, 14
Problems to be turned in:
- [10 pts] In the lecture and previous assignment, we have investigated bounds on the probability of a given event. There are also techniques to bound the expectation of a random variable. Below you will consider two examples. Each is a version of Jensen’s inequality: If f is a convex function^1 , and X a r.v., then E[f (X)] ≥ f (E[X]).
(a) Show that this is true when the function is affine: f (x) = a + bx, x ∈ R, for some constants a and b. (b) Show that this is true when f (x) = x^2 , x ∈ R. Hint: consider the representation of the variance of X in terms of first and second moments. (c) You will now establish Jensen’s inequality for f (x) = (^) x^1 , x > 0. First, note that for any a > 0,
x−^1 = f (x) ≥ fa(x) = a−^1 − a−^2 (x − a).
Sketch f and fa on a single graph for one fixed value of a > 0 to verify this. Argue that for any a, with this f, fa, E[f (X)] ≥ E[fa(X)]. Now, choose the best value of a in this bound: this yields Jensen’s inequality!
- [5 pts] An ectopic pregnancy is twice as likely to develop when the pregnant woman is a smoker as it is when she is a nonsmoker. If 32 percent of woman of childbearing age are smokers, what percentage of women having ectopic pregnancies are smokers?
- [5 pts] Suppose that A and B are two events satisfying A ∪ B = Ω and P (A) = P (B). Show that
P (A | B) = P (B | A) = 2 − P (A)−^1. (^1) A twice-differentiable real-valued function f (x) is called convex if f ′′(x) ≥ 0 for all x.
- [15 pts] We now return to Problem 5 in Homework # 4. Recall that this concerns an examination in which there are N multiple-choice questions with 6 possible answers. Suppose that we have some prior information: The probability that an examinee knows the correct answer to a given problem is equal to p, and 1 − p is the probability that the student will guess. The value of p is known as a prior probability since it is assumed to be available before seeing the completed exam results.
(a) The probability of answering the question correctly is unity if the student knows the answer. It is equal to 1/6 otherwise. Compute the probability that an examinee knew the answer to a question, given that he or she has correctly answered it. (b) A student knows the answer to K of the N problems, and marks the answer sheet accordingly. The examiner determines W , the number of incorrect answers on the examiner’s sheet. Assume that the value of p is identical in each problem, and that K has a binomial (N, p) distribution. Argue that W is also binomial (N, q), where q is the probability that the first answer is incorrect. (c) What is the conditional probability that K = k, given W = w, for given k and w? Hint: It is far easier to first compute P (W = w | K = k).
- [15 pts] Two motion detectors are used to detect the presence of a person in a room, as part of an energy saving temperature control system. The first sensor outputs a value X and the second sensor outputs a value Y. Both outputs have possible values { 0 , 1 , 2 }, with larger numbers tending to indicate that a person is present. Let H 0 denote the event (or hypothesis) that a person is absent and let H 1 denote the event that a person is present. The likelihood matrices for X and for Y , giving the probabilities for possible outputs given H 0 or H 1 , are as follows:
X = 0 X = 1 X = 2 Y = 0 Y = 1 Y = 2
H 0 0.8 0.1 0.1 0.7 0.2 0.
H 1 0.1 0.3 0.6 0.1 0.1 0.
For example, P [Y = 2 | H 1 ] = 0.8. Suppose that, given one of the hypotheses is true, the sensors provide conditionally independent readings, so that P [X = i, Y = j | Hk] = P [X = i | Hk]P [Y = j | Hk] for 0 ≤ i, j ≤ 2 and 0 ≤ k ≤ 1. Since both X and Y are observed, there are nine possible values for the observation (X, Y ).
(a) Find the likelihood matrix for the observation given the hypothesis. This matrix specifies P [X = i, Y = j | Hk] for each hypothesis Hk and for each possible observation value (i, j). (The matrix has two rows, the first row corresponding to H 0 and the second row corresponding to H 1. The matrix has nine columns of numbers, each corresponding to the nine possible values of the observation (X, Y ) : 00, 01 , 02 , 10 , 11 , 12 , 20 , 21 , 22. The sum of the entries in each row is one.) Also, indicate for each of the nine possible observation values, the maximum likelihood (ML) decision of which hypothesis is true. This is done by simply circling the larger number in each column. Break ties in favor of H 1. (b) For the ML decision rule, compute the false alarm (conditional) probability and miss (condi- tional) probability defined by
pfalse alarm = P [decide H 1 is true | H 0 ] pmiss = P [decide H 0 is true | H 1 ].
For example, pfalse alarm is the sum of all the entries in the first row of the likelihood matrix that were not circled in part (a). (c) Suppose it is decided on the basis of observations of occupancy patterns that π 0 = P [H 0 ] = 0. 7 and π 1 = P [H 1 ] = 0.3. These probabilities π 0 and π 1 are prior probabilities: they are assumed
