Download Likelihood - Bayesian Inference - Exam and more Exams Mathematics in PDF only on Docsity!
LANCASTER UNIVERSITY
2010 EXAMINATIONS
PART II (Third year)
MATHEMATICS & STATISTICS 2 hours
Math 351: Bayesian Inference
You should answer ALL Section A questions and TWO Section B questions. In Section A there are questions worth a total of 50 marks, but the maximum mark that you can gain there is capped at 40. There is a formula sheet at the end of this exam paper.
SECTION A
A1. (a) State Bayes’ theorem for the conditional probability^ p(θ|x) of a discrete parameter^ θ^ in terms of the likelihood p(x|θ) and prior p(θ). [2] (b) A bag contains 4 balls. Each ball is either black or white. Let θ denote the number of white balls that the bag contains. My prior belief is that the probability of a ball being white is always 0.5 and independent from ball to ball. I pick out two balls, without replacement, and both are white. (i) Write down the likelihood function. [4] (ii) Calculate the prior probabilities for all values of θ. [2] (iii) Calculate the posterior probabilities for all values of θ. [4]
A2. (a) Consider a likelihood model where the distribution of an observation,^ x, depends on the parameter, θ, according to the probability density function f (x|θ) = e− xθ!^ θ x x = 0, 1 ,... with a prior distribution f (θ) = e−θ, θ ≥ 0. (i) For a random sample of size m, x 1 ,... , xm, calculate the posterior distribution of θ up to a normalising constant. [4] (ii) What parametric family does this posterior distribution belong to, and what are the posterior parameter values? [2] (b) An unknown quantity Y has a Negative-Binomial (θ, k), where k is assumed known. (i) Identify a class of conjugate prior densities for θ. [2] (ii) Let Y 1 ,... , Yn ∼ Negative-Binomial (θ, k). Find the posterior distribution of θ given Y 1 ,... , Yn, using a prior from your conjugate class. [4] please turn over
SECTION A continued
A3. (^) (a) An observation x is made from the pdf f (x|θ). Define Jeffreys’ prior distribution fθ(θ) for the parameter θ in this context. State the invariance property of this prior distribution, with respect to a parameter transformation φ = φ(θ). [4] (b) Let f (x|θ) ∝ x^4 θ^3 exp(−xθ) for x ≥ 0. Derive Jeffreys’ prior fθ(θ) for this model. [5] (c) Hence derive the Jeffreys’ prior fφ(φ) for this model when θ = exp(φ). [4]
A4. (^) (a) An unknown parameter θ has a known density f (θ|x) and it is required to construct an estimate a of θ. (i) Explain how a loss function L(θ, a) can be used for this purpose. [2] (ii) State the form of the two commonly used loss functions, the squared loss function, and the absolute error loss function. [2] (iii) State also the value taken by a in each of these cases when the posterior is
θ|x ∼ Beta (3, 1).
[5] (b) Consider the case when f (θ|x) = 1 for 0 ≤ θ ≤ 1 and the loss is defined by
L(θ, a) =
2(θ − a), θ ≥ a 1 , θ < a. Find the expected loss and obtain Bayes rule for estimating θ. [4]
please turn over
SECTION B continued
B2. Suppose X|μ ∼ Normal
μ, σ^2
and Y |μ, δ ∼ N (μ + δ, σ^2 ), where σ^2 is known and X and Y are conditionally independent given μ and δ. (a) Find the joint distribution of X and Y given μ and δ up to a normalising constant. [4] (b) Consider the improper noninformative joint prior distribution
f (μ, δ) ∝ 1.
Find, up to a constant of proportionality, the joint posterior distribution of μ and δ given x and y. Are μ|x, y and δ|x, y independent? If not why not? [6] (c) Show that the marginal posterior distribution f (μ|x, y) has a normal distribution and that μ|x, y ∼ Normal
x, σ^2
[5]
(d) Find the marginal posterior distribution f (δ|x, y). [6] (e) Consider a future observation Z, where Z|μ ∼ Normal
μ, τ 2
and τ is assumed known. Define the predictive distribution of Z and find it in this case. Why is the predictive distribution to be preferred to the estimative distribution? (Note: The estimative distribution is the distribution of a future observation given that μ is equal to its maximum likelihood estimate.) [9]
please turn over
SECTION B continued
B3. (^) (a) (i) Consider, given θ, a sequence of independent Bernoulli trials with parameter θ. We wish to make inferences about θ. We count the total number X of trials up to and including the rth success so that X | θ ∼ Negative-Binomial (θ, r). Obtain Jeffreys’ prior distribution for θ. You may find it useful to note that for the negative binomial distribution E(X|θ) = r(1 θ− θ). Is this prior proper or improper? [8] (ii) You observe X = r. Calculate the posterior distribution for θ using the Jeffreys’ prior calculated in (i). [6]
(b) Consider the following loss function
L(θ, d) = d θ − log
( (^) d θ
(i) Find the Bayes rule of an immediate decision. [8] (ii) Let X 1 ,... , Xn be exchangeable so that the Xi are conditionally independent given parameter θ. Suppose that Xi|θ ∼ Poisson (θ) and θ ∼ Gamma (α, β) with α > 1. Under the loss given above, find the Bayes rule after observing x = (x 1 ,... , xn). [8]
end of exam
∗ A Negative binomial distribution with parameters 0 ≤ θ ≤ 1 and k ∈ { 1 ,.. .} is denoted by Negative-Binomial (θ, k), and the corresponding probability mass function is:
p(y|θ, k) =
y + k − 1 k − 1
θk(1 − θ)y, y = 0, 1 , 2 ,...
