Download Posterior Probability - Bayesian Inference - Exam and more Exams Mathematics in PDF only on Docsity!
LANCASTER UNIVERSITY
2007 EXAMINATIONS
Part II (Third or Fourth Year)
MATHEMATICS & STATISTICS 2 hours
Math 351: Bayesian Inference
You should answer ALL Section A questions and TWO Section B questions. In Section A there are questions worth a total of 50 marks, but the maximum mark that you can gain there is capped at 40.
SECTION A
A1 (a) Consider a discrete parameter θ which takes values θ 1 , θ 2 ,... , θm. State Bayes’ theorem for the posterior probability f (θi | x), i = 1,... , m, in terms of the likelihood f (x | θ) and the prior distribution f (θ) of θ. [2] (b) A parameter θ can take one of three possible values {θ 1 , θ 2 , θ 3 }. A random variable X can take two possible values, and its distribution depends on θ according to the likelihood table: θ 1 θ 2 θ 3 X = 1 1/3 1/ X = 2 1/ Calculate the missing values in this table. If your prior for θ is that each value is equally likely, calculate the posterior distribution for θ given two independent observations x 1 = 1, x 2 = 2. [10]
please turn over
SECTION A continued
A2. A density f (x | θ) belongs to the exponential family if f (x | θ) = h(x)g(θ) exp{t(x)c(θ)}. (a) What is the conjugate prior for a likelihood which belongs to the exponential family? [3] (b) Suppose that f (x | θ) is the Gamma(α, βθ) distribution, where α and β are known positive constants, and θ is an unknown parameter. Does it belong to the exponential family? If it does calculate and recognise the conjugate prior distribution for θ. [5] (c) Observations xi, i = 1,... , n are modelled as realisations of independent random vari- ables Xi with Gamma(αi, βiθ) densities, where αi and βi are known positive constants, and θ is an unknown parameter. Derive the likelihood f (x | θ), given the vector of observations x = (x 1 ,... , xn), and simplify it ignoring unnecessary constants of propor- tionality. Using a Gamma(p, q) prior obtain and recognise the posterior distribution of θ. [7] You may use the fact that the Gamma(p, q) density for a variable θ is f (θ) = q
p Γ(p) θ
p− (^1) exp(−qθ), θ ≥ 0 , p, q > 0.
A3. (a) Consider the quadratic loss function,^ L(θ, a) = (θ^ −^ a)^2 , for estimating a parameter by^ a when its true value is θ. Show that the best estimator with respect to this loss function is the posterior mean, E(θ | x). [4] (b) If the posterior density for θ is f (θ | x) ∝ θ, 0 ≤ θ ≤ 1 , calculate the best estimator of θ with respect to quadratic loss. [6]
A4. (a) Define the predictive distribution of a future observation^ y^ in terms of the probability density function f (y | θ) and the posterior distribution f (θ | x). [3] (b) A random sample x 1 ,... , xn is observed from a Poisson(θ) distribution. The prior dis- tribution of θ is Gamma(p, q), where p and q are integers. Calculate the posterior distribution of θ. [3] (c) Calculate and recognise the predictive distribution for a future observation, y, from the Poisson(θ) distribution. [7] You may use the fact that the Poisson(θ) distribution for a variable x is f (x) = θ
x (^) exp(−θ) x! , x^ = 0,^1 ,... , θ >^0. please turn over
SECTION B continued
B2. (a) Prove that, given a single observation xi from the Normal(θ, τ −^1 ) distribution (where τ is known), the likelihood of θ is given by
f (xi | θ) ∝ exp
− τ θ
2 2 +^ τ xiθ
. [4]
(b) Prove that
θ ∼ Normal(μ, c−^1 ) if and only if f (θ) ∝ exp
− cθ
2 2 +^ cμθ
where f (θ) is the pdf of θ. [5] (c) Let x 1 ,... , xn be a random sample from the Normal(θ, τ −^1 ) distribution (where τ is known). The prior distribution for θ is the following mixture density
f (θ) = p
c 1 2 π exp
− c 2 1 (θ − μ 1 )^2
c 2 2 π exp
− c 2 2 (θ − μ 2 )^2
Calculate and recognise the posterior distribution for θ. Is the above prior a conjugate prior for θ? [10] (d) Calculate and recognise the predictive distribution for a future observation, y, from the Normal(θ, τ −^1 ) distribution (where τ is known). [11]
please turn over
SECTION B continued
B3. (a) (i) Consider the weighted quadratic loss function
L(θ, a) = w(θ)(θ − a)^2 ,
where w(θ) is a nonnegative function. Show that the best estimator with respect to this loss function is E[w(θ)θ | x] E[w(θ) | x].^
[8]
(ii) Derive the best estimator of the parameter θ if θ | x ∼ Beta(p, q) and the loss function is L(θ − a)^2 = θ(θ − a)^2. [6] (b) (i) A parameter has posterior density f (θ | x). State the property which must be satisfied if Ca(x) is a 100(1 − a)% credible region for θ. What further property must Ca(x) satisfy if it is to be as small a region as possible? [4] (ii) What is the form of the 100(1 − a)% higher posterior density region for a unimodal and symmetric posterior distribution of a scalar parameter θ 1? What is the form of the 100(1 − a)% higher posterior density region for a bimodal posterior distribution with modes of equal heights of another scalar parameter θ 2? Justify your answer by sketching the plots of f (θ 1 | x) and f (θ 2 | x). [4] (iii) Suppose that the posterior distribution of θ is Beta(p, p), p > 1. State the two conditions that the 100(1 − a)% higher posterior density region for θ satisfies in this case. What is the form of this region? [5] (iv) If the posterior distribution of θ is Beta(2, 2), is it possible to compute the 100(1 − a)% higher posterior density region for θ analytically? Justify your answer. [3]
You may use the fact that the Beta(p, q) density for a variable θ is
f (θ) = (^) B(p, q^1 ) θp−^1 (1 − θ)q−^1 , 0 ≤ θ ≤ 1 , p, q > 0 ,
and that E(θ) = (^) p+pq and Var(θ) = (^) (p+q) 2 pq(p+q+1).
end of exam
