Download Random Sample - Bayesian Inference - Exam and more Exams Mathematics in PDF only on Docsity!
LANCASTER UNIVERSITY
2009 EXAMINATIONS
PART II (Final Year)
MATHEMATICS & STATISTICS
Math 351 : Bayesian Inference 2 hours
You should answer ALL questions from Section A, and TWO questions from Section B. In Section A there are questions worth a total of 50 marks, but the maximum mark that you can gain there is capped at 40. There is a formula sheet at the end of this examination paper.
SECTION A
A1. A random sample of 10 items was taken from the output of a machine and one of them was found to be defective. An engineer expressed the view that only three defective rates were possible for this type of machine and he supplied the following table of prior probabilities:
Defective rate Prior belief θ f (θ) 0.01 0. 0.02 0. 0.03 0. In order to obtain the likelihood, we use the binomial distribution with 10 trials to find the probability of one fault
P (X = 1 | θ = 0.01) = 0. 09 P (X = 1 | θ = 0.02) = 0. 17 P (X = 1 |θ = 0.03) = 0. 22 (a) Find the maximum likelihood estimator of θ. [1] (b) Find the marginal likelihood, P (X). [2] (c) Hence deduce the posterior distribution and find the posterior mean. [4] please turn over
SECTION A continued
A2. A density f (x|θ) belongs to the exponential family if
f (x|θ) = h(x)g(θ) exp{t(x)c(θ)}.
(a) State the conjugate prior for a likelihood which belongs to this exponential family. [2] (b) Explain why the following density belongs to the exponential family
f (x|θ) = θ 3 θx−(θ+1), x > 3 , θ > 0. [3]
(c) Calculate and identify the conjugate prior. [4] (d) Find Jeffreys’ prior for the above likelihood. To what is Jeffreys’ prior invariant? [5]
A3. (a) Urn A contains three balls: one black, and two white; urn B contains three balls: two black, and one white. One of the urns is selected at random and one ball is drawn. The ball is black. Find the probability that the selected urn is urn A given that each is a priori equally probable. [5] (b) Consider the posterior distribution
f (λ|x) = ke−^2 λ, for λ > 0 ,
for some constant k. (i) Find the normalisng constant, k. Calculate the posterior median of λ. [4] (ii) Explain the difference between a 95% classical confidence interval and a Bayesian 95% credible interval. [2] (iii) State an extra condition for the Bayesian credible interval C also to be a 95% highest posterior density (HPD) interval. Find a Bayesian 95% HPD interval in the case of the density given in the above expression. [5]
please turn over
SECTION B
B1. Independent observations, x = (x 1 , x 2 ,... , xn), are observed from a Poisson (λ) distribution. The rate parameter λ is allocated an improper prior distribution p(λ) ∝ λ^1. (a) Show that the posterior distribution is of the form: λ | x ∼ Gamma (α, β). Find the parameters α and β in terms of the data. [3] (b) Show that the posterior mean of λ is equal to the maximum likelihood estimator (MLE), ˆλ. [3] (c) Find the posterior variance of λ. [1] (d) The predictive distribution of a future observation, y, can be found by evaluating the following integral:
p(y | x) =
λ=
p(y | λ) p(λ | x) dλ.
In terms of α and β show that the predictive distribution of y is given by
f (y|x) =
(y + a − 1 α − 1
1 + β
)y ( 1 − (^) 1 +^1 β
)α , y = 0, 1 ,... ,
for integer values α. (Use the fact that Γ(n + 1) = n! if n is an integer.) [8] (e) Identify this distribution and state its parameters. [3] (f) The estimative distribution is the distribution of future values based on the maximum likelihood estimator, λˆ. Find the variance of this distribution, var (y | ˆλ). [3] (g) Find the variance of the predictive distribution, var (y | x). Hint: You can use the fact that var (y | x) = Eλ var (y | (λ | x))+ var (^) λE (y | (λ | x)). [5] (h) Compare your results from (c) and (d) and comment on the relative merits of the estimative and the predictive distributions. [6]
please turn over
SECTION B continued
B2. Observations x = (x 1 , x 2 ,... , xn) are modelled as independent exponential with Xi ∼ Exp(aiθ) where ai are known positive constants and θ is an unknown parameter. (a) Derive the likelihood L(θ|x), given the vector of observations, x, and simplify it, ignoring unnecessary constants of proportionality. [6] (b) Explain what is meant by a family of prior distributions f (θ) for θ which is conjugate for this model. [3] (c) Show that the Gamma (p, q) density forms a family of conjugate priors for this model. Evaluate an expression for the posterior density. Identify this distribu- tion and state its parameters. [6] (d) Derive Jeffreys’ prior distribution of θ for this model, showing that it can be expressed as an improper Gamma (0, 0) density. [9] (e) Show that the Jeffreys’ prior is equivalent to a uniform prior for φ = log(θ) over −∞ < φ < ∞. [6]
please turn over
Formula Sheet
You may use the following:
∗ A univariate Normal with mean μ ∈ IR and variance (^1) τ > 0 is denoted by Normal
μ, (^) τ^1
and the corresponding density function is:
p (y | μ, τ ) = τ^
(^12) √ 2 π exp
− τ 2 (y − μ)^2
for y ∈ R.
∗ A Gamma distribution with shape parameter α > 0 and rate parameter β > 0 is denoted by Gamma (α, β), and the corresponding density is:
p(y | α, β) = β
α Γ(α) y
α− (^1) exp(−βy) for y > 0.
The Gamma distribution, Gamma (α, β), has mean α β and variance (^) βα 2.
∗ An Exponential distribution with rate parameter λ > 0 is denoted by Exp (λ) and the corresponding density is:
p(y|λ) = λ exp(−λy) for y > 0.
∗ A Beta distribution with parameters α > 0 and β > 0 is denoted by Beta (α, β), and the corresponding density is:
p(y | α, β) = (^) B(α, β^1 ) yα−^1 (1 − y)β−^1 for 0 < y < 1.
where B(α, β) = Γ( Γ(αα)Γ(+ββ)) where Γ(n + 1) = n! for integer n. The Beta distribution, Beta (α, β), has mean (^) αα+β.
∗ A Binomial distribution with parameter 0 ≤ p ≤ 1 is denoted by Binomial (n, p), and the corresponding probability mass function is:
p(y | n, p) =
n y
py(1 − p)n−y^ for y = 0, 1 ,... , n.
∗ A Poisson distribution with parameter λ > 0 is denoted by Poisson (λ), and the corresponding probability mass function is:
p(y | λ) = e
−λ y! λ
y (^) for y = 0, 1 ,....
The mean and variance of a Poisson distribution is λ.
∗ A Negative binomial distribution with parameters 0 ≤ θ ≤ 1 and k ∈ { 1 ,.. .} is denoted by Negative-Binomial (θ, k), and the corresponding probability mass function is:
p(y|θ, k) =
y + k − 1 k − 1
θk(1 − θ)y, y = 0, 1 , 2 ,...
