Stat613: Intermediate Theory of Statistics
Bonus Problem II (10/13/07)
Instructor: Jianhua Huang Student:
TA: Seokho Lee
This is Problem 2 of Homework 0. Since no perfect solution has been provided, I post the problem again.
Identifiability of Parametrization
A statistical model X∼P(x, θ) is called identifiable if and only if P(x, θ ) = P(x, θ′) implies that θ=θ′
or, equivalently, if and only if θ6=θ′implies that P(x, θ)6=P(x, θ′). Identifiability means that the model
parameters are totally determined by the observed data. Only when the model is identifiable, interpretation
of the model parameters is meaningful. If a model is not identified, it can be made identifiable by imposing
constraints on the model parameter θ.
The notation used above is generic. The Xcould represent a random variable, a vector, or even more
complicated object. Similarly, the θcould be a number, a vector, or a function, or more complicated object.
Discuss if the following models are identifiable. If not, suggest a re-parameterization to make the model
identifiable.
(a) Single index model. Suppose we have data (Xi, Yi) independent identically distributed as (X,Y ). The
(X, Y )∈Rd×Rsatisfies
Y=f(XTβ) + ǫ, ǫ ∼N(0, σ2) (1)
(b) Regression model with collinearity. Suppose the data (X,Y), with Xan×pmatrix and Yan-vector,
are from the following model:
Y=Xβ+ǫ, E(ǫ) = 0.(2)
The rank of the Xmatrix satisfies rank(X) = r < p.
(c) Factor analysis. Let Ybe a random vector in Rd. Consider the model
Y=Fα+ǫ,ǫ∼N(0, σ2I),(3)
where Fis a d×kunknown matrix and αis a kdimensional zero mean Normal random vector. The data
are i.i.d. copies of Y. Usually kis much smaller than dso this model is useful for dimension reduction.
1