Download STAT 210A: Minimal Sufficiency, Completeness, and Ancillarity and more Study notes Data Analysis & Statistical Methods in PDF only on Docsity!
STAT 210A: Theoretical Statistics Fall 2006
Lecture 4 — September 7
Lecturer: Martin Wainwright Scribe: Robert Gibbons
Warning: These scribe notes have only been mildly proofread.
4.1 Minimal Sufficiency
A given problem can admit many sufficient statistics.
4.1.1 Example
Consider the Normal location model Xi ∼ N (θ, 1). The data itself, (X 1 ,... , Xn) is sufficient. Also, a re-ordering (X(1),... , X(n)) is sufficient. We will show that the mean X = (^1) n
∑n i=1 Xi is minimally sufficient.
4.1.2 Definition
A statistic T is “Minimally sufficient” if T is sufficient and for all other sufficient statistics U there exists some function f such that T = f (U ), a.s.Pθ, ∀θ ∈ Θ
We want simpler criteria to recognize minimal sufficiency.
Theorem 4.1. (a) Let P = {Pθ|θ 0 ,... , θk} be a regular family with finitely many members.
Then T (X) =
p(x;θ 1 ) p(x;θ 0 ) ,... ,^
p(x;θk ) p(x;θ 0 )
is minimal sufficient.
(b) Given nested families P ⊆ P˜ such that ( P˜a.e.) implies (Pa.e.), if T is minimal suf- ficient for P˜ and sufficient for P then T is minimal sufficient for P.
4.1.3 Example: Location families
Consider Xi ∼ F (x − θ), i = 1,... , n i.i.d, where F is known and θ is an unknown location parameter. (a) Suppose Xi ∼ N (θ, 1). We want to show that X is minimal sufficient. Set P˜ =
{Pθ 1 , Pθ 0 }. Part (a) of the theorem says that p p((xx;;θθ^10 )) is minimal sufficient. This implies that
X is minimal sufficient for this small family. (b) Suppose Xi ∼ U
[
θ − 12 , θ + (^12)
]
, a shifted uniform model. A similar argument will show that T =
X(1), X(n)
, the extreme order statistics, are minimal sufficient.
STAT 210A Lecture 4 — September 7 Fall 2006
4.2 Completeness and ancillarity
At the other extreme...
4.2.1 Definition
A statistic V is “ancillary” if its distribution does not depend on θ.
4.2.2 Definition
A statistic T is “complete” for P = {Pθ|θ ∈ Θ} if Eθ [f (T )] = 0, ∀θ ∈ Θ ⇒ f (T ) ≡ 0 , a.e.P
4.2.3 Example
Consider Xi ∼ U [0, θ] i.i.d. We saw last lecture that X(n) = max {X 1 ,... Xn} is sufficient, and an argument like the one in the earlier example will show that X(n) is minimal sufficient.
Now we claim that T = X(n) is complete, since Pθ(T ≤ t) =
( (^) t n
)n ⇒ p(t; θ) = nt n− 1 θn^ ,^ ∀t^ ∈ (0, θ) Suppose that Eθ [f (t)] = 0 = (^) θnn
∫ (^) θ 0 f^ (t)t
n− (^1) dt, ∀θ ∈ R. This implies f ≡ 0 , a.e.P.
Theorem 4.2. If T is complete and sufficient, then it is minimally sufficient.
Proof: Let V be a minimal sufficient statistic, and let T be complete and sufficient. With- out loss of generality, assume both T, V are one-dimensional random variables. Since V is minimal sufficient and T is sufficient, then there exists a function f such that V = f (T ). Define g(V ) = Eθ[T |V ]. g(V ) is independent of θ by sufficiency of V. Use Tower property to get Eθ[g(V )] = Eθ(T ). That is, Eθ[g(V ) − T ] = 0, a.e.P, and g(V ) = g(f (T )) and so g(V ) − T = h(T ) for some function h. Then by completeness of T , g(V ) ≡ T, a.e.P.
Unfortunately, minimal sufficiency does not imply completeness.
4.2.4 Example
Consider Xi ∼ U
[
θ − 12 , θ + (^12)
]
. We saw that T =
X(1), X(n)
is minimal sufficient. But T is not, in fact, complete. You can show that
X(1), X(n)
is ancillary (the distribution does not depend on θ), and Eθ
[
X(1) − X(n)
]
(n− 1 n+
, ∀θ ∈ Θ and therefore, T is not complete.
4.3 Link between completeness and ancillarity
Theorem 4.3. If T is complete and sufficient, and if V is ancillary, then T and V are independent under Pθ, ∀θ ∈ Θ.
