Download Generalized Linear Mixed Models: Extending GLMs with Random Effects and more Study notes Statistics in PDF only on Docsity!
Generalized Linear Mixed Models
- Extend the generalized linear model to include both fixed effects and random effects.
- This is still a fairly new class of models.
An example of this would arise in a clinical study. Sup- pose that we are interested in testing a drug in humans. The response that we are interested in is the incidence of nausea in the subjects. We give the different individu- als each of several drugs (including placebo) and we ask them if they felt nauseous during the morning following the drug treatment.
Here, the responses are either a 0 (no nausea) or a 1 (nausea). Thus, we would like to have a logistic re- gression model to predict the probability of nausea for a particular drug. However, some people are much more susceptible to nausea than others, and thus responses on an individual may be correlated.
To develop the generalized linear mixed model, we should consider the following. Recall that for the linear mixed model, we specified f (y|α) and f (α), and thus also f (y).
For linear mixed models, we assumed that all three of these distributions were linear and gaussian. Suppose now that we relax the normality assumption for f (y|α). Rather, let’s assume
yi|α ∼ indep.f (yi|α),
Generalized Linear Mixed Models, Cont.
and f (y|α) is an exponential family distribution,
f (y|α) = exp [{yiθi − b(θi)}/a(φ) − c(yi, φ)].
The conditional mean of yi is related to θi according to the model
μi =
∂b(θi) ∂θi
It is a transformation of this mean that we desire to model as a linear model in both the fixed and random effects. Let
E(yi|α) = μi and g(μi) = X′ iβ + Z′ iα,
where g() is a known link function. Notice the similarity in notation to that for GLM. Here, we have replaced b with α and let μi be the conditional mean.
Finally, to complete the specification of this model, let
α ∼ f (α),
be some distribution.
Consequences of having random effects:
Recall that the mean of y has the form
E(yi) = Eα{E(yi|α)} = Eα(μi) = E{g−^1 (X′ iβ + Z′ iα)}.
In general, the form cannot be simplified because the form g−^1 () is nonlinear.
Consequences of Random Effects, Cont.
Covariance of y:
As in the case of a linear mixed model, the presence of random effects introduces a correlation among ob- servations which share any random effect in common. Here,
cov(yi, yj) = cov{E(yi|α), E(yj|α)} + E{cov(yi, yj|α)} = cov(μi, μj) + E(0) = cov{g−^1 (X′ iβ + Z′ iα), g−^1 (X′ jβ + Z′ jα)}.
Here, we notice the importance of conditioning to in- duce the covariability.
Example:
Consider the problem of modelling data in correlated “clusters”, which are thought to come from a Poisson distribution. For example, Diggle, et al (1994) consider the number of epileptic seizures in patients on a drug or placebo with repeated measurements on the same patients. Let
yij be the jth count taken on the ith patient.
Then, let the model be yij|α ∼ indep. Poi(μij). Here,
log(μij) = X′ ijβ + αi, where αi ∼ iid N(0, σ^2 α).
Note that in this example, we are using a log-link with a random patient effect.
Estimation for GLMMs
Likelihood:
It is easy in principal to write down the likelihood:
L =
f (y|α)f (α)dα,
a q-dimensional integral, since α has dimension q.
However, in nearly all cases, this cannot be evaluated in a closed form. In some cases, for example, low dimen- sional α, standard numerical integration approaches can be considered. Then, we must consider ∂`/∂β, which can be shown to have the form
∂` ∂β
= X′E(W ∗|y) − X′E(W ∗μ|y),
where
W ∗^ = diag[{a(φ)V (μi)g(μi)}−^1 ].
The likelihood equation for β is then
X′E(W ∗|y) = X′E(W ∗μ|y).
In some cases, for example, the Poisson, W ∗^ = I, so the likelihood equation is
X′y = X′E(μ|y).
Even this equation must be solved numerically in gen- eral.
For random effects, we could consider the corresponding forms for ∂`/∂φ.
Generalized Estimating Equations, Cont.
Then, GEE (according to Liang and Zeger, 1986) esti- mates for β have the form
S(β) =
∑^ K
i=
∂μ′ i ∂β
V −^1 {Y (^) i − μi(β)} = 0.
Now, since g(μij) = X′ ijβ,
∂μ′ i ∂β
Xi 11 g′(μi 1 ) · · ·^
Xini 1 g′(μini ) ...... ... Xi 1 p g′(μi 1 ) · · ·^
Xinip g′(μini )
Let Ri(a) be an ni×ni “working” correlation matrix spec-
ified up to some parameters a. Then, Vi = φA 1 / 2 i R(a)A
1 / 2 i , where Ai is an ni × ni diagonal matrix with V (μij) on the jth diagonal. If R(a) is the true correlation matrix of Y (^) i, then Vi is the true covariance matrix.
Usually, the working correlation matrix must be esti- mated iteratively. We can do this according to the fit- ting algorithm.
- Compute the initial estimate of β (often using GLM under the independence assumption).
- Compute the working correlation matrix R based upon the studentized residuals.
Generalized Estimating Equations, Cont.
- Compute the estimated covariance, ˆVi.
- Update β according to
βr+1 = βr +
( K
i=
∂μ′ i ∂β
V (^) i−^1
∂μi ∂β
×
{ K
i=
∂μ′ i ∂β
V (^) i− 1 (Y (^) i − μi)
- Iterate until the algorithm converges.
Note that the is in general not a likelihood estimator. Thus, we cannot make any inferences based upon the likelihood, as they would not be appropriate.
Other Approaches:
Some other approaches that we could consider include penalized quasi-likelihood, or conditional likelihood. Ad- ditionally, we could consider a Bayesian method.
We would begin by putting priors on our parameters
f (α, β|y) ∝ f (y|α, β)f (α)f (β).
Then, numerical techniques, such as Markov Chain Monte Carlo (MCMC) can be used to find the desired distribu- tion. Under certain conditions, this can be the “best” method.
