Biostat/Epi 536
October 21, 2008
Is each of the following important
for ASSOCIATION? for PREDICTION?
regression coefficient estimates Yes No
what variables are in the model Yes No*
the use of automated procedures No Yes
the area under the ROC curve No Yes
goodness of fit tests No Yes
* Once we’ve chosen the set of variables that we want to consider, we don’t care which of them are actually included in the final model.
However, the choice of this set of variables may depend on what data is available or what type of data we want to use in our prediction.
Model Selection for ASSOCIATION:
Confirmatory
[can be used to test hypotheses]
Exploratory
[can generate hypotheses, but cannot test them]
Main Association
Variable
- include whether or not significant
- include in form consistent with prior hypothesis
- include whether or not significant
- include in best-fitting form
Interactions with Main
Association Variable
- include only if specified in prior hypothesis
- include only in form consistent with prior hypothesis
- include only if specified in prior hypothesis
- explore functional forms and choose one that fits well
[but include main effects if interaction is included]
Adjustment Variables
(e.g. confounders,
precision variables)
- include only if specified in prior hypothesis
- include in as rich a form as possible to minimize
residual confounding
- statistical significance is irrelevant
- examine possible confounders to see if controlling for
them changes coefficient of interest
- examine different forms, and choose richer model when
there is a difference in the coefficient of interest**
Presentation - explanation of what was controlled, and how
- adjusted odds ratios, with CIs
- possibly unadjusted odds ratios, with CIs
- sometimes partially adjusted odds ratios
- explanation of what was controlled, and how
- adjusted odds ratios, with CIs
- possibly unadjusted odds ratios, with CIs
- sometimes partially adjusted odds ratios
- brief description of model selection and suggestions for
future studies of hypotheses that were generated
** F-tests should only be used to compare functional form (e.g. splines versus linear) if the splines are created with the main association variable. If they are created with an adjustment
variable, then the choice of form should be made based on the coefficient of interest only. If this coefficient is different with different forms of adjustment, choose the richer one.
