Internal and External Validity - Experimental Research Methods - Lecture Slides, Slides of Research Methodology

Download Internal and External Validity - Experimental Research Methods - Lecture Slides and more Slides Research Methodology in PDF only on Docsity!

Internal and External Validity

Internal Validity

• Is your experiment being performed correctly?
  • Are confounds occurring?
  • Are things occurring at one level of the IV but not at the others?
• If internal validity is low, then BAD variability will be high.
• Issues of internal validity can be addressed in the laboratory. - The concept of internal validity revolves around the question of whether your IV actually caused any change that you observe in your DV. - If you use adequate control techniques, your experiment should be free from confounding and you can, indeed, conclude that your IV caused the change in your DV.

Threats to Internal Validity Participant Selection

• Participant Selection: Different types of participants placed at the different levels of the IV. - Most extreme case, through random chance, only males in control group and only females in experimental group. - Can happen if participants choose conditions. - If you test different groups at different times, then people will be different.
• Solution: Random assignment, matching

Threats to Internal Validity Participant Attrition

• Participant Attrition: Participants drop from the experiment in some nonrandom way. - Pretend we have the following experiment - IV=Level of shock, 0 volts, 50 volts, 100 volts - DV=Reported level of arousal after shocking - After coming to the experiment some told they have been assigned to a high shock condition. - Some people will drop out, but who? Will the people that drop out be random? - a.k.a. Mortality
• Solution: Get participants to withdraw prior to random assignment. docsity.com

Threats to Internal Validity Maturation

• Maturation: One group changes differently from another. - One condition exciting, one boring. - One group gets tired due to tedious task.
• Solution: Hold experiences constant.

Control group takes 5 minutes

Experimental group takes 2 hours

Threats to Internal Validity Effects of Testing

• Effects of Testing: Participants in one group might become aware of the experiments intention. - Pretest alerts participants about experiment. - Merely taking a test once necessitates improvement if retaken.
• Solution: Use behavioral or nonreactive measures.

Threats to Internal Validity Experimenter Effects

• Experimenter Effects: Experimenter subtly influences the participants. - Maze bright and maze dull rats. - Fish and chemicals
• Solution: Keep experimenter blind to hypotheses or condition assignment; automate procedures.

Threats to Internal Validity Demand Characteristics

• Demand Characteristics: Participants that know they are being tested react to this. - a.k.a. Hawthorne effect
• Solution: Hold experience constant for all groups; disguise IV or measurement.

Threats to Internal Validity Diffusion of Treatment

• Diffusion of Treatment: Occurs if the control group learns about the manipulation. - Can occur any time participants talk about an ongoing experiment.
• Solution: Do not debrief any participants until ALL participants have finished.

Threats to Internal Validity Compensatory Rivalry

• Compensatory Rivalry: If control group learns about manipulation, they may become “jealous” and work extra hard to show up the experimenter. - Control group in an educational experiment forms their own tutoring sessions.

Protecting Internal Validity

• How Important is Internal Validity?
  • It is the most important property of any experiment. - If you do not concern yourself with the internal validity of your experiment, you are wasting your time. - With low internal validity comes low power!

External Validity

• Does the IV represent the concept we intend?
  • A measure is externally valid if it truly measures the hypothetical construct intended.
  • An experiment is externally valid if it is similar to phenomenon in the real world.
• Having high external validity (as field experiments might) often means having a lack of control of confounds.

External Validity: Generalizing Your Experiment to the Outside

• There are three customary types of generalization in which we are interested: - Population Generalization: Applying the results from an experiment to a group of participants that is different and more encompassing than those used in the original experiment. - Environmental Generalization: Applying the results from an experiment to a situation or environment that differs from that of the original experiment. - Temporal Generalization: Applying the results from an experiment to a time that is different from that when the original experiment was conducted. docsity.com

Threats to External Validity

• Here is a summary of factors relating to external validity from Campbell and Stanley (1966): - Interaction of Testing and Treatment: A threat to external validity that occurs when a pretest sensitizes participants to the treatment yet to come. - Occurs for the pretest-posttest control group design - Because of a pretest, your participants’ reaction to the treatment will be different.