Download Understanding Experimental Design: Crowding Study and Confounding Variables and more Schemes and Mind Maps Design in PDF only on Docsity!
Chapter 8
Experimental Design
RCS 6740
Supplemental
Refresher of the Experimental
Method
āŗExperimental Method: A method of
determining whether variables are related in
which the researcher manipulates the
INDEPENDENT variable and controls all
other variables (DEPENDENT) either by
randomization or by direct experimental
control (Cozby, 2004).
Refresher of the Experimental
Method
Random Sampling vs. Random Assignment
āŗRandom Sampling: Using a random table of
numbers, a researcher selects people from a
sample to become participants in a study
āŗRandom Assignment: Participants (already
chosen using random sampling) are
assigned to groups using a random table of
numbers
Refresher of the Experimental
Method Cont.
āŗ So, a true experimental study needs:
Ā Randomization and Ā Control
āŗ Independent Variable: The variable that is manipulated to observe its effect on the dependent variable.
āŗ Dependent Variable: The variable that is the subjectās response to, and dependent on, the level of the manipulated independent variable
Refresher of the Experimental
Method Cont.
āŗExample: To measure the effects of
crowding (Independent variable) on
cognitive performance (Dependent
variable), participants are randomly
assigned to one of two groups:
Ā Group 1 takes a test in a crowded room Ā Group 2 takes a test alone in a room Is this a good example of the experimental method?
Refresher of the Experimental
Method Cont.
āŗYes, if you accounted for and minimized
other differences between the groups!
āŗConfounding Variable: A variable that is not
controlled in an experiment and that can
effect the dependent variable.
āŗSo, from the study on crowding and
cognitive performance, what are possible
confounding variables?
Coding for Lecture
āŗTo simplify the designs, the following coding
system will be used throughout the
remainder of this lecture
Ā O = observation or measure
Ā X = experimental intervention or manipulation
Ā R = random assignment to groups
Basic Experiments
āŗ The most basic experimental design has two variables Ā Independent Variable Ā Dependent Variable āŗ The independent variable has two Levels Ā Experimental Group (Usually receives treatment) Ā Control Group (Usually does not receive treatment) Ā A study can also have two different amounts of an independent variable āŗ 10 mg of Prozac for one group and 20 mg of Prozac for another group āŗ Example: A Randomized and Controlled study looking at the effects of exercise (Independent) on body fat (Dependent) Ā Group 1 exercises 3 times a week for 6 weeks Ā Group 2 does not exercise at all for three weeks Researchers will compare the body fat of those who exercise to those who do not. The question is, when should body fat be measured?
Basic Experiments: Posttest Only
Posttest Only Design
āŗ A researcher using a Posttest only design must: Ā Obtain two equivalent groups* Ā Introduce the independent variable Ā Measure the effect of the independent variable on the dependent variable
- To eliminate selection differences among participants, a researcher needs to either randomly assign participants to conditions or have the same participants participate in both conditions
Basic Experiments: Posttest Only
(R) X O
(R) O
(R) X (Exercise/IV) O (Body Fat Measure/DV) (R) O (Body Fat Measure/DV)
āŗ Upon completion of all interventions, the researcher will measure the effects of the independent variable on the dependent variable to look for statistically significant differences
Basic Experiments: Posttest Only
āŗPosttest only design using our exercise
example:
Ā Participants are randomly assigned to one of two groups. Group 1 exercises 3 times a week for 6 weeks and Group 2 does not exercise for 6 weeks. After 6 weeks, both groups will undergo a test to gauge their body fat. It was found that people in the exercise group had 5% less body fat than the people who did not exercise. Results of this study are statistically significant.
Basic Experiments: Posttest Only
Advantages of the Posttest Only Design āŗ Less expensive and time consuming than Pretest- Posttest āŗ Does not sensitize participants to what you are studying āŗ Pretests are not usually given in the real world so this design provides the researcher with the most confidence for findings generalization āŗ In this design the possibility of the interaction of the pretest effects and the independent variable are not present, therefore that source as a potential threat to external validity is eliminated
Basic Experiments: Pretest-Posttest
Design
Advantages of the Pretest-Posttest Design āŗ Allows a researcher to account for mortality āŗ Establishes a baseline āŗ Enables a researcher to do before and after analyses of data āŗ Allows the researcher to establish whether or not groups were random/equal to begin with āŗ Allows researcher the opportunity to select participants for a study Ā Matching (discussed in detail later in lecture) Ā Smoking Example: I smoke regularly but answers only 1 a day on pretest
Basic Experiments: Pretest-Posttest
Design
Disadvantages of the Pretest-Posttest Design
āŗ More costly and time consuming than Posttest only design
āŗ May sensitize participants to the study
Ā A researcher can disguise the pretest by: āŗ Having a different experimenter give it in a different setting āŗ Embed the pretest with a few irrelevant measures
āŗ Does not allow for confidence in generalizing
results as pretests are not given in the real world
Combination of Pretest only and
Pretest-Posttest Designs
āŗ The Solomon Four Group Design combines
pretest-posttest and posttest only designs
āŗ 1/2 participants receive pretest and posttest
āŗ 1/2 participants receive only posttest
āŗ Randomly assigned
āŗ To examine and control for effects of pre-test Ā If there is no impact of the pretest, posttest scores for both groups (pre and pre/post) should be equivalent
Solomon Four Group Design: No
Pretest Effect (Posttest Scores)
0
5
10
15
20
25
30
35
40
Posttest Only Pretest-Posttest
Control Treatment
Solomon Four Group Design: Pretest
Effect (Posttest Scores)
0
5
10
15
20
25
30
35
40
Posttest Only Pretest-Posttest
Control Treatment
Assigning Participants to
Experimental Conditions
āŗIndependent Groups Design
Ā Participants are randomly assigned to various conditions so that each participates in only one group
āŗRepeated Measures Design
Ā Participants participate in all conditions and are eventually assigned to all levels of the Independent Variable
Repeated Measures Design
Participant 92 85 +
3
Participant 81 78 + 2
Participant 68 64 + 1
Score of Difference Jackson Measure after EM
Score of Jackson Measure after ET
Repeated Measures Design
Disadvantages of the Repeated Measures Design āŗ Different conditions must be presented in a particular sequence āŗ Order Effect: The order of treatments may effect the dependent variable Ā Example: In our speech study, lets say that everyone received Melodic Intonation first. Most of the participants improved on the Jackson Speech Assessment Inventory after the other two treatments (ELT & ELM) due to a technique learned from Melodic Intonation. Can you attribute this gain to the effects of ELT or ELM?
Repeated Measures Design
Types of Order Effects āŗ Practice Effect Ā Improvement in performance as a result of repeated practice āŗ Fatigue Effect Ā Drop in performance due to being tired, bored, or distracted āŗ Contrast Effect Ā Responses to the second condition of an experiment are altered because both conditions are contrasted to one another āŗCrime Study Example Ā Seeing a Mild Crime First and then Seeing a Severe Crime Second Ā Seeing a Severe Crime First and then seeing a Mild Crime Second Basically, seeing one first will influence how you perceive the other
Repeated Measures Design
āŗThere are two way to account for and
minimize order effects
Ā Employ Counter Balancing Techniques
Ā Ensure that adequate time elapses
between conditions
Counterbalancing
āŗCounterbalancing entails including all
possible orders of treatment/conditions in
an experiment
āŗIt helps researchers identify any order
effects
āŗExample: Lets use our speech study and
focus on ELM and ELTā¦
Counterbalancing
Participants 11 through 20 receive ELT second
Participants 11 through 20 receive ELM first
Participants 1 through 10 receive ELM second
Participants 1 through 10 receive ELT first
Randomized Blocks
āŗ Some Repeated Measures Design experiments with multiple treatment orders are repeated over and over again to look for effects
āŗ Example: Assigning participants a ālucky numberā and seeing what mood it puts them in.
āŗ Each repetition of the basic experiment (i.e. assigning of the first lucky number) is called a Block of Trials
āŗ To control for order effects, the assigning of the numbers should be done in a random manner
Time Interval Between Treatments
āŗ In addition to counterbalancing, researchers need
to carefully control how much time elapses between treatments/interventions
āŗ Examples:
Ā Giving a sufficient rest period after a month of therapy Ā Allowing enough time to let a drug wear off Ā Also, ensuring that too much time does not elapse as participants may become upset and drop out
When to Choose an Independent Groups Design or a Repeated Measures Design
āŗ Basically, time, money, and number of participants will affect your choice āŗ Also, it will depend on how your study generalizes to the āreal worldā āŗ Example: If you want to look at how characteristics of a defendant affect a juror, you would probably use an Independent Groups Design (jurors focus on only one defendant) āŗ If you wanted to see the effects of job applicants on employers, you would probably use the Repeated Measures Design (employers interview multiple applicants)
Matched Pairs Design
āŗThe Matched Pairs design is a way of
assigning participants to groups by using,
and focusing on, certain participant
characteristics (Matching Variable)
āŗThe goal is to achieve the same equivalency
of groups as the Repeated Measures Design
without having the same participants in
multiple conditions
Matched Pairs Design
Steps of the Matched Pairs Design āŗ Obtain a measure of the matching variable from each participant Ā Example: IQ scores āŗ Rank the participants from highest to lowest on the matching variable Ā Example: (136, 126, 118, 118, 103, 101) āŗ Match pairs that are approximately equal on the matching variable Ā Example: (136 and 126, 118 and 118, 103 and 101) āŗ Finally, randomly assign members of each pair to one of two treatment groups Ā Example: 136 goes to ELT and 126 goes to ELM, 118 goes to ELM and 118 goes to ELT, 103 goes to ELT and 101 goes to ELM
Analysis of Covariance
āŗAnalysis of Covariance is another way to
statistically control the correlation between
a subject variable (e.g. IQ scores) and the
dependent variable.
āŗBasically, information on the subject
variable is collected after the fact and
analyzed statistically to determine if it is
causing any effect on the dependent
variable
Sequential Method
āŗThe sequential method is a mixed method
beginning with the cross-sectional method
and then using the longitudinal method for
at least one more set of measurements on
the dependent variable
āŗExample: Health care use of 55 and 65 year
olds are gauged. Then, 5 years later, both
groupsā use of health care will be gauged
again
Questions or Comments
