Download EXPERIMENTAL RESEARCH DESIGNS and more Study notes Experimental Psychology in PDF only on Docsity!
As a strict technical definition, an experiment is a study or research design in which we MANIPULATE variables.
This also implies that experimental designs are characterized by RANDOM ASSIGNMENT to groups, treatments, and/or conditions.
That is, the researcher has high levels of control over the WHO, WHAT, WHEN, WHERE and HOW of the study.
MANIPULATION and RANDOM ASSIGNMENT are the defining characteristics of experimental designs.
RANDOM SAMPLING AND RANDOM ASSIGNMENT
! Ideally, all research designs and studies should use random sampling.
! In addition, experimental research designs call for random assignment to groups.
! The use of random sampling and random assignment gives the strongest case for causal inferences and generalizability.
- Random Sampling —the process of choosing a "representative" sample from an entire population such that every member of the population has an equal and independent chance of being selected into the sample.
- Random Assignment (Randomization) —a control technique that equates groups of participants by ensuring every member (of the sample) an equal chance of being assigned to any group.
- Controls for both known and unknown effects and threats.
- Randomization is of concern in experimental research where there is some manipulation or treatment imposed.
- As a systematic procedure for avoiding bias in assignment to conditions or groups, if we can avoid said bias, then we can assert that any differences between groups (conditions) prior to the introduction of the IV are due solely to chance.
Topic
EXPERIMENTAL RESEARCH DESIGNS
- The principal concern is whether differences between groups AFTER the introduction of the IV are due solely to chance fluctuations or to the effect of the IV plus chance fluctuations.
EXAMPLES OF SOME EXPERIMENTAL RESEARCH DESIGNS
- Control experiment with control group and experimental group
PRETEST TREATMENT POSTTEST
G ROUP I YES YES YES
G ROUP II YES NO YES
- Control experiment with no control group
PRETEST TREATMENT POSTTEST
G ROUP I YES^ A 1 YES
G ROUP II YES A 2 YES
- Control experiment with control condition within-subjects
ALL PARTICIPANTS PRETEST TREATMENT POSTTEST
CONDITION I YES^ YES^ YES
CONDITION II YES^ NO^ YES
EXAMPLES OF SOME RESEARCH DESIGNS TO AVOID
- One-group posttest only design
TREATMENT POSTTEST
G ROUP I YES YES
- One-group pretest-posttest design
PRETEST TREATMENT POSTTEST
G ROUP I YES YES YES
- Posttest only design with nonequivalent control groups
ALLOCATION TO
G ROUPS
TREATMENT POSTTEST
G ROUP I NONEQUIVALENT
NATURALLY OCCURRING GROUPS
YES [A 1 ] YES
G ROUP II NO [A 2 ] YES
- Nonequivalent control group of participants that is not randomly selected from the same population as the experimental group
WITHIN - SUBJECTS , BETWEEN - SUBJECTS , AND M IXED FACTORIAL DESIGNS
- Within-subjects design —a research design in which each participant experiences every condition of the experiment or study.
A. Advantages
- do not need as many participants
- equivalence is certain
B. Disadvantages
- effects of repeated testing
- dependability of treatment effects
- irreversibility of treatment effects
- Between-subjects design —a research design in which each participant experiences only one of the conditions in the experiment or study.
A. Advantages
- effects of testing are minimized
B. Disadvantages
- equivalency is less assured
- greater number of participants needed
- Mixed factorial design —a research design that combines/uses between- and within-subject variables in the same design.
SUMMARY OF K EY CONCEPTS —EXPERIMENTAL DESIGNS
- Control
- Any means used to rule out possible threats to a piece of research
- Techniques used to eliminate or hold constant the effects of extraneous variables
- Control Group
- Participants in a control condition
- Participants not exposed to the experimental manipulation
- Experimental Group
- Participants in an experimental condition
CAUSAL INFERENCES
! An advantage that experimental designs have over other research designs is that they permit us to make causal inferences.
! Causation implies the ability to make statements about the absence or presence of cause-effect relationships.
! A causal inference is a much stronger statement than a simply and association between variables.
! While there are several methods —some of which are discussed in the text—to experimentally identify causality, in conjunction with manipulation (and use of random assignment), there are three additional conditions that must be met to infer cause.
(a) Contiguity —between the presumed cause and effect. (b) Temporal precedence —the cause has to precede the effect in time. (c) Constant conjunction —the cause has to be present whenever the effect is obtained.
! The ability to make causal inferences is dependent on how well or the extent to which alternative causes or explanations are ruled out.
! Cause—is a necessary and sufficient condition.
! An event that only causes an effect sometimes is NOT a cause.
! The assessment of causation technically demands the use of manipulation.
! Caveats to determining causality:
(a) Concerning cause-effect relationships, it cannot be said that they are true. We can only say that they have NOT been falsified.
(b) The use of correlational methods to infer casual relationships should be avoided.
(c) Although one might find that r =/ 0 or that the regression equation is significant, this does NOT prove or indicate a causal relationship (i.e., that X caused Y to change]. At best we can only say that there is a relationship between X and Y.
Thus, from the perspective of a methodological purist, one cannot make causal inferences on the basis of correlational data or designs.
Summary of Causation
With the scientific method, a number of conditions must be met to make strong causal claims (weaker conclusions of causality can be made when less than all these conditions are present).
O The cause X must precede the consequence Y in time. Thus, X is manipulated (or measured) and then Y is measured. [temporal precedence and constant conjunction]
O Statistical covariation between X and Y must be present. [constant conjunction and contiguity] This covariation must be statistically significant , and thus unlikely to be due to random chance fluctuation alone. Stated differently, random chance should be ruled out as a plausible alternative cause of the observed covariation between X and Y.
O Alternative causes of Y must be controlled, either via random assignment to groups (perhaps with a preceding matching procedure for the most plausible alternative cause) or via statistical controls.
- Factorial Design —a design in which 2 or more variables or factors are employed in such a way that all of the possible combinations of selected values of each variable are used.
- Examples of factorial designs
2×2 factorial design P IVA = 2 levels P IVB = 2 levels
2×2×2 factorial design P IVA = 2 levels P IVB = 2 levels P IVC = 2 levels
2×3 factorial design P IVA = 2 levels P IVB = 3 levels
- An issue that arises when we use factorial designs is that of main effects and interactions.
- Main Effect —the effect of one IV averaged over all levels of the other IV.
- That is, the effect of IVA independent of IVB or holding IVB constant; can also be described as the mean of A 1 and A 2 across levels of B.
- A main effect is really no different from a t -test for differences between means (assuming there are only 2 means).
- Interaction —when the effect of one IV depends on the level of the other IV.
- Two or more variables are said to interact when they act on each other.
- Thus, an interaction of IVs is their joint effect on the DV, which cannot be predicted simply by knowing the main effect of each IV separately.
- Main effects are qualified by interactions (interpreted within the context of interactions). Specifically, we do not interpret main effects when interactions are significant.
- The occurrence of an interaction is analyzed by comparing differences among cell means rather than among main effect means.
- Graphical plots are commonly used to illustrate the results of the ANOVA test. We plot graphs to aid us in interpreting the ANOVA results after we have run the test.
P It has been alleged that SUV A has a very high propensity to rollover and thus is patently unsafe. The manufacturer of SUV A argues that the problem is not with its vehicle but instead the tires that are mounted on the vehicle, specifically, TIRE A.
R How would one determine whether the problem is with the tires or the vehicle?
R An experimental design investigating this issue would be a basic 2×2 factorial design.
- IVA = 2 types of SUVs (SUV A and SUV B)
- IVB = 2 types of tires (Tire A and Tire B)
SUV
SUV A SUV B
TIRES
Tire A
Tire B
- How many conditions?
- Within-subjects, between-subjects, or mixed factorial design?
- Depending on whether the problem is with SUV A or Tire A, what will the data look like?
- Main effects or an interaction?
- Plot the data.
- What would an interaction look like?
- How would you interpret a SUV A/Tire A interaction?
0
10
20
30
40
50
60
A1 A2 A
B B
A and B main effect significant (d)
B
B
0
10
20
30
40
50
A1 A2 A
B
B
B
B
A main effect and interaction significant (e)
0
10
20
30
40
50
A1 A2 A
B
B
B
B
B main effect and interaction significant (f)
Hypothetical Data Illustrating Different Kinds of Main and Interaction Effects (continued)
- A and B are significant; the interaction is not significant.
A1 A2 A3 Mean
B1 10 20 30 20
B2 40 50 60 50
Mean 25 35 45
- A and the interaction are significant; B is not significant.
A1 A2 A3 Mean
B1 20 30 40 30
B2 30 30 30 30
Mean 25 30 35
- B and the interaction are significant; A is not significant.
A1 A2 A3 Mean
B1 10 20 30 20
B2 50 40 30 40
Mean 30 30 30
20
30
40
50
60
70
A1 A2 A
B
B
B
B
A and B main effect and interaction significant (g)
Hypothetical Data Illustrating Different Kinds of Main and Interaction Effects (continued)
- A, B, and the interaction are significant.
A1 A2 A3 Mean
B1 30 50 70 50
B2 20 30 40 30
Mean 25 40 55
- Field Experiment
A. Advantages :
- Very realistic.
- Results are highly generalizable.
- Suggestions of causal inference are possible.
- Broader research issues dealing with complex behavior in real-life contexts can be addressed.
B. Disadvantages :
- Precision and exactness of control is relatively weaker.
- Individuals or groups may refuse to participate.
- Often cannot gain access to "natural" (business, organization, home or other) environment.
