Download Understanding Experimental Design: Focus on Randomized Controlled Experiments and more Study notes Statistics in PDF only on Docsity!
(C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture
Lecture 3
Designing Studies
(Focus on Experiments)
Definitions Randomization Control Blind Experiment Pitfalls Specific Experimental Designs (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 2
Looking Back: Review
4 Stages of Statistics Data Production Displaying and Summarizing Probability Statistical Inference (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 3
Looking Back: Review
2 Types of Study Design Observational study: record variables’ values as they naturally occur Drawback: confounding variables due to self- assignment to explanatory values Example: Men who drink beer are more prone to lung cancer than those who drink red wine (what is the confounding variable here?) Experiment: researchers control values of explanatory variable If well-designed, provides more convincing evidence of causation. (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 4
Definitions
Factor : an explanatory variable in an experiment. Treatment: value of explanatory variable imposed by researchers in an experiment. A control group (individuals receiving no treatment or base-line treatment) may be included for comparison. If individuals are human, we call them subjects.
(C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 5 Example: Randomized controlled experiment Background : To test if sugar causes hyperactivity, researchers randomly assign some children to low and others to high levels of sugar consumption… Question: What is the advantage of random assignments? X X X X X X X X X X Sample L Population H XX H L H L (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 7 Example: Randomized controlled experiment Background : To test if sugar causes hyperactivity, researchers randomly assign some children to low and others to high levels of sugar consumption… Response: (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 8
Experiment vs. Observational Study
In an experiment, researchers decide who has low sugar intake (L) and who has high (H). X X X X X X X X X X Sample L Population H XX H L H L Sugar intake has not yet been determined. Researchers assign sugar intake L or H. (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 9
Experiment vs. Observational Study
In observational study, individuals have already chosen low (L) or high (H) sugar intake. Researchers make no changes to sugar intake. L L H L L H L H L H Population Sample HL
(C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 14
Double-blind experiments
Two pitfalls may prevent us from drawing a conclusion of causation when results of an experiment show a relationship between the so-called explanatory and response variables. If subjects are aware of treatment assignment If researchers are aware of treatment assignment (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 15
Definitions
The placebo effect is when subjects respond to the idea of treatment, not the treatment itself. A placebo is a “dummy” treatment. A blind subject is unaware of which treatment he/she is receiving. The experimenter effect is biased assessment of (or attempt to influence) response due to knowledge of treatment assignment. A blind experimenter is unaware of which treatment a subject has received. (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 16 Example: Subjects not blind Background : Suppose after children are randomly assigned to consume either low or high amounts of sugar, researchers find proportion hyperactive is greater for those who consumed higher amounts. Question: Can we conclude sugar causes hyperactivity? (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 18 Example: Subjects not blind Background : Suppose after children are randomly assigned to consume either low or high amounts of sugar, researchers find proportion hyperactive is greater for those who consumed higher amounts. Response: Improvement:
(C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 19 Example: Experimenters not blind Background : Suppose after children are randomly assigned to diets sweetened either artificially or with sugar, researchers find proportion hyperactive is greater for those who consumed sugar. Question: Can we conclude sugar causes hyperactivity? (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 21 Example: Experimenters not blind Background : Suppose after children are randomly assigned to diets sweetened either artificially or with sugar, researchers find proportion hyperactive is greater for those who consumed sugar. Response: More problematic if responses are assessed __________ Improvement: (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 22
Best evidence of causation
In general, conclusions of causation are most convincing if a relationship has been established in a randomized controlled double-blind experiment. A Closer Look: In the original studies reporting a relationship between sugar and hyperactivity, conducted in the 1970’s, experimenters may have been aware of the children’s diet when they assessed behavior (randomized controlled single- blind). Many studies since then have failed to establish a relationship. (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 23
Other pitfalls in experimentation
Lack of realism (lack of ecological validity) Hawthorne effect (subjects behave differently because of awareness of participation in experiment) Non-compliance Treatments unethical Treatments impractical/impossible to impose
(C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 30
Example: Another flawed experiment
Background : To test if stuttering is a learned (rather than inborn) trait, a researcher in Iowa in 1939 randomly assigned subjects to… Control: 11 orphans in ordinary speech therapy Treatment: 11 orphans badgered and interrupted in sessions with speech therapist Of the 11 in treatment group, 8 became stutterers. Question: What’s wrong with this experiment? (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 32
Example: Another flawed experiment
Background : To test if stuttering is a learned (rather than inborn) trait, a researcher in Iowa in 1939 randomly assigned subjects to… Control: 11 orphans in ordinary speech therapy Treatment: 11 orphans badgered and interrupted in sessions with speech therapist Of the 11 in treatment group, 8 became stutterers. Response: (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 33 Examples: Treatments impossible/impractical Taller men get married sooner, promoted quicker, and earn higher wages… There is a link between obesity and low socio-economic status in women… Height is impossible to control. Weight is difficult to control. Socio-economic status is too costly to control. (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 34
Modifications to randomized experiment
Blocking: Divide first into groups of individuals who are similar with respect to an outside variable that may be important in relationship studied. Paired design: Randomly assign one of each pair to receive treatment, the other control. ( Before-and-after is a common paired design.) Looking Back: blocking is to experimentation as stratification is to sampling.
(C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 35
Example: Blocked experiment
Background : Study tested theory that use of stronger sunscreen causes more time in sun. Before vacation, 40+ students given weak sunscreen, 40+ given strong. Students recorded time spent in sun each day. Question: How to incorporate blocking, if researchers suspect location plays a role in relationship between type of sunscreen and amount of time spent in sun? (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 37
Example: Blocked experiment
Background : Study tested theory that use of stronger sunscreen causes more time in sun. Before vacation, 40+ students given weak sunscreen, 40+ given strong. Students recorded time spent in sun each day. Response: (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 38
Example: Paired experiment
Background : Study tested theory that use of stronger sunscreen causes more time in sun. Before vacation, 40+ students given weak sunscreen, 40+ given strong. Students recorded time spent in sun each day. Question: How to incorporate paired design, if researchers suspect location plays a role in relationship between type of sunscreen and amount of time spent in sun? (C) 2007 Nancy Pfenning Elementary Statistics: Looking at the Big Picture L3. 40
Example: Paired experiment
Background : Study tested theory that use of stronger sunscreen causes more time in sun. Before vacation, 40+ students given weak sunscreen, 40+ given strong. Students recorded time spent in sun each day. Response:
