Download Data Collection Methods: Experiments vs. Observational Studies - Prof. Rickie Domangue and more Study notes Statistics in PDF only on Docsity!
Chapter 4
Gathering Data
4.1 Should We Experiment or Should We Merely Observe?
A. Types of Studies: Experimental or Observa- tional
- Experiment, page 148. A researcher conducts an experiment by as- signing subjects to certain experimental condi- tions and then observing outcomes on the re- sponse variable. The experimental conditions, which correspond to assigned values of the ex- planatory variable, are called treatments
Examples:
- Observational Study, page 149 In an observational study, the researcher ob- serves values of the response variable and ex- planatory variables for the sampled subjects, with- 1
out anything being done to the subjects (subject as imposing a treatment).
Examples:
B. Advantages of Experiments over Observational Studies
- Lurking variables possible in observational stud- ies
- Lurking variables less likely in experiments, can balance groups with random selection so there is no association between lurking variable and ex- planatory variable (treatment variable).
- Establishing cause and effect is central to sci- ence (page 151). Observational studies cannot give definitive answers about cause and effect be- cause of lurking variables. (Remember: Associa- tion does not imply causation)
- Can study the effect of an explanatory varible on a response variable more accurately with an ex- periments because researcher has control of which treatments subjects receive.
C. What Type of Study is Possible?
- If experiments preferred why conduct observa- tional studies? - Possible ethical issues with experimentation
4.2 What are Good Ways and Poor Ways to Sample?
A. Steps in Carrying out a Sample Survey
- Define population targeted by the study
- Compile list of subjects (Sampling frame) in the population from which the sample is to be taken Not always easy
- Specify sampling design, method for selecting subjects from sampling frame. B. Methods of selecting subjects
- Nonrandom sampling (See Section G)
- Random Sampling
- More likely to obtain repesentative sample
- Enable researcher to measure likelihood of sam- ple characteristic falling close to correspond- ing population characteristic C. Simple Random Sampling - One type of random sam- pling
- A simple random sample of n subjects from a population is one in which each possible sample of that size has the same chance of being selected.
- How to Select a Simple Random Sample
- “Pull names from a hat”
- Using random numbers to select a simple ran- dom sample, See page 158
D. Methods of Collecting Data from Selected Individu- als in Sample Surveys Methods:
- Personal Interview
- Telephone interview random digit dialing - often used, don’t need sampling frame.
- Self-administered questionnaire
E. How Accurate are Results from Surveys with Ran- dom Sampling?
- margin of error
- √^1 n x 100% = rough approximation for margin of error
F. Be Wary of Sources of Potential Bias in Sample Sur- veys
- Bias in Sample Surveys Results from sample not representative of the population.
- Types of Bias in Sample Surveys, page 163
- Sampling bias occurs from using nonran- dom samples (see section G) or having un- dercoverage
Examples:
Sampling bias - one segment of population may be more likely to volunteer than other segments.
- Convenience samples sometimes necessary, both in observational studies and in experiments.
H. Large Sample Size Does Not Guarantee Unbiased Samples Page 165: “Almost always better off with a simple random sam- ple of 100 people than with a volunteer sample of thousands of people”
4.3 What are Good Ways and Poor Ways to Ex- periment?
A. Terminology
Recall: In an experiment researcher assigns sub- jects to experimental conditions, called treatments to see the effects that treatments have on response variable
experimental units The people, animals, objects, etc. that receive the treatments
B. The Elements of a Good Experiment
- Control Comparison Group
- Example 4.36, page 173
- Control Group, no vitamin C
- Placebo Effect
- Control Group, placebo
- Control Group, existing treatment
- Randomizing in an experiment Randomizatin refers to randomly assigning ex- perimental units to treatments Randomization used to: - Eliminate bias that may result if the researcher assigns subjects to treatments - Balance groups on variables that you know affect the response - Balance groups on lurking variables that may be unknown to researcher
- Blinding the study Single Blind Experiment Subjects do not know what treatment they have received Not always possible Double Blind Experiment Neither subjects nor those measuring response know which treatment are given to subjects
- Replication
C. Generalizing Results to Broader Population
C. Blocks in an Exp. - Generalization of Matched Pairs (more than 2 treatments)
- MP Type I: Block = subject that receives both treatments MP Type II: Block = pair of units
- More than two treatments, say K, treatments Block = subject that receives all K treatments. Order of treatments is randomized.
Block = set of units (number equal to K) which have been grouped according to similar values on some extraneous variable. K treatments assigned at random to units within each block
- Purpose of Blocking (similar to matching): Keeps potential lurking variables from affecting the results
- Randomized Block Design A block design with random assignments of treatments to units within a block.
D. Samples Surveys: Other Random Sampling Designs Useful in Practice
- Cluster Random Sample
- Definition, page 176. Divide the population into a large number of clusters, such as city blocks. Select a simple random sample of the clusters. Use all the subjects in the sampled clusters as the sample. This is a cluster ran- dom sample.
- Cluster sampling preferable if ∗ a reliable sampling frame is not available ∗ cost of selecting a simple random sample is excessive
- Disadvantage: usually need a larger sample size as compared to simple random sampling
- Stratified Random Sample
- Definition, page 177. A stratified random sam- ple divides the population into separate groups, called strata, and then selects a simple ran- dom sample from each stratum. The different simple random samples put together make up the stratified random sample.
- Advantage: can include in your sample how- ever many subjects you want from each stra- tum
- Disadvantage: must have a sampling frame and know the stratum into which each subject belongs.
- In practice, complex surveys: some combination of simple, cluster, and stratified sampling used.
E. Types of Observational Studies
- Types of Observational Studies
- Cross-sectional (surveys). Take a cross section of a population at the current time
- Retrospective Look back in time to obtain some information
