Download Data Collection Methods: Observational, Self-Report, and Trace Data and more Schemes and Mind Maps Decision Making in PDF only on Docsity!
Data Collection
- Observational -- Self-report -- Trace Data Collection• Primary vs. Archival Data• Data Collection Settings• Data Integrity
- Experimenter expectancy effects– Participant Expectancy Effects– Single- and Double-blind designs– Effects of attrition on initial equivalence
All data are collected using one of three major methods…Behavioral Observation Data
Studies actual behavior of participants
Can require elaborate data collection & coding techniques
Quality of data can depend upon secrecy (naturalistic,disguised participant) or rapport (habituation ordesensitization)
Self-Report Data
Allows us to learn about non-public “behavior” – thoughts,feelings, intentions, personality, etc.
Added structure/completeness of prepared set of ?s
Participation & data quality/honesty dependent upon rapport
Trace Data
Limited to studying behaviors that do leave a “trace”
Least susceptible to participant dishonesty
Can require elaborate data collection & coding techniques
Behavioral Observation Data CollectionIt is useful to discriminate among different types of observation …Naturalistic Observation
- Participants don’t know that they are being observed
- requires “camouflage” or “distance”• researchers can be VERY creative & committed !!!!
Participant Observation
(which has two types)
- Participants know “someone” is there – researcher is a
participant in the situation• Undisguised
- the “someone” is an observer who is in plain view– Maybe the participant knows they’re collecting data…
- Disguised
- the observer looks like “someone who belongs there”
Naturalistic ObservationAdvantages & Possibilities
- Probably offers the best external validity– Participants don’t know they are being observed, and so,
“act naturally”
- Experimental or nonexperimental designs can be used
- RA and Manip can require creativity – but are possible!
Disadvantages & Challenges
- Limited to studying behavior– Important ethical point
Æ
Limited to the observation of
“public behaviors”
- Requires reliable/accurate coding to produce useful data
Oops! Observing behavior without changing that
behavior is more difficult than we thought!
Undisguised Participant ObservationAdvantages & Possibilities
- Behavior can be very “natural” after participants are “used
to the observer”• Habituation -- observer shows up and waits until participant “gets
used to” observer and then begins data collection
- Desensitization -- observer slowly approaches so participant can
gradually “get used to” them
- Experimental or nonexperimental designs can be used
- RA and Manip can require creativity – but are possible!
Disadvantages & Challenges
- Limited to studying behavior– Important ethical point
Æ
Limited to the observation of
“public behaviors”
- Some behaviors/participants don’t habituate/desensitize– Requires reliable/accurate coding to produce useful data
Self-Report Data CollectionAdvantages & Possibilities
- can get data about “non-observables” or “mental behavior”
- thoughts, opinions, attitudes, intentions, plans, etc.
- Experimental or nonexperimental designs can be used
- RA and Manip are readily possible!
Disadvantages & Challenges
- Dependent upon accuracy and honesty of the participant– Ways to improve response honesty
- Promises of anonymity and/or confidentiality• Rapport between researcher and participant
- Ways to improve response accuracy
- Careful construction of questions and their sequence
Trace data
are data collected from the “marks & remains leftbehind” by the behavior we are trying to measure.
There are two major types of trace data…Accretion – when behavior “adds something” to the environment
- trash, noseprints, graffiti Deletion – when behaviors “wears away” the environment - wear of steps or walkways, “shiny places” Advantages –• “unobtrusive measures” – much like naturalistic observation• seldom “modified” or “biased” on purposeDisadvantages –• subject to “differential deposit” & “differential retention” (can’t be
sure that nothing has modified the trace)
- limited range of behaviors leave a durable trace
A famous example of trace-based research began the study ofGarbageology
- the scientific study of society based on what
it discards -- its garbage !!!• Researchers looking at family eating habits used a questionnaire
to collect data from several thousand families about howoften families ate take-out food
- Responses suggested that people ate take-out food about 1.
times per week
- These data seemed “at odds” with economic data obtained from
fast food restaurants, which suggest more like 3 times perweek
- The Solution – they dug through the trash of several hundred
families’ garbage cans before pick-up for 3 weeks –suggested about 2.8 take-out meals eaten each week
This is a good example of the use of “multimethod” data collection – aspart of programmatic research to provide convergent evidence
Data Sources …It is useful to discriminate between two kinds of data sources…Primary Data Sources
- Sampling, questions and data collection completed for the
purpose of this specific research
- Researcher has maximal control of planning and
completion of the study – substantial time and costs
Archival Data Sources (AKA secondary analysis)
- Sampling, questions and data collection completed for
some previous research, or as standard practice
- Data that are later made available to the researcher for
secondary analysis
- Often quicker and less expensive, but not always the data
you would have collected if you had greater control.
Is each primary or archival data?•
Collect data to compare the outcome of thosepatients I’ve treated using Behavior vs. usingCognitive interventions
-^
Go through past patient records to compareBehavior vs. Cognitive interventions
-^
Purchase copies of sales receipts from a storeto explore shopping patterns
-^
Ask shoppers what they bought to exploreshopping patterns
-^
Using the data from some else’s research toconduct a pilot study for your own research
-^
Using a database available from the web toperform your own research analyses
-^
Collecting new survey data using the web
Data collection SettingsSame thing we discussed as an element of external validity…Any time we collect data, we have to collect it somewhere –
there are three general categories of settings
Field•^
Usually defined as “where the participants naturally behave”
-^
Helps external validity, but can make control (internal validity)more difficult (RA and Manip possible with some creativity)
Laboratory•^
Helps with control (internal validity) but can make externalvalidity more difficult (remember ecological validity?)
Structured Setting•^
A “natural appearing” setting that promotes “natural behavior”while increasing opportunity for “control”
-^
An attempt to blend the best attributes of Field and Laboratorysettings !!!
Participant Expectancy EffectsA kind of “demand characteristic” during which participants modify
their behavior to respond/conform to “how they should act”.
Social Desirability
- When participants intentionally or unintentionally modify their
behavior to match “how they are expected to behave”
- Well-known social psychological phenomenon that usually
happens between individual’s and their “peer group”
- Can also happen between researcher and participants
Acquiescence/Rejection Response
- If participant thinks they know the research hypothesis or
know the behavior that is expected of them they can “try toplay along” (acquiescence) or “try to mess things up”(rejection response)
- Particularly important during within-groups designs – if
participants think study is “trying to change their behavior”
Participant Expectancy Effects
:^
Reactivity & Response Bias
Both of these refer to getting “less than accurate” data from the participantsReactivity
is the term commonly used when talking about observational data
collection– the participant may behave “not naturally” if they know they are being
observed or are part of a study
- Naturalistic & disguised participant observation methods are intended to
avoid this
- Habituation and desensitization help when using undisguised participant
observation
Response Bias
is the term commonly used when talking about self-report
data collection and describes a situation in which the participant respondshow they think they “should”–
The response might be a reaction to cues the researcher provides– Social Desirability is when participants describe their character, opinionsor behavior as they think they “should” or to present a certain impressionof themselves– Protecting participants’ anonymity and participant-researcher rapport areintended to increase the honesty of participant responses
Type of Data Collection
Observational
Self-report
Researcher Participant
Expectancy Expectancy
Reactivity“reacting” tobeing observed
Response Bias“dishonest”responding
Observer Bias“inaccurate datarecording/coding”
Data collection biases & inaccuracies -- summary
Interviewer Bias“coaching” or“inaccuraterecording/coding”
Single & Double-blind ProceduresOne way to limit or minimize the various biasing effects we’ve
discussed is to limit the information everybody involved has In
Single Blind Procedures
the participant doesn’t know the
hypotheses, the other conditions in the study, and ideally, theparticular condition they are in (i.e., we don’t tell how the taskor manipulation is designed to change their behavior) In
Double-blind Procedures
neither the participant nor the
data collector/data coder knows the hypotheses or otherinformation that could bias the interaction/reporting/coding ofthe researcher or the responses of the participants Sometimes this simply can’t be done (especially the researcher-
blind part) because of the nature of the variables or thehypotheses involved (e.g., hard to hide the gender of aparticipant from the researcher who is coding the video tape)
Attrition
- also known as drop-out, data loss, response refusal,
& experimental mortality
Attrition endangers initial equivalence of subject variables• random assignment is intended to produce initial equivalence of
subject variables – so that the groups (IV conditions) haveequivalent means on all subject variables
(e.g., age, gender,
motivation, prior experience, intelligence, topical knowledge, etc.)
- attrition can disrupt the initial equivalence
- producing inequalities
- “differential attrition” – related to IV condition differences – is
particularly likely to produce inequalities
- e.g., If one condition is “harder” and so more participants
drop out of that condition, there is likely to be a“motivation” difference between the participantsremaining in the two conditions (i.e., those remaining inthe harder condition are more motivated).
So, “attrition” works much like “self assignment” to trash
initial equivalence
Both involve a non-random determination of who provides data forwhat condition of the study!Imagine a study that involves a “standard treatment” and an“experimental treatment”…• random assignment would be used to ensure that the
participants in the two groups are equivalent
- self-assignment is likely to produce non-equivalence (different
“kinds” of folks likely to elect the different treatments)
- attrition (i.e., rejecting the randomly assigned condition) is
similarly likely to produce non-equivalence (different “kinds”of folks likely to remain in the different treatments)
