Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Naturalistic observation, field research and laboratory research become symbiotic, with strengths and weaknesses that complement one.
Typology: Lecture notes
1 / 40
This page cannot be seen from the preview
Don't miss anything!
Field Research Methods Elizabeth Levy Paluck Princeton University Robert B. Cialdini Arizona State University Chapter for the Handbook of Research Methods in Social and Personality Psychology , Second Edition, Ed. Harry Reis and Charles Judd In press, November 6, 2011
Introduction “... In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless…” Jorge Luis Borges, “On exactitude in science,” 1999 Theories are like maps. Social psychologists typically use theories as maps to find phenomena that are worth investigating, or to deduce unexplored pathways between those phenomena. After formulating their hypotheses, they design a laboratory experiment that simulates the local conditions described in their theoretical map. Social psychologists often do not see a place for field research in this cycle between theory and laboratory experimentation. Field research, they fear, will bloat their theoretical maps with too many added variables, distorting causal pathways. Observational fieldwork used to identify interesting phenomena, or experimentation outside of the highly controlled laboratory environment, so the argument goes, will produce unwieldy theoretical maps akin to Borges’ maps that were the exact size and scale of the Empire. Years ago, one of us (Cialdini, 1980) pointed out that on the contrary, field research can help social psychologists draw accurate theoretical maps that identify the most consequential social psychological phenomena. While theoretically driven laboratory experimentation can produce accurate maps, they may not tell social psychologists about the most interesting or important locations. Furthermore, it is by cycling through field observation, experimentation, and theory that social psychological theories can become precise as well as meaningful. This claim is rooted in a longstanding call for more fieldwork
in abstract theory and to observe and test these theoretical processes in laboratories. Many important figures in social psychology, themselves experts in laboratory experiments (e.g., McGuire 1969 ), have lamented this rather myopic methodological focus. There are a few reasons why field-based observation, measurement, and experimentation have not historically been as prominent as laboratory work (Cialdini, 2009b). One simple reason is that social psychologists are not typically trained as a matter of course in field methodology. We have designed this chapter to be a systematic treatment of various options in field research, so as to redirect students’ and researchers’ eyes toward these methods. It is our hope that awareness of the uses and advantages of these field methods, paired with an understanding of when and how they can be implemented, will promote more social psychological research in the field. The chapter is laid out as follows. We first explain what we mean by field research as opposed to laboratory research, and discuss advantages that come from finding and testing ideas in the field. We explore the range of theoretical goals that can be accomplished with field research. We point out strengths and weaknesses of various field research techniques, and some best practices of each one. We conclude with practical suggestions and reasons for researchers at various stages of experience to engage in field research. Laboratory and field research What is field research? “Field” research is, of course, not defined by its physical locale, but by the work’s degree of naturalism. Defining field research as relatively more naturalistic elements allows for a continuum-based (rather than dichotomous) conceptualization of the approach. After all,
the laboratory can be the site of very realistic interventions, and conversely, artificial interventions may be tested in a non-laboratory setting. When assessing the degree to which studies qualify as field studies, one must consider the naturalism of four aspects of the study: (a) participants, (b) the intervention and its target, (c) the obtrusiveness of intervention delivery, and (d) the assessed response to the intervention. For example, a study on the effects of interpersonal empathy might involve (a) undergraduate psychology majors, (b) written instructions aimed at the participant’s perception of a sad story, which vary systematically according to whether (c) instructions to empathize with the protagonist of the story are included or not, and (d) outcome measures such as empathy and willingness to help scales. Relatively more naturalistic versions of each aspect of this study are (a) non-psychology major young adults or citizens of the local town, (b) a television station broadcasting a sad story, which varies systematically in terms of its (c) language and imagery that encourages or does not encourage the participant to take the perspective of the protagonist, and (d) measures such as the participant’s facial expression as they watch the screen, or their response to a nearby confederate who disparages the protagonist. Note that this experiment could be conducted in a laboratory that has been outfitted to look like a waiting room with a television, which would make the laboratory more naturalistic. Cronbach (1982) suggests an acronym, UTOS, to use when assessing the naturalism of a study: Units, Treatments, Observations, and Setting. To this list of considerations, Reis and Gosling (2010) add that non-laboratory research settings differ from the laboratory in the goals that are likely to be activated by the setting, the setting’s correspondence with the
Cialdini ( 1980 ) wrote that theory-driven experimentation without attention to the real world could result in an accurate but less consequential map, or even a misleading map. As is often the case, the greatest strength of laboratory research—its control—is also part of its weakness. We elaborate on this point below in terms of Cronbach’s (1982) scheme of Units, Treatments, Observations, and Settings, and in terms of psychologists’ concerns about culture, complex systems, and identification of “extraneous” influences. Units. Undergraduate students are predominantly used as participants in psychological laboratory settings because of their convenience, tradition, and financial discount for academic investigators. For the purposes of building widely applicable theories, undergraduates present several troubling bias. Their developmental stage and their particular social and educational backgrounds may exaggerate some effects and diminish others, or restrict the range of variation on the dimensions being studied (Henry, 2008; Sears 1986). Treatments. The treatments administered in laboratory settings are typically weaker, briefer, and less varied than the naturally occurring phenomena in the world that they are designed to simulate. The fact that treatments are weaker is often due to experimenters’ ethical obligation to avoid intense or distressing events, such as authority coercion, severe disappointment or sadness, or sexual harassment (cf. LaFrance & Woodzicka, 2005). Laboratory treatments are brief to accommodate the typical hour-long sessions allotted to participants. As a consequence, researchers use a “reactive or acute form” of a variable to stand in for longer-term phenomena. For example, when studying low self-esteem, laboratory experimenters must lower self-esteem with negative feedback or an experience of failure, rather than observe the process of erosion of self-esteem over a longer term. Unfortunately, “an occasion of low self-esteem may have nothing to do with a lifetime of low self-esteem”
(Ellsworth, 1977 , p. 607). Short-term states may not operate under the same underlying processes as chronic states, which are almost impossible to study experimentally in the laboratory over long time periods (cf. Cook’s [1978] months-long laboratory studies of interracial workgroups). Finally, there is often little variety in the types of laboratory treatments used in the laboratory. Investigators rely on a few established paradigms to study a variety of outcomes, and very rarely translate their abstract theoretical ideas into new operationalizations. “The mental dexterity demonstrated in dealing with abstractions often seems to vanish at the translation stage, as the old standard treatments and measures are used and reused with little consideration of their suitability for the task at hand (i.e., choosing a concrete version of an abstract question)” (Ellsworth, 1977, p. 604; see also Webb, Campbell, Schwartz, & Sechrest, 1966 ). Observations. Very rarely are the behaviors measured in the laboratory commensurate with the behaviors that investigators wish to explain in the real world. First, many outcomes examined in the lab, such as reaction times, are rarely important outcomes in and of themselves in real world settings. Second, lab-based pseudo-behaviors like deciding the salary of a fictional person, or assigning a sentence to a fictional criminal in a jury vignette, may not result from the same interpersonal and intrapersonal processes that produce these behaviors in the world. This difference is troubling for theory testing and building. Most often, laboratory investigations measure self-report rather than behavior (Baumeister, Vohs, & Funder, 2007). Settings. Social psychologists strive to study the interaction of the person and the situation, but there is very little work that describes situations themselves (cf. Kelley, Holmes,
“More and more, we are coming to recognize that [variables’] interrelations may be causal but much more complicated than we can assess with our usual methods” Ellsworth (1977) asserts; “[i]t is in just these instances [of complex relationships] that the typical laboratory experiment is weakest; so much is held constant that there is no opportunity for this sort of complex causation to manifest itself” (p. 614). Moreover, there are many unobservable variables operating in a real world context, variables of which investigators may not be aware when they set out to simulate that context in he laboratory. Consequently, a relationship uncovered between two variables in the laboratory may not exist or may occur rarely in the world because of the interference of this unobserved variable. Preceding, co-occurring, or proceeding variables in the real world may diminish the relationship identified. While we have catalogued many disadvantages of laboratory settings and research paradigms, these critiques should not be taken solely as arguments to incorporate fieldwork into a research program. These preceding points can also be used to inspire more rigorous laboratory experiments that test and produce theoretical maps that are, in Lewin’s words, “closer to life.” Advantages of the field The most obvious advantage of the field is that the investigator does not always have to work as hard to make the units, treatments, observations, or settings of a study naturalistic. Participants are those people involved in the treatment or who come from the social group of interest; treatments can be more high impact and lengthy than a laboratory intervention; outcome measures can be those that already occur in that setting.
Definition of constructs. Selecting the location and the participants for a field-based study helps investigators to precisely define the nature and scope of their theoretical constructs. Take the following example. Suppose that you are interested in studying cooperation. You understand that your choice of setting (e.g., households of married couples, a cheese cooperative in Berkeley, a financial trading floor) and your participant population (e.g., adults, kindergarteners, residents of a small-scale agriculturalist society) change what you mean by “cooperation” and how you will measure it. As you eliminate certain types of settings and populations, you refine your concept of what kind of cooperation you will be able to describe and theoretically map in relationship to other constructs. Leaving the laboratory’s standardized paradigms and generating a list of possible settings, participants, and measurements reveals implicit assumptions or theoretical confusions about your construct (Ellsworth, 1977). Inductive power. Another advantage of research in field settings is that it can provide an inductive approach to theory that begins with facts about cognition, emotion, and behavior in the world, rather than a deductive approach that begins with abstract theory. Cialdini (1980) described the inductive capacity of field research as a “steadily developing sense of which of our formulations account not just for aspects of human behavior, but also for aspects of the behaviors that matter” (Cialdini, 1980 , p. 26). Moreover, to generate ideas in the first place, McGuire (1969) suggested that investigators would spend more productive time observing field settings as opposed to reading the top journals. Using fieldwork to establish the strength, frequency, and surrounding conditions of an effect is a powerful approach to assembling the building blocks of a new theory or to modifying an existing theory.
viewed in this way by certain important individuals comes from a pair of experiences of one of the authors. At two separate meetings of high-level government officials, he was labeled not as a social psychologist but as a behavioral economist because, he was informed, it was judged to be more palatable to the participants. Field observational methods Observation in the field is not a supplement to empirical work—it is empirical work. For example, Cialdini’s (1980) full-cycle model endorses “a more empirical science that is based firmly in the observation of everyday worlds.” From this perspective, observation and experimentation are each “but one tool in the social psychologist’s repertoire,” and each is “better suited to some tasks than others” (Adams & Stocks, 2008, p. 1900). Observational methods can be put to many important purposes in field settings. Observation of individual and group behavior can generate hypotheses and theoretical insights, or point researchers toward phenomena that are powerful and prevalent in the community (Cialdini & Mortensen, 2010). Or, as Solomon Asch once pointed out, observation can help researchers to become more familiar with the phenomenon of interest: “[b]efore we inquire into origins and functional relations, it is necessary to know the thing we are trying to explain” (Asch 1952, p. 65, cited in Reis and Gosling, 2010). This includes observing and describing the types of situations that give rise to the phenomenon (Kelley et al., 2002). In addition, observational measurement techniques like interviews or behavioral trace indicators, described below, can be used as outcome measurements for field or laboratory experiments. All observational measures, particularly those that are highly unobtrusive, can serve as strong verification of self-report data.
Qualitative methods Qualitative methods that are used to explore or describe phenomena include personal observation, participant observation, structured interviews, and ethnography. Notes that are produced by these methods can be coded and written up for publication (see Emerson, Fretz, & Shaw, 1995). If the data were collected in a systematic manner, qualitative outcomes may be used as outcome measurements in a study by quantitatively coding and analyzing the data as events or ratings (see Paluck, 2010). Personal observation. Personal observation is a time-honored tradition of hypothesis generation in social psychology. A classic example is Festinger’s (1957) observation that catastrophes are followed by rumors of further disaster rather than reassurances of relief, which led to his formulation of cognitive dissonance theory. Although many social psychologists report that ideas and counterintuitive notions were inspired by real world observation, observational skills are not often recognized as part of the social psychologist’s official toolkit. McGuire ( 1973 ) urged psychologists to “[cultivate] habits of observation that focus one’s attention on fertile aspects of natural experience,” adding “[we should] restructure our graduate programs somewhat to keep the novice's eye on the real rather than distracting and obscuring his view behind a wall of data” (p. 453). Keeping one’s “eye on the real” could involve training oneself to be more alert in everyday life, delving into written accounts of everyday life through various peoples’ eyes in blogs or newspapers, or using more systematic observation techniques like participant observation. Participant observation. Participant observation involves observation while participating in an institution, social group, or activity. For example, investigators can participate in skilled practitioner trainings, as did Cialdini (1993) in his observation of sales
(2005) combined interviews with field observation to uncover the concept of enemyship (a personal relationship of hatred) in West Africa and North America. Observation-based estimates of individual or population characteristics Individual and population characteristics can be inferred from observational field methods such as daily diary techniques, trace measures, ambulatory assessment, and social network mapping (see also Reis, Maniaci, & Gable, this volume). These types of observational data can be collected in person, in archives, or can be harvested from the Internet. Individual characteristics. Daily diary methods “capture life as it is lived” (Bolger et al 2003, p. 95 ). Participant are asked to fill out reports about their behavior, affect, cognition, and/or surroundings at regular intervals or when prompted at random times by a PDA or a cellphone. Diary methods serve the important descriptive purpose of cataloguing information about the prevalence, the chronological timing, and the co-occurrence of events and situations (Reis & Gosling, 2010). Trace measures bring out the Sherlock Holmes in social psychologists. To unobtrusively track psychologically meaningful behavior, psychologists seek out systematically or automatically recorded traces of behavior in official archives or unofficial spaces of everyday life. The advantage of these measures is that the subjects of study are unaware that they are being watched. For example, from official public records, investigators can study government voting or hospital immunization records and yearbook photos. Investigators might even obtain data from retail stores on customer loyalty card activity showing individuals’ purchases of fruits and vegetables, cigarettes, or other products. From the “unofficial” records, social psychologists have mined trash cans to measure alcohol
consumption (Rathje & Hughes, 1975), and counted the number of vinyl tiles that needed to be replaced around various museum exhibits as a measure of interest in the exhibit (Duncan, personal communication, cited in Webb et al., 1981). Analyzing the composition of personal Internet profiles on social networking websites is one of the latest ways to use trace measures (Reis & Gosling, 2010). Online social networking sites also help investigators to identify individual’s network of potential social influences (e.g., Goel, Mason, & Watts, 2010). Another individual-level measurement technique is ambulatory assessment, which uses electronic equipment to measure an individual’s movement and states of being throughout their daily lives. This includes blood pressure monitors, sound recording (Pennebaker, Mehl, Crow, Dabbs, & Price, 2001), and GPS tracking devices located in individuals’ mobile phones. Some tracking devices can even assess which people in a social network, such as a school social network, interact the most frequently, and for how long. The advantages of these observational methods is that they capture daily experiences as they occur in the stream of natural activity across different situations. Many of these methods produce time series data, which means they can evaluate hypotheses regarding the chronological ordering of a particular process and within-person processes (Reis & Gosling, 2010, pp. 96 - 97). Depending on the way they are collected, observational measures can overcome biases of self-report. For example, in the garbage trace measures collected by Rathje and Hughes (1975), 15% of households reported at the front door that they drink beer, while beer cans were found in the trash can at the back door in 77% of the same group of households. Disadvantages of individual observational methods include noncompliance or misreporting, in the case of daily diaries. In addition, there may be imprecise translation
however, prevent the investigator from connecting individuals to behaviors, which means that these outcome measures are best used for description, hypothesis generation, or for an experiment in which the community is the unit of randomization and analysis. Observation of situation characteristics Despite the stated importance of the situation in social psychological analysis (Rozin, 2001), very little observational work has been devoted to establishing a taxonomy of different situations. The work of Kelley and colleagues (2003) is a notable exception, in which the authors classify and describe 21 of the most common everyday situations thought to influence various aspects of interpersonal behavior. We believe that psychologists could make greater use of this work, and expand upon it with situational taxonomies relevant to other types of behavior, cognition, or emotion. Ultimately, observational research in field settings involves detecting “pieces of data not specifically produced for the purpose of comparison and inference but available to be exploited opportunistically by the alert investigator” (Webb et al., 1981, p. 5 ). We now turn to experimental research in the field that is explicitly designed for the purpose of comparison and causal inference. Field experimental methods Experimentation in field settings can be just as rigorous as in a laboratory setting. Treatments can be randomly assigned and delivered in a standardized manner to individuals, groups, or institutions, and standardized outcome measures and evidence speaking to the process of change can be collected. Causal inference in field settings has greatly improved over the years, mostly through innovations in field experimental design that address
challenges particular to field settings (Green & Gerber, in press; Rubin, 2005; Shadish, Cook, & Campbell, 2002; West, this volume). Randomization and control in field settings Psychologists who conduct laboratory experiments may approach field experimentation with two types of trepidation. One concern is that ongoing activity in field settings will destroy pure randomization and segregation of experimental and control groups. A second is that many things are simply impossible to randomize in a field setting. On the first point, psychologists might be pleasantly surprised to find the many varieties of experimental designs that field experimentalists have developed to preserve the integrity of experimental design against special situations that arise in the field. On the second point, canvassing the types of psychological field experiments conducted over the last decade reveals few limits on the kinds of treatments that have been randomized. Social psychologists have randomized a variety of interventions, in national parks (Cialdini, 2003), on national radio (Paluck, 2009), in schools (Blackwell, Trzesniewski, & Dweck), and amusement parks (Gneezy, Gneezy, Nelson, & Brow, 2010), targeting and measuring psychological phenomena from perceived norms, beliefs, implicit theories, to social welfare concerns, and connecting them with real world behavior. For most social psychologists, whether their interests lie in basic or applied research, these are important and worthy investigations. Many field experiments involve simple random assignment of a treatment to individuals, households, or communities. Investigators deliver the treatment, or they collaborate with an organization that is already intending to deliver the treatment. However, many types of treatments are impossible to package neatly and randomly deliver to some
