Download Analysis During Data Collection and more Summaries Calculus in PDF only on Docsity!
Analysis During
Data Collection
In this chapter we describe methods for qualitative data analysis that are especially useful during the ongoing process of data collection. Most analysis methods can be used during data collection, of course-these methods are especially helpful ones. Why analyze during data collection at all? Some qualitative researchers put primary energy into data collection forweeks, months, or even years, then retire from the field to "work over their notes." We believe this is a serious mistake. It rules out the possibility of collecting new data to fill in gaps, orto test new hypoth- eses that emerge during analysis; it tends to reduce the production of what might be termed "rival hypotheses" that question the fieldworker's routine assumptions and biases; and it makes analysis into a giant, over- whelming task that both demotivates the researcher and reduces the quality of the work produced. To take the approximate obverse of these points: Analysis during data collection lets the fieldworker cycle back and forth between thinking about the existing data and generating strategies for collecting new-often better quality-data; it can be a healthy corrective for built-in blind spots; and it makes analysis an ongoing, lively enterprise that is linked to the energizing effects of fieldwork. Furthermore, ongoing analysis permits the production of the interim reports that are a part of most evaluation and policy studies. So the ideal model for data collection and analysis is one that interweaves them from the beginning. Periodic field visits are interspersed with time for data reduction and display, for drawing conclusions, and for testing those conclusions-either through other analyses in the existing data base, o r through a new round of data collection. We are only reiterating here the interactive, cyclical nature of qualitative data analysis already outlined in Chapter I.
This chapter describes six major methods useful for analysis during data collection, along with six supple- mentary ones. As we have indicated, each of the major methods is presented in this format:
Name of method. Analysis problem. The problem, need, or difficulty faced by a qualitative data analyst, for which the method is a useful solution. Brief description. What the method is and how it works. Illustration. In more detail, a "minicase," showing how the method is developed and used. Usually, this section will have a variety of subheadings, such as "developing the format," "entering the data," and "analyzing the data." Variations. Alternative approaches that use the same general principle. Work of other researchers is cited. Advice. Summarizingcomments about the useof the method, and tips for using it well. Time required. Approximate estimates to guide the researcher (these will naturally vary according to subject matter, the researcher's skill, the research questions being asked, the number of sites, and so on).
Thesupplementary methods are described in boxes, usually o n o n e or two pages. The aim is to suggest simple methods that can be used profitably in con- junction with the major method being discussed. The format varies, but usually includes a brief statement of the problem for which the method is a solution, plus a brief exhibit or illustration, and concluding advice. Our assumptions about "data."The methods being described in this and following chapters assume that the fieldworker has collected information in the form of handwritten field notes, or notes dictated in the field, or (more rarely) tape recordings of events in the
50 Qualitative Data Analysis
field setting. In all cases we are focusing on words as the basic form in which the data are found.' We further assume that the basic, raw data (the scribbled field notes, the dictated tapes, the direct tape recordings) are subjected to more processing before they are available for analysis. Field notes must be con- verted into "write-ups," either through typing or dictati0n.A write-up is a product intelligible to anyone, not just the fieldworker. It can be read, coded, and analyzed using any of the methods we are about t o describe. Raw field notes themselves are usually par- tially illegible, and contain many private abbreviations. They are also sketchy. O n e estimate is that field notes of an interview usually contain one-half or less of the actual content. But a write-up will usually add back some of the missing content, since the raw field notes, when reviewed, stimulate the fieldworker t o remember things said at that time that are not in the notes. Such additions should, of course, be marked specially, to guard against bias. Similarly, dictated notes are not ready for analysis, but must ordinarily be transcribed onto paper and usually edited for accuracy by the fieldworker before they are ready for use. Finally, direct tape recordings of field events must be either transcribed fully (if the aim is to have a full record of speech and other audible events), or pro- cessed in some way (for example, the fieldworker listens to the recording, makes notes, selects excerpts, makes judgments o r ratings, and s o on). Thus, for the methods we review below, we are focusing on words as the basic medium, and we assume that the words involved have been refined one step beyond their form at the point of data collection (raw notes, tape recordings), so that they are clear to any reader or analyst. Now, on to the methods. They are roughly arranged from early t o late in data collection, and from simple to complex. Beginning with the contact summary sheet, a simple way to summarize time-limited data, we proceed through first-level coding, second-level or pattern codes, and the process of deriving even more general themes called "memoing. "As more and more data pile up, the site analysis meeting and the interim site summary prove more and more crucial for understanding.
1II.A CONTACT SUMMARY SHEET
Analysis Problem After an intensive field contact (from one t o several days) has been completed, and field notes are written
up in systematic form, there is often a need to pause and consider: What were the main themes, issues,
problems, and questions that I saw duringthis contact?
Without such reflection, it is easy to get lost in a welter of detail. And communicating important things about acontact to one's colleagues is essential for any project with more than one fieldworker.
Brief Description A contact s u m m a y is a single sheet containing a series of focusing or summarizing questions about a particular field contact. The fieldworker reviews the written-up field notes, and answers each question briefly to develop an overall s u m m a y of the main points in the contact.
Deciding on the questions. The main thing here is being clear about what you (or your colleagues) need to know quickly about a particular field contact (which may itself have run t o anywhere from a half dozen t o a hundred or more pages of written-up field notes), and which questions will locate the essence of the data in the contact. Some possibilities follow: What people, events, or situations were involved? What were the main themes or issue? in the contact? Which research questions did the contact bear most centrally on? What new hypotheses, speculations, or guesses about the field situations were suggested by the contact? Where should the fieldworker place most energy during the next contact, and what sorts of infor- mation should be sought? Making the form. The questions should bearranged on a single sheet of paper (using more than both sides of one sheet defeats the purpose), with space for the fieldworker's answers. Identifying information on the site, the contact, the fieldworker, and the date should be indicated as well. Entering the data. A contact summary sheet is usually best filled out as soon as fully written-up field notes have been reviewed and corrected by the fieldworker. At that point, one has a perspective that combines reasonable immediacy with a reflective overview of what went on in the contact. One can include one's own reflective remarks (see Box III.B.a), as well as unanswered questions for the next contact. On the other hand, waiting until a contact has been thoroughly and fully coded is probably too late. In addition, the process of coding usually adds s o many additional hunches and thoughts about the contact
52 Qualitative Data Analysis
Chart 5a Contact Summary Form: Illustration c o n t a c t t y p c : v i s i t - P h o n e( w i t h whom?) W r i t t c n b yT o d a y ' s^ date- BLT
W h a t w e r e t h c m a i n i z o r t h c m c s t h a t s t r u c k y o u i n t h l s c o n t a c t? interplay b e t w e e n h i g h l y p r e s c r i p t i v e , " t e a c h e r - p r o o f "^ c u r r i c u l u m t h a t i s top-down i m p o s e d a n d t h e a c t u a l w r i t i n g o f t h c c u r r i c u l u m b y t h e t e a c h e r s themselves. S p l l t b e t w e e n t h e " w a t c h d o g s " ( a d m i n i s t r a t o r s ) a n d t h e " h o u s e m a s t e r s " ( d e p t. c h a i r s 6 t e a c h e r s ) v i s - a - v l s j o b f o c i. D i s t r i c t c u r r l c. c o o r d ' r a s d e c i s l o n m a k e r re s c h o o l ' s a c c e p t a n c e o f r e s e a r c h r e l a t ~ o n s h i p.
S u m m a r i z e t h e i n f o r m a t i o n y o u g o t ( o r f a i l e d t o g c t ) o n e a c h o f t h c t a r g e t q u e s t i o n s y o u h a d f o r t h i s c o n t a c t. Q u e s t i o n I n f o r m a t i o n N a t u r e o f t h e i n n o v ' n P r e s c r i p t i v e ~ n g l l s h , 1 y r r e a d i n ge a c h i n p r o g ' mm a t h 6 ( 4 s c l e n c e l y r s i n H i s t o r y o f d e v. o f i n n o v ' n C o n c e p t u a l i z e d E n o l i s h C h a i r m a n^ b y 6 C u r r i c. A s s o c.^ c o o r d ' r ,C h a i r m a n ; w r i t t e n b y t e a c h e r s i n summer; r e v i s e d b y t e a c h e r s f o l l o w r n g summer w l t h f i e l d t e s t i n g d a t a S c h o o l ' s o r g ' l s t r u c t u r e P r i n c i p a ld i s c i p l i n e ; (^6) a d m l n ' r sd c p t. chairs r e s p o n s i b l e a r e e 2 u c ' l f o r l e a d e r s D e m o g r a p h i c s
T e a c h e r r c s p o n s e t o r n n o v ' n R e s e a r c h a c c e s s
R a c i a l c o n f l i c t s i n l a t e 60's: 602 b l a c k s t u d. p o p. : h e a v y e m p h a s i s o n d i s c i p l i n e 6 o n k e e p i n g o u t n o n - d i s t r i c t s t u d e n t s s l i p p i n g i n f r o m C h l c a g o R i g l d , s t r u c t u r e d , e t c. a t f i r s t ; now, t h e y s a y t h e y l i k e it//NEEDS EXPLOR'N V e r y g o o d ; o n l y r c s t r i c t l o n : t e a c h e r s n o t r e q u i r e d t o c o c p e r a t c 3. A n y t h i n g e l s e t h a t s t r u c k y o u a s s a l i e n t , i n t e r e s t i n g , i l l u m l n n t i n g o r i m p o r t a n t i n t h l s c o n t a c t? T h o r o u g h n e s s o f t h e i n n o v ' n ' s d e v e l o p m e n t a n d t r a i n i n g. I t s e m b e d d e d n e s s I n t n e d l s t r l c t ' s curriculum, p l a n n e d a n d e x e c u t e d b y t n e d l s t r l c t c u r r i c u l u m c o o r d i n a t o r. The i n i t l a 1 r e s i s t a n c e t o i t s h i g h p r e s c p t i v e n e s s ( a s r e p o r t e d b y u s e r s ) a s c o n t r a s t e d w i t h t h e i r c u r r e n t a c c e p t a n c e a n d a p p r o v a l o f i t ( a g a i n , a s r e p o r t e d by u s e r s ).
- What new ( o r remaining) t a r g e t q u e s t i o n s d o y o u h a v e i n c o n s i d e r i n g t h e n e x t c o n t a c t w i t h t h i s s i t e? How d o u s e r s r e a l l y p e r c e i v e t h e i n n o v ' n? I f t h e y d o i n d e e d e m b r a c e i t , w h a t a c c o u n t s f o r t h e c h a n g e f r o m e a r l y r e s i s t a n c e? N a t u r e a n d a m o u n t o f n e t w o r k i n g among u s e r s o f i n n o v ' n HOW much a t t e n t i o n t o g i v e t o t h e o t h e r h i g h s c h o o l u s i n g t h e i n n o v ' n ( T l n d a l e W e s t ). E f f e c t s u p o n n o n - u s e r s o f r e c e i v i n g s t u d e n t s f r o m t h e i n n o v ' n who a r e " m a i n s t r e a m e d " i n t o r e g u l a r E n g l i s h c l a s s e s. I n f o o n " s t u b b o r n " m a t h t e a c h e r s w h o s e i d e a s w e r e n ' t h e a r d i n i t i a l l y -- who a r e t h e y? S i t u a t i o n p a r t i c u l a r s? R e s o l u t i o n? Particulars o f c o n t e n t t e a c h e r s ' work w i t h r e a d i n g s p e c i a l i s t t o d c v c l o p c u r r i c u l u m o f i n n o v ' n. F o l l o w - u p o n E n g l i s h t e a c h e r R e i l l y ' s " f a l l fr5m t h c c h a i r m a n s h i p. " T n l k o f s e c u r i t y n k e d s f r o m o l d - t i m e r s l r k e R e l l l y a n d Kennedy r a i s e i s s u e o f p o s s i b l e t h r e a t f r o m a b o v e ( t h e d i s t r i c t ) I £ t h e y d o n ' t u s c t h e i n n o v ' n w e l l. llow much o f a s q u e e z e p l a y d i d C r o r o c n make, i f a n y? F o l l o w a t e a m t h r o u g h a d a y o f r o t a t i o n , p l a n n i n g , e t c. T n l k w i t h M r. M a c d u f f i n E n g l i s h. COIICERN: The c o n s e q u e n c e s o f e a t i n g s c h o o l c a f e t e r i a f o o d t w o d a y s p e r week f o r t h e n e x t f o u r o r f i v e m o n t h s...
Analysis During Data Collection 53
Chart 5b Contact Summary Form: Illustration with Coded Themes
CONTACT SUMMARY FORM (^) SITE L&&.cf&
Type of c o n t a c t : Mtg. (^) -- 4 / i r c l a / r - -$/,'^ Coder^ & Who, whht group p l a c e d a t e Phone
Date coded /:4/
With whom, by whom p l a c e d a t e
I n f. I n t... w i t h whom,by whom p l a c e d a t e
- P i c k out t h e most s a l i e n t p o i n t s i n t h e c o n t a c t. Number i n o r d e r on t h i s s h e e t and n o t e page number on which p o i n t appears. Number^ p o i n t^ i n^ t e x t^ of^ write-up.^ A t t a c h^ theme o r a s p e c t t o each p o i n t i n CAPITALS. I n v e n t^ themes where no^ e x i s t i n g ones^ a p p l y^ and a s t e r i s k t h o s e. Comments may a l s o b e included i n double p a r e n t h e s e s.
&LO- ~ ~ r s w PDK eF#l:'Ken r e p o r t s Board d e c i s i o n : approval of rezoning p l a n (was p u t o f f d e l i b e r a t e l y i n t i 1 a f t e r t h e e l e c t i o n ).
when t h e y t r a n s f e r. 0 - h-7- t e a c h e r s t o b e d i s t r i b u t e d acooss s c h o o l s --3- f o r #1, 3 foo #2, 2 f o r # 3 , 1 f o r #4 (covering 9 c l a s s e s ). Y r n ~ m v a E 1 5.^ Teachers v a r y i n t h e i r w i l l i n g n e s s^ t o i n e t g r a t e s p e c i a l ed k i d s i n t o t h e i r classrooms---some.,ase " a pain i n t h e elbow". -a+ O M ~ V N ~ C *&(7^ -
- ' 1 p o l n t s o u t t h a t t e n t a t i v e t e a c h e r assignment lists g o t leaked from t h e previous meeting ( ( i m p l i c i t l y d e p l o r e s t h i s ) ). P w a R q c s y
7. Ken s a y s t e a c ers a c t it a s i f ' they had t h e r i g h t t o d e c i d e
who should b e t r a n s f e r r e d " ( m k h would make o u t c r y .- - e t c. ) p- D(J?~U~ F L I C ~ M C ~ M 2, 8. ~ a c l t / e x pc l t d c l s l o n : I t ' s our d e c i s i o n t o make" (voiced by Ken, agreed t o by- ~ d )
- s n % % 7 Y P f 4 & Brown 9, P r i n c i p a l s gnd Ken,John?agree t h a t M s. E p s t e i n i s a " b i t c h " 5 % ) g ~ / , w c c y M G h T L 10. The e u l v a e n t of 0 t e a c h e r s w i l l have t o b e moved (#1 has s u r p l u s of 7%, #4 has s u r p l u s p f 2%, #2 needs 8 , #3 needs 2 ). W E M + 7- b 11.! ~% Ke/%id&t "? ' o l k c h e r s %cad of t i m e (now1 about t r a n s f e r s ("because then we'd have a f a i t accompli")
Analysis During Data Collection 55
Box 1II.A.a Document Summary Form: Illustration
S i t e Carson DOCUMENT FORbI Docurnent + L D a t e r e c e i v e d o r p i c k e d up: r e o. 1 ~
Name or d e s c r i p t i o n o f document: The Buffalo (weekly sheet)
E v e n t o r c o n t a c t , i f a n y , w i t h w h i c h d o c u m e n t i s a s s o c i a t e d : Paul's explanation of the (^) ~~t~ Feb. 1 3 admin. team's functioning S i g n i f i c a n c e or i m p o r t a n c e o f d o c u s e n t : Gives schedule for all events in the district for the week. Enables coordination, knmts 2 schools together. B r i e f aurrmary o f c o n t e n t s : Schedule of everything from freshman girls'basketball to "Secret Pals Week" in the elernntary school. Also includes "Did you know" items on the ll'h program (apparently integrating the IPA News). And a description of how admin team works (wlio is on team, what regular meetings deal with, gives working philosophy ("ex: " we establish personal goals and monitor progressW..We coordinate effort, K-12, and all proe,ramst'.. "Ue agree on staff selection".) Concluding comment: "It is our system of personnel man~gement': Also alludes to the 26 OPERATIONAL CUID?.LZ>ITS (Document 16) ((I'll Bucss that the admin explanation does not appear every week----need to check this.))
I F DOCti:.iEt!T I S CEXTPAL O R CRUCIAL TO )\ PAIITICUI,AR CO!IT:\CT ( e x : a m e e t i ~ l g ilqcr:i!a, n e v s p ~ p e rc l i p p i n g d i s c u s s e d i n a n i n t c r v i c : ~ , e t c. ) , make a c o p y and i n c l u d e w i t h w r i t e - u p. O t h e n : i s e , p u t i n d o c u n e n t f i l e.
into all kinds of mischief. O n e is thus assuming that the chief property of the words is that there are more of some than of others. This, of course, is only one of the things that the words are, and certainly not the most important one. Focusing solely on numbers shifts our attention from substance to arithmetic, and thereby throws out the whole notion of qualitativeness; o n e would have done better to have started with numbers in the first place and saved a lot of time.
Also, when word-derived numbers don't make sense, there is usually no very satisfactory way of making them more intelligible with more numbers, which is all o n e has at hand. The solution to this problem, as we will see in later sections, is to keep words and any associated numbers together through- out the analysis. Essentially, words and numbers keep o n e another analytically honest.
56 Qualitative Data Analysis
Word overload. The words that the qualitative analyst works with are usually in the form of written-up field notes and various kinds of documents that have words on them. They tend t o pile up quickly during data collection. Two weeks at a field site can result in something like 300-400 pages of typed-up field notes and ancillary materials, even with s o m e restraint. Everything looks important, especially at the outset, and the analyst wants t o get it all. What at first seemed simple gets rapidly more complex and has t o be fleshed out. New leads surface and need checkingout. All this adds bulk. The real danger is that, at the e n d of data collection, the analyst will be overloaded with more data than can be processed. Furthermore, the narrative text of field notes is very difficult t o use during analysis. It is spread over many pages, laid out in sequence rather than by topic, and usually has little inherent structure. It becomes difficult to retrieve the words that are most meaningful, t o assemble chunks of words that go together, and to reduce the bulk into readily analyzable units. How then to contend with this? Brief Description A common solution is that of coding field notes, observations, and archival materials. A code is an abbreviation or symbol applied t o a segment of words-most often a sentence or paragraph of tran- scribed field notes-in order to classify the words. Codes are categories. They usually derive from research questions, hypotheses, key concepts, or important themes. They are retrieval a n d organizing devices that allow the analyst t o spot quickly, pull out, then cluster all the segments relating t o the particular question, hypothesis, concept, or theme. Clustering sets the stage for analysis. Illustration Types of codes. Let us assume that a n analyst is interested, as we were in our school improvement study, in the reasons for which a new educational practice is adopted. This may be the sole or o n e of several research questions t o be addressed in a study. The researcher will typically begin by asking infor- mants at the field site why they or others decided t o try out the practice. A piece of the field notes might look like this: I asked him what the need for the new program was, and he responded that the students coming into the 9th grade were two years below grade Ivel, and that the old curriculum was ineffective. Through testing (the Nelson Reading Test) it was determined that students were growing academically only five to six months during the ten-month school year.
Assuming that the analyst found it possible t o apply asinglesummarizingnotation t o this chunk, it might be "MOT" t o indicate "motivation." That code would appear in the left-hand margin beside t he segment (the right-hand margin might be used for a comment; s e e Box 1II.B.b).I f the analyst wanted a little more differen- tiation, the code might separate teachers' motivations from administrators'; we then get "ADM-MOT." Or perhaps o n e might want t o specify the time period or phase in which that motivation appeared, (for instance, the "adoption1' phase, by lengthening the code to read "AD/MOT." Or, to include all these things, "AD/ADM- MOT." These are descriptive codes; they entail no inter- pretation, but simply the attribution of a class of phenomena t o a segment of text. 'The same segment could, of course, be handled more interpretively. Let us assume that, as the field researcher gets more savvy about local dynamics, a more complex, more back- stageweb of motives turns up. S o m e people may have adopted the new practice chiefly t o attract attention t o themse!ves and thereby t o set up a desirable promo- tion. We then have the official motive, such as the o n e in the segment shown above, and the more private or backstage motive. T h e segment we just saw could then be coded "OFF-MOT' (for official motivation) and the other segments "PRIV-MOT." A third class of codes is even more inferential and explanatory. The idea here is t o indicate that a segment of field notes illustrates an emergent leitmotiv or pattern that the analyst has deciphered while unrav- eling the meaning of local events and relationships. These codes can be called what they are-LM (leit- motiv), PATT (pattern), T H (theme), C L (causal link)-and should include a word indicating the theme or pattern. They typically get used later in the course of data collection, a s the patterns come clear. Here is a n example. In the field study of educational innovations, this segment appeared: But he (Mr.Walt) says that he does not knowthat much about what is exactly involved in the SCORE-ON pro- gram. He thinks that "it is a combination of a lot of things." The resource lab appears to be used in the morning for the FACILE program, which Mr. Walt knows a great deal more about.... In the afternoon, Mrs. Hampshire uses the lab for SCORE-ON purposes. Mr. Walt says that this is a different program, and therefore it is a different use. That clump looks innocent enough t o be taken descriptively, which is the way the field researcher saw it during initial interviewing. Several interviews and some causal observations later, however, it looked different. There was apparently an intense power struggle between different factions or "teams" in the
58 Qualitative Data Analysis
Chart 6a Illustration of a Start List of Codes INNOVATION PROPERTIES IP-OBJ 3. IP: OBJECTIVES IP-OC 3.1. IP: ORGANIZATION IP-ORG/DD, LS 3.1. IP: IMPLIED CHANGES--CLASSROOM IP-CH/CL 3.1. IP: IMPLIED CHANGES-ORGANIZATION IP-CH/ORG 3.1. IP: USER SALIENCI- IP-SALIENCI:.^ 3.1. IP: (INITIAL) USER ASSESSMENT IP-SIZUP/PRE.^ DUR^ 3.1.3,^ 3.4, 3. IP: PROGRAM DEVELOPMENT (IV-C) IP-DEV^ 3.1.1,3.3.3.^ 3.3. EXTERNAL CONTEXT EC (PRE) (DUR) 3.2. 3.3, 3. EC: DEMOGRAPHICS In county, school pcrsonncl Out county, nonschool pcrsonncl EC: ENDORSEMENT In county, school pcrsonncl Out county. nonschool pcrsonncl EC: CLIMATE In county, school pcrsonncl Out county, nonschool pcrsonncl
INTERNAL CONTEXT IC: CHARACTERISTICS IC: NORMS AND AUTHORITY IC: INNOVATION HISTORY IC: ORGANIZATION PROCEDURES IC: INNOVATION-ORGANIZATION CONGRUENCI ADOPTION PROCESS AP: EVENT CHRONOLOGYO1:I:ICIAL VERSION AP: EVENT CHRONOLOGY - SUBTERRANEAN AP: INSIDE/OUTSIDI< AP: CENTRALITY AP: MOTIVES AP. USER FIT AP: PLAN AP: READINESS AP: CRITICAL EVENTS
SITE DYNAMICS AND TRANST,'ORMATIONS
EC-DEM ECCO-DEM 3.2.3, 3.3, 3. ECEXT-DEM 3.2.3, 3.3, 3. EC-END 3.2.3, 3.3, 3. ECCO-END 3.2.3. 3.3, 3. ECEXT-END 3.2.3, 3.3, 3. EC-CLIM 3.2.3. 3.3, 3. ECCO-CLI M 3.2.3, 3.3, 3. I<CEXT-CLIM 3.2.3, 3.3. 3.
IC (PRE) (DUR) 3.2, 3.3, 3. IC-CHAR 3.2.2, 3.4. 3. IC-NORM 3.2.2, 3.4.3, 3. IC-HIST 3.2. IC-PROC 3.1.1, 3.24,3.3,3. IC-FIT 3.2.
AP-CI-NT 3.2. AP-MOT 3.2. AP-FIT 3.2. AP-PLAN 3.3. AP-RED1 3.3.4, 3.2. AP-CRIT 3.3.
TR: EVENT CHRONOLOGY-0TT:ICIAL VERSION TR: EVENT CHRONOLOGY - SUBTERRANEAN TR: INITIAL USER EXPERIENCIC TR: CHANGES IN INNOVATION TR: EF17F:CTSON ORGANIZATIONAL PRACTICES T R : EFFECTS ON ORGANIZATIONAL CLIMATE TR: EFFECTS ON CLASSROOM PRACTICE TR: EFFECTS ON USER CONSTRUCTS TR: IMPLEMENTATION PROBLEMS T R : CRITICAL EVENTS TR: EXTERNAL INTERVENTIONS TR: EXPLANATIONS FOR TRANSFORMATIONS T R : PROGRAM PROBLEM SOLVING
TR-CHRON/PUB TR-CHRON/PRIV TR-START TR-INMOD TR-ORG/PRAC TR-ORG/CLIM TR-CLASS TR-HEAD TR-PROBS TR-CRIT TR-EXT TR-SIZUP TR-PLAN
Analysis During Data Collection 59
Chart 6 a (Continued)
- -^ NEW CONFIGURATION AND ULTIMATE OUTCOMES NCO: STABILIZATION OF INNOVATION-CLASSROOM NCO: STABILIZATION 01; USER BEHAVIOR NC0:USER FIRST-LEVEL OUTCOMES Positive and nega tive Anticipated and unanticipated Combinations (when appropriate)
NCO: USER META OUTCOMES Pos~tivrand negative Anticipated and unanticipated Combinations (when appropriate)
NCO: USER SPIN0FI:S AND SlDE EFFECTS Positive and negative Anticipated and unanticipated Combinations (when appropriate)
NCO: CLASSROOM INSTITUTIONALIZATION NCO: STABILIZATION O F INNOVATION-ORGANIZATION NCO: STABILIZATION 01; ORGANIZATIONAL BEHAVIOR NCO: ORGANIZATIONAL INSTiTUTlONALlZATlON NCO: ORGANIZATIONAL 1:IRST-LEVEL OUTCOMES Positive and negative Anticipated and unanticipated Combinations (whcn appropriate)
NCO: OI<GANIZATIONAL META OUTCOMES Positivc and ncgatlvc Anticipated and unanticipatcd Combinations (when appropriate)
NCO: ORGANIZATIONAL SPINOI+'S AND SIDI.. EI.I:I:CTS Positive and negative Anticipatcd and unanticipatcd Co~iibinat~ons(when appropriate)
NCO: INSTITUTIONAL EXPANSION NCO: ORGANIZATIONAL RLCDUCTION
NCO NCDINNOSTABICLASS NCO-STABIUSER NCO-USER I OC
NCOUSER I OC/+, -
NCO-USER I OCIA, U NCO-USER I OC/A+. A-
U+. u-
NCO-USER META
NCO-USER META OC/+, -
NCO-USER META OCJA, U NCO-USER META OC/A+, A- U+, U- NCGUSER SlDE NCO-USER SlDE OC/+. - NCO-USER SlDE OC/A, U NCO-USER SlDE OC/A+, A- U+. U- NCO-INST/CLASS NCQ-INNOSTAB/ORG NCO-STARIORG NCO-INSTIORG NCO-ORG I OC NCO-ORG I OC/+, -
NCO-ORG IOCIA. u
NCO-ORG IOC/A+ A- Llt. IJ- NCO-ORG META
NCO-ORG META OC/+. -
NCO-ORG META OC/A, LI NCOORG META OC/A+. A- U+, U- NCO-ORG SIDI: NCO-ORG SlDE OC/+, - NCO-ORG SlDE OC/A. U NCO-ORG SIDE OC/A+. A- U+, U- NCO-INNOGRO/ORC NCO-INNODWIN/ORG
EXTERNAL AND INTIRNAL ASSISTANCE (SI-PARATI: CODIS FOR EXTERNAL, PEER. ADMINISTRATIVE) ASS: LOCATION ASS-LOC 3.6. ASS: RULES. NORMS ASS-RULE 3.6. ASS: ORIENTATION ASS-OR1 3.6. ASS: TYPE ASS-TY PE 3.6. ASS: EFFECTS ASS: ASSF.SSMENT BY RECIPIENTS ASS: LINKAGIC
ASS-EI'I' ASS-ASS ASS-LINK EMIiRGING CAUSAL LINKS CL CL: NETWORKS CL: RULES CL: RECURRENT PATTERNS Wilhin site Intersite CL: EXPLANATORY CLUSTER (researcher) (respondent) QUERIES - OU: SURPRISES OU: PUZZLES
CL-NET CL-RULI: CL-PATT CL-PATTI LS CL-PATTJOS CL-EXPL SITECL-EXPL --^ QU OU-! OU-O
N.A. N.A. N.A. N.A. N.A. N.A. N.A.
N.A. N.A.
Analysis During Data Collection 67
Chart 7 illustration o f a Poorly Structured List o f Codes (excerpt)
Actors PlanninR/Implementation Processes Aspects of New School
administrator advisory group architect Board (central) Board (district) builder chairperson citizen 132 community liaison
... principal (focal) principal (other) researcher (other) researcher (SA) salesman specialist (central off.) specialist (school) supt. (central) supt. (district) student teacher teaching team union representative voluntary organization
Informal* 151 buffer 106 core group 107 core member 152 evaluator 113 implementer 15 3 leader (socio-emotional) 154 leader (task) 155 linker 156 mass media 157 opinion leader 117 planner
commitment complexity management conflict management constituency development cooptation decision-making designing energy depletion energy mobilization energy overinvestment t‘uture-envisioning goal clarification (stud. outcome) goal clarification (sys. prop.) goal clarification (benefits) goal succession group-building
planning planning horizon planninglimplementation linkage planning model policy-making power base-building power struggle recruitment redesign reflexivity rehearsal research relationship resource acquisition resource allocation resource identification role accumulation role confusion role strain start-up task behavior thoroughness time investment timetabling training uncertainty management variety pool expansion
AFTER CODING, LOOK AT PROCESSES AND ASPECTS LISTS, AND PUT (*) BY THE MOST IMPORTANT KEY WORDS (MAXIMUM = 6).
boundary maintenance budget (district) budget (school) collective sentiments community control communication (formal) communication (informal) conflict management curriculum data collection/fecdback discipline departmentalization equipment evaluation (of school) cxtracurricular activities food service friendship grouping goals (student outcomes) goals (system properties) goals (benefits) governan ce group definitions influence (informal) in ter-organizational linkage
role definition rules salaries school-community linkage security space use staff assignments staff charactcristics staff selection staff utilization status/prestige student charactcristics student grade lcvels student gradinglevaluation student grouping basis student population student personnel services student recruitment student selection supplies teaching methods technology (non-stud. oriented) time use transportation zoning
*For actors actually present at the contact, circle the number. If an actor was not present, but is discussed in the contact, put parentheses around the key word.
62 Qualitative Data Analysis
Chart 6b Definitions of Selected Codes from Chart 6a
Site Dynamics and Transformations- TR
Event chronology - (^) Event chronology during initial and ongoing official version: implementation, as recounted by users, ad- TR-CHRONIPUB ministrators o r other respondents.
Event chronology- Event chronology during initial or ongoing subterranean version: implementation, as recounted by users, ad- TR-CHRON/PRIV ministrators or other respondents, and suggesting (a) a consensual but different scenario than the public version or (b) vary- ing accounts of the same events.
Initial user experience: Emotions, events, problems o r concerns, TR-START assessments, made by teachers and adminis- trators during f i s t six months of implemen- tation.
Changes in innovation: Reported modifications in components o f TR-INMOD the new practice or program, o n the part of teachers and administrators, during initial and ongoing implementation.
Effects o n organi- lndices of impact of new practice or pro- zational practices: (^) gram o n : (a) intraorganizational planning, TR-ORC/PRAC monitoring, and daily working arrangements (e.g., staffing, scheduling, use o f resources, communication among staff) and (b) inter- organizational practices (e.g., relationships with district office, school board, commu- nity, and parent groups).
Effects o n organi- lndices of impact of new practice or pro- zational climate: gram o n institutional norms and interper- TR-ORC/CL~M sonal relationships, including effects o n power and influence, social networks, institutional priorities for investing time and energy.
Effects o n classroom lndices of impact of new practice or practice: program on regular or routine classroom TR-CLASS practices (instructional planning and management).
Effects on user Indices of effects of new practice or pro- constructs: gram o n teacher and administrator percep- TR-HEAD tions, attitudes, motives, assumptions or theories of instruction, learning, or manage- ment (e.g., professional self-image, revised notions of what determines achievement or efficiency, other attitudes toward pupils, colleagues, other staff members, stance toward other innovative practices).
Implementation problems: TR-PROBS
Difficulties or concerns relating t o imple- mentation a t the personal, classroom, organizational, or extraorganizational levels, including reasons given for presence of difficulty or concern.
64 Qualitative Data Analysis
researcher to get sloppy, resentful, tired, and partial, thus damaging data quality. O n e simple rule of thumb is this: Always code the previous set of field notes before the next trip to the site. If the researcher is several weeks at the site, dictated notes should be sent home regularly for transcription and coding. This plan can be foiled by coding backlogs and slow turnaround in typing up transcripts, but ongoing coding is the right plan. Coding field notes while you are still in the data collection process uncovers real or potential sources of bias and, overall, sets the agenda for the next wave of data collection. Incomplete o r equivocal data can get resolution next time out. The more important point is that, since the ultimate power of field research lies in the researcher's emerging map of what is happening and what the strongest determinants appear to be, any device that will force more differentiation and integration of that map is a good investment. Advice Codes are efficient data-labeling and data-retrieval devices. They empower and speed up analysis. T o generate and use them t o their best advantage, we have offered a number of tips. Creating codes prior to fieldwork is helpful; it forces the analyst t o tie research questions or conceptual interests directly t o the data. But the analyst should be ready t o bend the codes when they look inapplicable, overbuilt, empirically ill-fitting, or overly abstract. O n e can also work more inductively, waiting for the field notes to suggest more empirically driven labels. O n e should not, however, wait too long or change the codes too often. Make certain that all the codes fit into a structure, that they relate to or are distinct from others in meaningful, study-important ways. Don't casually add, remove, or reconfigure codes. Keep the codes semantically close t o the terms they represent. Don't use numbers as codes. Have the codes on a single sheet for easy reference. Define the codes operationally and be sure all analysts understand the definitions and are able to identify rapidly a segment fitting the definition. Ordinarily, use a single code for a segment. Dual or even multiple coding is warranted if a segment is both descriptively and inferentially meaningful. Double-coding the same transcripts is essential for studies with more than o n e researcher and very useful for the lone researcher (get code-recode consistencies over 9 0 percent before going on). Coding should not be put off to the end of data gathering. Qualitative research depends heavily on
ongoing analysis, and coding is a good device for forcing that analysis. Time Required The time required for generating initial codes and definitions depends, of course, on how many you start with, and on the clarity of the conceptual framework and research questions. Forthe start list and definitions shown earlier, the first cut took two full days-one for the codes and o n e for the definitions. Revisions and completions added another two days. Coding itself takes varying amounts of time, depen- ding on the code'sconceptual structure and complexity, the quality of the field notes, and the coder'sskill. Here is some typical arithmetic from our experience. A single-spaced page of transcribed field notes has about 4 5 lines. As a rough rule of thumb, it might contain 8 - 1 0 codes. A 2-3-day field trip usually generates 4 0 - 8 0 such pages, even when the researcher is disciplined. Coding each page runs about 5 - 1 0 minutes once the codes are committed to memory; in the beginning, o n e should count 1 0 - 1 5 minutes. S o "inexperienced" coding of such a data set would take perhaps 2 days; later, it could be done in a day o r so. Taking longer than this is a signal that there are too fine units of analysis, too much dual coding, too many codes, or a weak conceptual structure. Coding is tiring. I f often feels "longer than it really is." It helps very much to use written-in marginal remarks (see Box 1I.B.b) of an active, musing sort- rather than just dully plowing through the application of codes. Breaking from time to time to do other related work-such as arguing with other coders, writing memos (see section III.D), or jotting down notes about what t o look for in the next site visit- also helps.
1II.B.a Reflective Remarks Raw field notes (the scribbles and jottings that enter your notebook as you are watching a situation or talking with someone) must, as we have pointed out, be converted into a write-up, a transcription that is legible to any reader. When doing a write-up, whether by typing or dicta- tion, the temptation is t o plod along, converting raw notes into a coherent account. But that misses an important resource: the fieldworker's reflections and commentary on issues that emerge duringthe process. As a write-up is being produced, reflections of several sorts typically swim into awareness. For example: what the relationship with the respondents was like
Analysis During Data Collection 65
Box 1II.B.a Reflective Remarks: Illustrations
Mike joked, 'haybe I c o u l d go and a c t l i k e a s e n i o r. " H e made a dumb monkey f a c e a s h e was s p e a k i n g. ( ( T h i s s t a f f does n o t seem t o p u t down s t u d e n t s , l e a l l y , b u t t h e y
can n o t resist o c c a s i o n a l jokes of t h i s s o r t - more of t h i s
l a t e r. ) ) .... J i m i n d i c a t e d t h a t t h e y had u n o f f i c i a l l y done t h e i r own a n a l y s i s of a t t e n d a n c e d a t a and s a i d , " I ' m s u r e i t ' s been e f f e c t i v e " , ( t h a t is,CARED i n i n c r e a s i n g a t t e n d a n c e r a t e s ). ( ( T h i s s o u n d e d p r e t t y s o f t and vague t o m e. ) ) ....' John w e n t on t o e x p l a i n t h a t d u r i n g t h e second s e m e s t e r h e would be doing p r e t t y much t h e same, t h a t i s , " n o t h i n g much a t a l l. " ( ( T h i s d e n i a l of h i s a c t i v i t y was l a t e r picked up i n f o r m a l l y by J i m , I t h i n k , i n a c o n v e r s a t i o n w i t h m e. I t was, i n f a c t , a s o r t of m i n i m i z a t i o n , and perhaps a d e f l e c t i o n from t h e f a c t t h a t he was away a g r e a t d e a l of t h e t i m e , when presumably, he m i g h t be a b l e t o be h e l p f u l w i t h i s s u e s i n t h e program i t s e l f. ) )
second thoughts on the meaning of what a respon- dent was saying doubts about the quality of data being recorded a new hypothesis explaining what was happening a mental note to pursue an issue further in the next contact cross-allusions to something in another part of the data - own feelings about what was being said or done elaboration or clarification of something said or done
When something like this arises in your mind, it is useful t o enter it directly into t h e write-up. A good convention is t o mark off t h e remark with double parentheses, t o signal that it is of a different order than the data it comments on. T h e material in Box 1II.B.a gives s o m e examples. Remarks such a s these a d d sub- stantial meaning t o t h e write-up for other readers. A n d they usually are a n aid during coding, because they often point t o deeper o r underlying issues that deserve analytic attention. Note: T h e ((reflective remark)) technique can also b e used while you are jotting down raw field notes a s well. Doing so, in fact, improves the usefulness of field notes considerably; o n e is simultaneously aware of events in t h e site, a n d of one's own feelings, reactions, insights, and interpretations, a s Patton (1980) sug- gests. S e e Bogdan a n d Biklen (1982), who divided reflective remarks into those o n analysis, method,
ethical dilemmas, own frame of mind, a n d points of calibration.
1II.B.b Marginal Remarks Coding, as we have noted, can get boring a n d tedious, if o n e treats oneself as a sort of machine picking o u t chunks of data a n d assigning category labels t o them. T h e sensation of being bored is usually a signal that you have ceased t o think. O n e way of retaining a thoughtful stance t o coding is t h e marginal remark. It is analogous t o the reflective remark (see Box 1II.B.a). As coding proceeds, if you a r e being alert and non- routine about what you are doing, ideas a n d reactions t o t h e meaning of what you are seeing will well u p steadily. These ideas are important: They suggest new interpretations, leads, connections with other parts of the data-and they usually point toward analytic work, like the pattern codes discussed in t h e next section, and t h e memoing (Section 1II.D)that leads further a n d further into analysis. Assuming that your convention is that codes a p p e a r in the left margin, it is useful t o put preanalytic remarks of all sorts in the right margin. Box 1II.B.b presents a couple of examples. Marginal remarks, like reflective remarks, a d d meaning a n d clarity t o field notes. They
Analysis During Data Collection 67
"Analytic files" contain cut-up chunks of field notes. Each analytic file contains material on some major issue, theme, code, or family of codes. New analytic files get generated as t h e fieldwork proceeds, often through "memos" (see Section 1II.D).Material in o n e
file may be cross-referenced t o another. A loose-leaf
notebook can also be used. Bogdan and Biklen (1982) also suggest folders organized by single codes, and note that analysis usefully involves clumping/rearranging/connecting data chunks, perhaps o n a thumbtack board. Fora fas- cinating survey of procedures for field note storage and retrieval, see also Bolton (1982). Cards and file folders are reasonably workable if the number of sites is small and the data collection not extended. But they are increasingly difficult and very time-consuming as the data base gets larger. The obvious way t o store and retrieve text quickly and easily is to use a computer. As Werner (1982) has noted, it's fully practical t o use a microcomputer in the field t o write up and code field notes directly. For various approaches using computers for storage and retrieval, see the thoughtful collection of articles edited by Conrad and Reinharz (1984), Patton (1980, pp. 301-302), Dow (1982), and Sproull and Sproull (1982), whose TEXTAN program also permits easy analysis of text line by line. For a good discussion of o n e large-scale application of computerized storage and retrieval, see Stern (1977) and Yates (1977). The main things to avoid in developing a comput- erized approach are (a) elaborateness-for example, assigning each chunk complex and multiple codes, just because retrieval is so easy, forgetting that this chews u p large amounts of coding time, and (b) atomism and context-stripping. We have seen, for example, a program that produced something like this when asked t o retrieve lines with the word "principal" in them, ordering them by position of the word: 622PRlNClPALOFTHE SCHOOLISJOHN NEUMANN. HE IS N O T 6 7 3 PRINCIPAL REMAINED QUIET, B U T I WAS UNSURE A S T O H I S 501 T H E PRINCIPAL IS WIDELY SEEN A S SUP- PORTIVE O F THE EFFORT BUT 9 9 8 T H E PRINCIPAL S T A Y S IN THE OFFICE, NEVER SEEMING T O 4 4 3 T A S K G R O U P A G R E E D T H A T PRINCIPAL WOULD FOLLOW 9 9 9 AND IN ANY CASE THOMPSON A S PRINCIPAL WOULD BE
T h e net result was that one could never understand the semantic, let alone the event context in which the retrieved word appeared.
The rapid diffusion of microcomputers and asso- ciated word-processing software is making for decided advances in text storage and retrieval (not to mention analysis) capabilities; anyone planning a field study should take a serious look at what's available before settling for cards and file folders. Indexing. As Levine (1982)explains,"indexing" is a generic term including three processes: (a) defining clear categories (codes); (b) organization of these into a structure using an "index language"; and (c) pairing of the codes with appropriate places in the data base. As such, "indexing" is the heart of storage and retrieval; a strong, well-organized indexing sytem takes a good deal of energy t o set up, but is crucial for data reduction, display, and conclusion drawing. To avoid semantic confusion, let's focus for a minute on tables of contents, also called "indexes." Here we mean only a list of places where specific data chunks can be found. Such a list is often a useful part of the overall "indexing" system. For example, file cards can be prepared, one for each code. Each card has a notation on it for each instance of acoded chunk in the field notes, giving the page number and line number (this requires use of prenumbered paper for typed-up notes).Retrieval is slow this way; its best use is forsmall data bases. Some researchers add an index to the front of each meaningful block of field notes (Dobbert, 1982). Tables of contents can of course easily be produced through available microcomputer programs (for
example, GETSTUD.BAS; W. A. Firestone, personal
communication, 1983), helping enormously in rapid access to a large body of field notes.
1II.C PATTERN CODING Analysis Problem Given a working set of reasonably clear codes that describe the phenomena and events that are depicted in transcribed field notes, how can the researcher move t o a second, more general, perhaps more explanatory level? Just naming or classifying what is out there is usually not enough. We need t o under- stand the patterns, t h e recurrences, the whys. As Kaplan (1964) remarks, the bedrock of inquiry is the researcher's quest for "repeatable regularities." Brief Description Pattern codes are explanatory or inferential codes, ones that identify an emergent theme, pattern, or explanation that the site suggests t o the analyst. They act to pull a lot of material together into more
68 Qualitative Data Analysis
meaningful and parsimonious units of analysis. They are a sort of meta-code. First-level coding is a device for summarizing segments of data. Pattern coding is a way of grouping those summaries into a smaller number of overarching themes or constructs. It is, for qualitative researchers, an analogue to the cluster-analytic and factor-analytic devices used in statistical analysis. The quantitative researcher works with sets of variables that either put p e o p l e into distinct families built around what they d o o r say (Q analysis) or, alternatively, cluster such a c t i o n s a n d p e r c e p t i o n s across informants (R a n a l y ~ i s ). ~ For the qualitative analyst, pattern coding has four important functions: (1) It reduces large amounts of data into a smaller number of analytic units. (3) it gets t h e researcher into analysis during data collection, s o that later d a t a collection can b e more focused. (3) It helps t h e researcher build a cognitive map, a n evolving schema for understanding what is hap- pening locally. (4) When several researchers are engaged in individual case study work, it lays thegroundworkforcross-site analysis by surfacing common themes a n d causal processes. l l l u s t r a t i o n s These four functions can be clarified a s we discuss how pattern codes are generated, what they look like, and what the field researcher does with them in the course of data collection. G e n e r a t i n g p a t t e r n c o d e s. This is easy-sometimes too easy. As in everyday life, the researcher needs t o reduce and channel the stimuli with which he or she is being bombarded into asmallernumber of chunks that can be mentally encoded, stored, and readily retrieved. Already during the initial fieldwork, the researcher is looking for threads that tie bits of data together. For example, if two or three informants say independently that they resent a decision made by their boss, we may be on to several different phenomena-a conflict, an organizational climate factor, or a disgruntled sub- group of employees. Any of these interpretations involves sorting and chunking data (function 1, above). These first bits of data are also leads; they suggest t o the researcher what may be important variables to check out, factors that may account for other local perceptions and behaviors (function 2, above). Seeing the "resentment" data in any of these alternative ways also helps the researcher make sense of observations that had u p now been puzzling o r sur- prising. These several bits come together into an initial plot of the terrain t o be gone over in progressively greater detail (function 3). Finally, if another field
researcher in a multisite study comes across a similar batch of resentment or, alternatively, finds no resent- ment of decisions at all in a place otherwise similar to the more "resentful" site, we have the first threads of cross-site comparisons (function 4). The danger is that of getting locked too quickly into naming a pattern and assuming you understand it, then thrusting the name onto data that fit it only poorly. Premature analytic closure is hard t o shake, in part because the analyst often isn't aware of what is happening (a second analyst, reading over the field notes, usually is, however). Patterning happens fast because it is the way we habitually process infor- m a t i ~ n. ~The trick here is t o work with loosely held chunks of meaning, t o be ready t o unfreeze and recon- figure them as the data shape up otherwise, t o subject the best patterns t o merciless cross-checking, and t o lay the most tenuous ones aside until other informants and observations give them more persuasive empirical grounding. W h a t p a t t e r n c o d e s l o o k like. Pattern codes usually turn around four, often interrelated, summarizers: themes, causes/explanations, relationships among people, and more-theoretical constructs. Here are some examples from a recent study, with codes we assigned in capital letters. Themes: PATT (pattern):All supervisors seem t o be using benevo- lent, fatherly terms when talkjng about employees ("my" staff,-"my" people, "my" junior guys), but employees use mostly bureaucratic, regulation-type terms ("the office," "upstairs," "the management"). RULE: You don't talk earnestly about your problems or your successes in t h e staff lounge. PATT/OS (theme appearing in other sites a s well a s this one): It seems easier t o get new projects adopted among lower-class students o r in vocational tracks. Causes/Explanations: EXPL multiple role of t h e "helping teacher" seems t o b e an important ingredient o f success. SITE-EXPL (informants' explanations): T h e best projects are o n e s that put together t h e best practitioner's recipes. MET (metaphor): T h e idea of career "trajectoriesw- people a r e using these projects t o get away from s o m e jobs a n d places to other ones. Relationships Among People: NET (social network): T h e money-and-support club: A. Becker, P. Harrison, V. Wales. Theoretical Constructs: B S P (basic social processes, a s in Glaser, 1978): Nego- tiating o r bargaining s e e m s t o b e t h e way decisions get made; a conflict model is more plausible than a rational-technological model. U s i n g p a t t e r n c o d e s in a n a l y s i s. There are at least three ways t o use pattern codes. First, they are added in tentative form t o the list of codes. and tried out on
