National Reading Panel, Lecture notes of Technology

Download National Reading Panel and more Lecture notes Technology in PDF only on Docsity!

Acknowledgments

The National Reading Panel wishes to express its gratitude to the following individuals for their contributions to its effort.

Marilyn Adams Ed Bouchard Harris Cooper Gerald Duffy Michelle Eidlitz Barbara Foorman David Francis Ester Halberstam Blair Johnson Alisa Kenny Helen S. Kim Marjolaine Limbos Khalil Nourani Simone Nunes Elizabeth S. Pang Joan Pagnucco Michael Pressley David Reinking Scott J. Ross Barbara Schuster Robin Sidhu Steven Stahl Maggie Toplak Zoreh Yaghoubzadeh

Reports of the Subgroups ii

Table of Contents

Acknowledgments .................................................................................................................. ii

Members of the National Reading Panel .......................................................................... iii

Chapter 1: Introduction and Methodology

Introduction .......................................................................................................................... 1-

Methodology: Processes Applied to the Selection, Review, and

Analysis of Research Relevant to Reading Instruction ...................................................... 1-

Chapter 2: Alphabetics

Part I: Phonemic Awareness Instruction

Executive Summary ....................................................................................................... 2-

Report

Part II: Phonics Instruction

Appendices ................................................................................................................... 2-

Executive Summary ....................................................................................................... 2-

Report ........................................................................................................................... 2-

Appendices ................................................................................................................... 2-

Chapter 3: Fluency

Executive Summary ............................................................................................................. 3-

Report .................................................................................................................................. 3-

Appendices .......................................................................................................................... 3-

Chapter 4: Comprehension

Executive Summary ............................................................................................................. 4-

Introduction .......................................................................................................................... 4-

Part I. Vocabulary Instruction

Report ........................................................................................................................... 4-

Appendices ................................................................................................................... 4-

Part II: Text Comprehension Instruction

Report ........................................................................................................................... 4-

Appendices ................................................................................................................... 4-

Part III. Teacher Preparation and Comprehension Strategies Instruction

Report ........................................................................................................................... 4-

Appendices ................................................................................................................... 4-

v National Reading Panel

Chapter 5: Teacher Education and Reading Instruction

Executive Summary ............................................................................................................. 5-

Report .............................................................................................................................................. 5- Appendices ...................................................................................................................................... 5-

Chapter 6: Computer Technology and Reading Instruction

Executive Summary............................................................................................................... 6-

Report .................................................................................................................................. 6-

Appendices .......................................................................................................................... 6-

Minority View

Reports of the Subgroups vi

Introduction

REPORTS OF THE SUBGROUPS

Introduction

Congressional Charge

In 1997, Congress asked the “Director of the National Institute of Child Health and Human Development (NICHD), in consultation with the Secretary of Education, to convene a national panel to assess the status of research-based knowledge, including the effectiveness of various approaches to teaching children to read.” The panel was charged with providing a report that “should present the panel’s conclusions, an indication of the readiness for application in the classroom of the results of this research, and, if appropriate, a strategy for rapidly disseminating this information to facilitate effective reading instruction in the schools. If found warranted, the panel should also recommend a plan for additional research regarding early reading development and instruction.”

Establishment of

the National Reading Panel

In response to this Congressional request, the Director of NICHD, in consultation with the Secretary of Education, constituted and charged a National Reading Panel (the NRP or the Panel). The NRP was composed of 14 individuals, including (as specified by Congress) “leading scientists in reading research, representatives of colleges of education, reading teachers, educational administrators, and parents.” The original charge to the NRP asked that a final report be submitted by November 1998.

When the Panel began its work, it quickly became apparent that the Panel could not respond properly to its charge within that time constraint. Permission was sought and received to postpone the report’s submission deadline. A progress report was submitted to the Congress in February 1999. The information provided in the NRP Progress Report, the Report of the National Reading Panel, and this Report of the National Reading Panel: Reports of the Subgroups reflects the findings and determinations of the National Reading Panel.

NRP Approach to Achieving the Objectives of Its Charge and Initial Topic Selection

The charge to the NRP took into account the foundational work of the National Research Council (NRC) Committee on Preventing Reading Difficulties in Young Children (Snow, Burns, & Griffin, 1998). The NRC report is a consensus document based on the best judgments of a diverse group of experts in reading research and reading instruction. The NRC Committee identified and summarized research literature relevant to the critical skills, environments, and early developmental interactions that are instrumental in the acquisition of beginning reading skills. The NRC Committee did not specifically address “how” critical reading skills are most effectively taught and what instructional methods, materials, and approaches are most beneficial for students of varying abilities.

In order to build upon and expand the work of the NRC Committee, the NRP first developed an objective research review methodology. The Panel then applied this methodology to undertake comprehensive, formal, evidence-based analyses of the experimental and quasi- experimental research literature relevant to a set of selected topics judged to be of central importance in teaching children to read. An examination of a variety of public databases by Panel staff revealed that approximately 100,000 research studies on reading have been published since 1966, with perhaps another 15, appearing before that time. Obviously, it was not possible for a panel of volunteers to examine critically this entire body of research literature. Selection of prioritized topics was necessitated by the large amount of published reading research literature relevant to the Panel’s charge to determine the effectiveness of reading instructional methods and approaches. A screening process was, therefore, essential.

The Panel’s initial screening task involved selection of the set of topics to be addressed. Recognizing that this selection would require the use of informed judgment, the Panel chose to begin its work by broadening its

1-1 National Reading Panel

Chapter 1: Introduction and Methodology

understanding of reading issues through a thorough analysis of the findings of the NRC report, Preventing Reading Difficulties in Young Children (Snow, Burns, & Griffin, 1998). Early in its deliberations the Panel made a tentative decision to establish subgroups of its members and to assign to each subgroup one of the major topic areas designated by the NRC Committee as central to learning to read—Alphabetics, Fluency, and Comprehension.

Regional Public Hearings

As part of its information gathering, the Panel publicly announced, planned, and held regional hearings in Chicago, IL (May 29,1998), Portland, OR (June 5, 1998), Houston, TX (June 8, 1998), New York, NY (June 23, 1998), and Jackson, MS (July 9, 1998). The Panel believed that it would not have been possible to accomplish the mandate of Congress without first hearing directly from consumers of this information— teachers, parents, students, and policymakers—about their needs and their understanding of the research. Although the regional hearings were not intended as a substitute for scientific research, the hearings gave the Panel an opportunity to listen to the voices of those who will need to consider implementation of the Panel’s findings and determinations. The regional hearings gave members a clearer understanding of the issues important to the public.

As a result of these hearings, the Panel received oral and written testimony from approximately 125 individuals or organizations representing citizens—teachers, parents, students, university faculty, educational policy experts, and scientists—who would be the ultimate users and beneficiaries of the research-derived findings and determinations of the Panel.

At the regional hearings, several key themes were expressed repeatedly:

• The importance of the role of parents and other concerned individuals, especially in providing children with early language and literacy experiences that foster reading development;
• The importance of early identification and intervention for all children at risk for reading failure; - The importance of phonemic awareness, phonics, and good literature in reading instruction, and the need to develop a clear understanding of how best to integrate different reading approaches to enhance the effectiveness of instruction for all students; - The need for clear, objective, and scientifically based information on the effectiveness of different types of reading instruction and the need to have such research inform policy and practice; - The importance of applying the highest standards of scientific evidence to the research review process so that conclusions and determinations are based on findings obtained from experimental studies characterized by methodological rigor with demonstrated reliability, validity, replicability, and applicability; - The importance of the role of teachers, their professional development, and their interactions and collaborations with researchers, which should be recognized and encouraged; and - The importance of widely disseminating the information that is developed by the Panel.

Adoption of Topics To Be Studied

Following the regional hearings, the Panel considered, discussed, and debated several dozen possible topic areas and then settled on the following topics for intensive study:

• Alphabetics
  • Phonemic Awareness Instruction
  • Phonics Instruction
• Fluency
• Comprehension
  • Vocabulary Instruction
  • Text Comprehension Instruction
  • Teacher Preparation and Comprehension Strategies Instruction
• Teacher Education and Reading Instruction
• Computer Technology and Reading Instruction

Reports of the Subgroups 1-

Methodology

R E P O R T S O F T H E S U B G R O U P S

Methodology: Processes Applied to the Selection, Review, and Analysis of Research Relevant to Reading Instruction

In an important action critical to its Congressional charge, the NRP elected to develop and adopt a set of rigorous research methodological standards. These standards, which are defined in this section, guided the screening of the research literature relevant to each topic area addressed by the Panel. This screening process identified a final set of experimental or quasi- experimental research studies that were then subjected to detailed analysis. The evidence-based methodological standards adopted by the Panel are essentially those normally used in research studies of the efficacy of interventions in psychological and medical research. These include behaviorally based interventions, medications or medical procedures proposed for use in the fostering of robust health and psychological development and the prevention or treatment of disease.

It is the view of the Panel that the efficacy of materials and methodologies used in the teaching of reading and in the prevention or treatment of reading disabilities should be tested no less rigorously. However, such standards have not been universally accepted or used in reading education research. Unfortunately, only a small fraction of the total reading research literature met the Panel’s standards for use in the topic analyses.

With this as background, the Panel understood that criteria had to be developed as it considered which research studies would be eligible for assessment. There were two reasons for determining such guidelines or rules a priori. First, the use of common search, selection, analysis, and reporting procedures would ensure that the Panel’s efforts could proceed, not as a diverse collection of independent—and possibly uneven—synthesis papers, but as parts of a greater whole. The use of common procedures permitted a more unified presentation of the combined methods and findings. Second, the amount of research synthesis that had to be accomplished was substantial. Consequently, the Panel had to work in diverse subgroups to identify,

screen, and evaluate the relevant research to complete their respective reports. Moreover, the Panel also had to arrive at findings that all or nearly all of the members of the NRP could endorse. Common procedures, grounded in scientific principles, helped the Panel to reach final agreements.

Search Procedures

Each subgroup conducted a search of the literature using common procedures, describing in detail the basis and rationale for its topical term selections, the strategies employed for combining terms or delimiting searches, and the search procedures used for each topical area.

Each subgroup limited the period of time covered by its searches on the basis of relative recentness and how much literature the search generated. For example, in some cases it was decided to limit the years searched to the number of most recent years that would identify between 300 to 400 potential sources. This scope could be expanded in later iterations if it appeared that the nature of the research had changed qualitatively over time, if the proportion of useable research identified was small (e.g., less than 25%), or if the search simply represented too limited a proportion of the total set of identifiable studies. Although the number of years searched varied among subgroup topics, decisions regarding the number of years to be searched were made in accord with shared criteria.

The initial criteria were established to focus the efforts of the Panel. First, any study selected had to focus directly on children’s reading development from preschool through grade 12. Second, the study had to be published in English in a refereed journal. At a minimum, each subgroup searched both PsycINFO and ERIC databases for studies meeting these initial criteria. Subgroups could, and did, use additional databases when appropriate. Although the use of a

1-5 National Reading Panel

Chapter 1: Introduction and Methodology

minimum of two databases identified duplicate literature, it also afforded the opportunity to expand perspective and locate articles that would not be identifiable through a single database.

Identification of each study selected was documented for the record and each was assigned to one or more members of the subgroup who examined the title and abstract. Based on this examination, the subgroup member(s) determined, if possible at this stage, whether the study addressed issues within the purview of the research questions being investigated. If it did not, the study was excluded and the reason(s) for the exclusion were detailed and documented for the record. If, however, it did address reading instructional issues relevant to the Panel’s selected topic areas, the study underwent further examination.

Following initial examination, if the study had not been excluded in accord with the preceding criteria, the full study report was located and examined in detail to determine whether the following criteria were met:

• Study participants must be carefully described (age; demographics; cognitive, academic, and behavioral characteristics).
• Study interventions must be described in sufficient detail to allow for replicability, including how long the interventions lasted and how long the effects lasted.
• Study methods must allow judgments about how instruction fidelity was ensured.
• Studies must include a full description of outcome measures.

These criteria for evaluating research literature are widely accepted by scientists in disciplines involved in medical, behavioral, and social research. The application of these criteria increased the probability that objective, rigorous standards were used and that therefore the information obtained from the studies would contribute to the validity of any conclusions drawn.

If a study did not meet these criteria or could not be located, it was excluded from subgroup analysis and the reason(s) for its exclusion was detailed and documented for the record. If the study was located and met the criteria, the study became one of the subgroup’s core

working set of studies. The core working sets of studies gathered by the subgroups were then coded as described below and then analyzed to address the questions posed in the introduction and in the charge to the Panel.

If a core set of studies identified by the subgroup was insufficient to answer critical instructional questions, less recent studies were screened for eligibility for, and inclusion in, the core working sets of studies. This second search used the reference lists of all core studies and known literature reviews. This process identified cited studies that could meet the Panel’s methodological criteria for inclusion in the subgroups’ core working sets of studies. Any second search was described in detail and applied precisely the same search, selection, exclusion, and inclusion criteria and documentation requirements as were applied in the subgroups’ initial searches.

Manual searches, again applying precisely the same search, selection, exclusion and inclusion criteria, and documentation requirements as were applied in the subgroups’ electronic searches, were also conducted to supplement the electronic database searches. Manual searching of recent journals that publish research on specific NRP subgroup topics was performed to compensate for the delay in appearance of these journal articles in the electronic databases. Other manual searching was carried out in relevant journals to include eligible articles that should have been selected, but were missed in electronic searches.

Source of Publications: The Issue of Refereed and Non-Refereed Articles

The subgroup searches focused exclusively on research that had been published or had been scheduled for publication in refereed (peer reviewed) journals. The Panel reached consensus that determinations and findings for claims and assumptions guiding instructional practice depended on such studies. Any search or review of studies that had not been published through the peer review process but was consulted in any subgroups review was treated as separate and distinct from evidence drawn from peer-reviewed sources (i.e., in an appendix) and is not referenced in the Panel’s report. These non-peer-reviewed data were treated as preliminary/pilot data that might illuminate potential

Reports of the Subgroups 1-

Chapter 1: Introduction and Methodology

• Whether participants were drawn from urban, suburban, or rural setting
• List any pretests that were administered prior to treatment
• List any special characteristics of participants including the following if relevant:
• Socioeconomic status (SES)
• Ethnicity
• Exceptional learning characteristics, such as:
  • Learning disabled
  • Reading disabled
  • Hearing impaired
• English Language Learners (ELL)—also known as Limited English Proficient (LEP) students
• Explain any selection restrictions that were applied to limit the sample of participants (e.g., only those low in phonemic awareness were included)
• Contextual information: Concurrent reading instruction that participants received in their classrooms during the study
  • Was the classroom curriculum described in the study (code = yes/no)
  • Describe the curriculum
• Describe how sample was obtained:
  • Schools or classrooms or students were selected from the population of those available
  • Convenience or purposive sample
  • Not reported
  • Sample was obtained from another study (specify study)
• Attrition:
  • Number of participants lost per group during the study
  • Was attrition greater for some groups than for others? (yes/no)

4. Setting of the Study

• Classroom
• Laboratory
• Clinic
  • Pullout program (e.g., Reading Recovery© )
  • Tutorial

5. Design of Study

• Random assignment of participants to treatments (randomized experiment)
  • With vs. without a pretest
• Nonequivalent control group design (quasi experiment) (Example: existing groups assigned to treatment or control conditions, no random assignment)
  • With vs. without matching or statistical control to address nonequivalence issue
• One-group repeated measure design (i.e., one group receives multiple treatments, considered a quasi- experiment)
  • Treatment components administered in a fixed order vs. order counterbalanced across subgroups of participants
• Multiple baseline (quasi-experiment)
  • Single-subject design
  • Aggregated-subjects design

6. Independent Variables

a. Treatment Variables

• Describe all treatments and control conditions; be sure to describe nature and components of reading instruction provided to control group
• For each treatment, indicate whether instruction was explicitly or implicitly delivered and, if explicit instruction, specify the unit of analysis (sound- symbol; onset/rime; whole word) or specific responses taught. [ Note: If this category is omitted in the coding of data, justification must be provided.]
• If text is involved in treatments, indicate difficulty level and nature of texts used
• Duration of treatments (given to students)
  • Minutes per session
  • Sessions per week
  • Number of weeks

Reports of the Subgroups 1-

Methodology

• Was trainers’ fidelity in delivering treatment checked? (yes/no)
• Properties of teachers/trainers
• Number of trainers who administered treatments
• Teacher/student ratio: Number of participants to number of trainers
• Type of trainer (classroom teacher, student teacher, researcher, clinician, special education teacher, parent, peer, other)
• List any special qualifications of trainers
• Length of training given to trainers
• Source of training
• Assignment of trainers to groups:
  • Random
  • Choice/preference of trainer
  • All trainers taught all conditions
• Cost factors: List any features of the training such as special materials or staff development or outside consultants that represent potential costs

b. Moderator Variables List and describe other nontreatment independent variables included in the analyses of effects (e.g., attributes of participants, properties or types of text).

7. Dependent (Outcome) Variables

• List processes that were taught during training and measured during and at the end of training
• List names of reading outcomes measured
  • Code each as standardized or investigator- constructed measure
  • Code each as quantitative or qualitative measure
  • For each, is there any reason to suspect low reliability? (yes/no)
• List time points when dependent measures were assessed

8. Nonequivalence of Groups

• Any reason to believe that treatment/control group might not have been equivalent prior to treatments? (yes/no) - Were steps taken in statistical analyses to adjust for any lack of equivalence? (yes/no)

9. Result (for each measure)

• Record the name of the measure
• Record whether the difference—treatment mean minus control mean—is positive or negative
• Record the value of the effect size including its sign (+ or -)
• Record the type of summary statistics from which the effect size was derived
• Record number of people providing the effect size information

10. Coding Information

• Record length of time to code study
• Record name of coder If text was a variable, the coding indicated what is known about the difficulty level and nature of the texts being used. Any use of special personnel to deliver an intervention, use of special materials, staff development, or other features of the intervention that represent potential cost were noted. Finally, various threats to reliability and internal or external validity (group assignment, teacher assignment, fidelity of treatment, and confounding variables including equivalency of subjects prior to treatment and differential attrition) were coded. Each subgroup also coded additional items deemed appropriate or valuable to the specific question being studied by the subgroup members.

A study could be excluded at the coding stage only if it was found to have so serious a fundamental flaw that its use would be misleading. The reason(s) for exclusion of any such study was detailed and documented for the record. When quasi-experimental studies were selected, it was essential that each study included both pre treatment and post-treatment evaluations of performance and that there was a comparison group or condition.

Each subgroup conducted an independent re-analysis of a randomly designated 10% sample of studies. Absolute rating agreement was calculated for each category (not for forms). If absolute agreement fell below 0.90 for any

1-9 National Reading Panel

Methodology

References

Cooper, H., & Hedges, L.V. (1994). The handbook of research synthesis. New York: Russell Sage Foundation.

Snow, C. E., Burns, S. M., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press.

1-11 National Reading Panel

Executive Summary

P A R T I : P H O N E M I C A W A R E N E S S I N S T R U C T I O N

Executive Summary

Introduction

When today’s educators discuss the ingredients of effective programs to teach children to read, phonemic awareness (PA) receives much attention. However, not everyone is convinced. In education, particularly in the teaching of reading over the years, the choice of instructional methods has been heavily influenced by many factors, not only teachers’ own frontline experiences about what works, but also politics, economics, and the popular wisdom of the day. The pendulum has swung back and forth between holistic, meaning-centered approaches and phonics approaches without much hope of resolving disagreements. Meanwhile, substantial scientific evidence has accumulated purporting to shed light on reading acquisition processes and effective instructional approaches (Anderson et al., 1985; Adams, 1990; Snow, 1998). Many studies investigating the effectiveness of phonemic awareness instruction have contributed to this body of evidence. Proponents believe that this research holds promise of placing reading instruction on a more solid footing and ending the periodic upheavals and overhauls of reading instructional practices.

The purpose of this report of the National Reading Panel (NRP) was to examine the scientific evidence relevant to the impact of phonemic awareness instruction on reading and spelling development. In the analyses conducted, the NRP sought answers to questions such as the following: Is phonemic awareness instruction effective in helping children learn to read? Under what circumstances and for which children is it most effective? Were studies showing its effectiveness designed appropriately to yield scientifically valid findings? What does a careful analysis of the findings reveal? How applicable are these findings to classroom practice? To evaluate the adequacy and strength of the evidence, the NRP conducted a meta-analysis. The

literature was searched to locate all experimental studies that included a PA treatment and a control group and that measured reading as an outcome of the treatment.

There were several reasons why phonemic awareness instruction was selected for review and analysis. Correlational studies have identified phonemic awareness and letter knowledge as the two best school- entry predictors of how well children will learn to read during their first 2 years in school. This evidence suggests the potential instructional importance of teaching PA to children. Many experimental studies have evaluated the effectiveness of PA instruction in facilitating reading acquisition. Results are claimed to be positive and to provide a scientific basis documenting the efficacy of PA instruction. There is currently much interest in PA programs among teachers, principals, and publishers. State adoption committees have prescribed the inclusion of PA training in reading instruction materials approved for use in schools. It is thus important to determine whether PA instruction lives up to these claims and, if so, to identify circumstances that govern its effectiveness.

Phonemes are the smallest units constituting spoken language. English consists of about 41 phonemes. Phonemes combine to form syllables and words. A few words have only one phoneme, such as a or oh. Most words consist of a blend of phonemes, such as go with two phonemes, or check with three phonemes, or stop with four phonemes. Phonemes are different from graphemes, which are units of written language and which represent phonemes in the spellings of words. Graphemes may consist of one letter, for example, P, T, K, A, N, or multiple letters, CH, SH, TH, -CK, EA, IGH, each symbolizing one phoneme.

Phonemic awareness refers to the ability to focus on and manipulate phonemes in spoken words. The following tasks are commonly used to assess children’s PA or to improve their PA through instruction and practice:

2-1 National Reading Panel

Chapter 2, Part 1: Phonemic Awareness Instruction

1. Phoneme isolation, which requires recognizing individual sounds in words, for example, “Tell me the first sound in paste.” (/p/)
2. Phoneme identity, which requires recognizing the common sound in different words. For example, “Tell me the sound that is the same in bike, boy, and bell.” (/b/)
3. Phoneme categorization, which requires recognizing the word with the odd sound in a sequence of three or four words, for example, “Which word does not belong? bus, bun, rug.” (rug)
4. Phoneme blending, which requires listening to a sequence of separately spoken sounds and combining them to form a recognizable word. For example, “What word is /s/ /k/ /u/ /1/?” (school)
5. Phoneme segmentation, which requires breaking a word into its sounds by tapping out or counting the sounds or by pronouncing and positioning a marker for each sound. For example, “How many phonemes are there in ship? ” (three: /š/ /I/ /p/)
6. Phoneme deletion, which requires recognizing what word remains when a specified phoneme is removed. For example, “What is smile without the / s/?” (mile)

In the studies reviewed by the NRP, researchers used one or several of these tasks to assess how much PA children possessed before training and how much they had learned at the end of training. Also, these tasks were the basis for activities that children practiced during training. In some of the studies, children were taught to perform these tasks with letters, for example, segmenting words into phonemes and representing each with a grapheme. In other studies, phoneme manipulation was limited to speech.

To be clear, PA instruction is not synonymous with phonics instruction that entails teaching students how to use grapheme-phoneme correspondences to decode or spell words. PA instruction does not qualify as phonics instruction when it teaches children to manipulate phonemes in speech, but it does qualify when it teaches children to segment or blend phonemes with letters.

PA is thought to contribute to helping children learn to read because the structure of the English writing system is alphabetic. Moreover, it is not easy to figure out the system. Although most English words have prescribed spellings that consist of graphemes, symbolizing phonemes in predictable ways, being able to distinguish the separate phonemes in pronunciations of words so that they can be matched to graphemes is difficult. This is because spoken language is seamless; that is, there are no breaks in speech signaling where one phoneme ends and the next one begins. Rather, phonemes are folded into each other and are coarticulated. Discovering phonemic units requires instruction to learn how the system works.

Methodology

How was the analysis of the research

literature conducted?

Before conducting a meta-analysis, the NRP systematically searched the research literature relevant to PA instruction. After a methodology established by the Panel was followed, appropriate key words were entered to identify relevant studies in ERIC and PsycINFO. The search was limited to articles appearing in journals written in English, but no limit was placed on the year of publication. This yielded a total of 1,962 potentially relevant articles. Abstracts were printed and screened. In addition, references listed in these articles and in several review papers were hand- searched and screened. To qualify for analysis, studies had to meet the following criteria:

1. Studies had to adopt an experimental or quasi- experimental design with a control group or a multiple baseline method.
2. Studies had to appear in a refereed journal.
3. Studies had to test the hypothesis that instruction in phonemic awareness improves reading performance over alternative forms of instruction or no instruction.
4. Studies had to provide training in phonemic awareness that was not confounded with other instructional methods or activities.
5. Studies had to report statistics permitting the calculation or estimation of effect sizes.

Reports of the Subgroups 2-