Editor's Note: The content of this article was published by Pearson Assessments in the journal, EBP Briefs. It is offered for CEUs through SpeechPathology.com in cooperation with Pearson. The following supplemental materials are available for download:
Learning Objectives
After this course, readers will be able to:
- Identify the clinical question for an example case scenario
- List relevant search criteria for retrieving evidence related to the clinical question
- Identify factors to consider when evaluating the evidence
- Describe a clinical decision based on the evidence analysis
Structured Abstract
Clinical Question: Do preschool and kindergarten teachers (P) who participate in professional development programs that focus on emergent literacy skills with built-in individualized coaching (I) compared to those who do not participate in professional development programs (C) show enhanced knowledge and/or practice in the classroom environment (O)?
Method: Systematic Review
Study Sources: ERIC, PsycINFO, PubMed, Web of Science
Search Terms: teacher coaching AND language and literacy, teacher coaching professional development AND emergent literacy skills
Number of Included Studies: 9
Primary Results:
- Professional development programs with built-in individualized coaching were effective when targeting teachers’ code-focused and oral language instruction and improving environmental supports in the classroom but not effective when the outcome measures pertained to teachers’ knowledge and beliefs.
- Professional development programs that focused on one or two outcome measures and were shorter in duration displayed as great or greater effect sizes than more involved professional development programs.
Conclusions: Participating in a professional development program and receiving individualized coaching on a core set of emergent literacy instructional skills within a specific classroom context leads to improved educator practice and enhanced teacher-child interactions. Programs that incorporated videotaped feedback of teacher-child interactions during coaching sessions or written observations and feedback yielded better results than those that presented feedback verbally or through modeling. Further research is needed to determine the long-term effects of professional development with coaching on children’s literacy achievement as they enter grade school.
Clinical Scenario
Susan, a speech-language pathologist at a preschool in center city Philadelphia, was approached by her coworker, Amy, a lead teacher and frequent collaborator of Susan’s at the preschool. Recently, Amy received emails from kindergarten teachers concerned about the lack of literacy skills of children she taught in the previous year. Amy attended short seminars in the past regarding the core aspects of early language and literacy skills, such as phonemic awareness, alphabet knowledge, print awareness, and oral language. However, she does not feel competent implementing these in classroom activities. Amy contacted her district school board and asked for information regarding professional development programs that focus on emergent literacy skills for early school-age children. She was provided information regarding two professional development programs: The first program was an 18-hour group course focusing on providing teachers with strategies to enhance instruction of emergent language and literacy skills accompanied by three individual coaching visits on how to implement these skills into the classroom. The second program offered a 45-hour group course focusing on improving teacher knowledge and instruction of emergent language and literacy skills in addition to year-long coaching visits on a weekly basis in the classroom setting. Amy was hesitant to participate in the year-long program because of the required time commitment and constraints with her current schedule. However, she was worried that the shorter program would not be effective. To help Amy make an informed decision, Susan conducted a search for evidence-based research to determine the effectiveness of participating in a professional development program with individualized coaching on teacher practice and knowledge in relation to emergent literacy skills.
Background Information and Rationale
Approximately one-half to two-thirds of children in large urban centers attend nonparental childcare (Barnett & Yarosz, 2007), placing increasing pressure on educators to promote children’s school readiness. Emergent literacy skills such as phonological awareness, alphabet knowledge, print concepts, vocabulary, and oral language are foundations for reading and typically develop before kindergarten (Girolametto, Weitzman, & Greenberg, 2012). A longitudinal study conducted by Lonigan, Burgess, and Anthony (2000) found that more than half of the variance in children’s decoding abilities in kindergarten and Grade 1 was attributed to phonological awareness skills and alphabet knowledge learned in preschool. The quality of educator-child interactions, specifically, the quality of instruction that targets emergent literacy skills, is key to supporting early literacy learning (Milburn et al., 2015).
The need to improve early literacy outcomes originates from reports of major gaps in literacy and language skills in children at kindergarten entry that last throughout elementary school (Powell, Diamond, Burchinal, & Koehler, 2010). Adult-child interactions that focus on print provide opportunities for children to participate in conversations about the relationship between letters, sounds, and words and learn about the meaning of print (Milburn et al., 2015). Because many children spend a considerable amount of time with preschool teachers, it is important that early childhood educators have knowledge of foundational literacy as well as the ability to provide individualized instruction to young learners with diverse needs.
Professional development programs on emergent language and literacy skills for early childhood educators have been widely used to enhance teachers’ ability to provide a more language-enriched curriculum for young children. The term professional development (PD) is used in educational settings to refer to a wide variety of specialized training, formal education, or advanced professional learning intended to help administrators, teachers, and other educators to improve their knowledge, competence, skill, and effectiveness on a given topic (Great Schools Partnership, 2013). The specialized in-service training and extended coursework focus on specific skill-building, delivered by an expert, and include activities that have direct application to recommended practice. A systematic review of nine studies measuring the effects of PD on student achievement found that studies that had more than 14 hours of PD showed a positive and significant effect on student achievement. However, PD programs with a duration between 5 and 14 hours showed no statistically significant effects on student achievement (Yoon, Duncan, Lee, Scarloss, & Shapley, 2007). Garet, Porter, Desimone, Birman, and Yoon (2001) found that PD is most effective when it focuses on improving educators’ content knowledge, includes opportunities for active learning, and is integrated with other growth opportunities, such as individualized coaching and meaningful feedback.
Coaching consists of frequent interactions over a brief (Milburn, Girolametto, Weitzman, & Greenberg, 2014) or extended (Neuman & Cunningham, 2009) period of time between an experienced expert and an individual who desires to learn a specific skill or behavior. In more recent research, one-on-one coaching that responds directly to what happens in individual classrooms appears to be central to changing instructional interactions between the educator and the child. The current literature suggests that individualized and sustained coaching that provides guidance and feedback to teachers on the implementation of evidence-based practices in their own classrooms is more effective than one-hour workshops in improving instruction quality (Powell et al., 2010).
Speech-language pathologists (SLPs) are playing more and more important roles in the prevention and treatment of language-based literacy disorders (American Speech-Language-Hearing Association [ASHA], 2016). As the SLP’s scope of practice continues to evolve, clinicians are increasingly charged with the task of educating stakeholders, including parents, teachers, and other school professionals about the SLP’s role in addressing the oral language foundations of literacy disorders, advocating for clients, and delivering multitiered reading interventions (i.e., Response to Intervention) to reduce children’s risk for reading disability (Justice, 2006). Because emergent literacy skills are causally related to later reading achievement, they are high-priority teaching targets in the preschool setting. Given SLP’s unique training regarding the various domains of oral language and the link between oral and written language, SLPs have much to offer and often act as the agent of PD and coaching to enhance skill development in teachers (Girolametto et al., 2012).
Clinical Question
This brief’s purpose is to answer a clinical question important for early education teachers who are frequent collaborators and informants for school-based SLPs. Using the PICO framework, Susan constructed her question: Do preschool and kindergarten teachers (P) who participate in professional development programs that focus on emergent literacy skills with built-in individualized coaching (I) compared to those who do not participate in professional development programs (C) show enhanced knowledge and/or practice in the classroom environment (O)?
Search for the Evidence
Susan adopted six criteria for her search: (1) the participants involved in the study must teach preschool or kindergarten, (2) the participants must include Englishspeaking participants only, (3) the PD program must include oral language and emergent literacy skills as the training focus, (4) the study must include a business-as-usual control group that did not receive any form of PD, (5) the outcome measures must assess educators’ knowledge and/or practice, and (6) the studies were published in a peer-reviewed journal between 1990 and the present.
Susan searched the following four databases that covered the fields of education, psychology, and medicine: Educational Resource Information Center (ERIC), PsycINFO, Web of Science, and PubMed. The first set of search terms, teacher coaching AND language and literacy, yielded 144 unique articles from the four databases. After reading through the abstracts and/or full text, Susan found eight articles that met the selection criteria. Susan conducted a second search to locate additional articles using the search terms teacher coaching professional development AND emergent literacy skills. Out of a total of 26 unique articles yielded, Susan found one that met the criteria of her search; therefore, Susan found a total of nine articles relevant to the clinical question (see Table 1 for a summary of the articles) and began evaluating the evidence. Figure 1 presents a flowchart detailing the search and selection process.
Evaluating the Evidence
Assessing the Effect of Intervention
Overall, the nine studies demonstrated strong experimental design, including eight randomized control trials and one quasi-randomized control trial. The length of the PD coursework varied from 18 hours to 45 hours, and the duration of coaching ranged from three to 32 sessions with each session’s duration between 60 and 90 minutes. The coaches included early childhood educators, literacy coaches, special education teachers who held a master’s or doctoral degree, and SLPs.
Each study was critically appraised to determine the quality of evidence using Law, Garrett, and Nye’s (2004) 3-point scale (0 = inadequate, 1 = unclear, 2 = adequate). A total of 11 attributes were rated: randomization, recognizable participants, baseline equivalence, blinding, reliable outcome measures, statistical significance, practical significance, confidence intervals for effect sizes, attrition, teacher-intervention confound, and treatment fidelity. Attributes such as randomization, recognizable participants, and baseline equivalence contribute to the adequacy of the study design. If explained well, these qualities allow replication of the study. Careful consideration of blinding, reliable outcome measures, teacher-intervention confounds, and treatment fidelity ensures that rigorous procedures are performed to prevent confounds from affecting the outcome measures. Incorporating statistical significance, practical significance, and confidence intervals enables evaluation of the claims made in the text by the authors and allows for quantification of the effectiveness of intervention. Table 2 provides the critical appraisal rating for each study. Two raters independently rated all nine studies on the 11 attributes. Inter-rater reliability ranged from 81%–100%. All differences were within one point and resolved with discussion.
Calculating and Interpreting the Intervention Effect
Effect sizes (Cohen’s d) were calculated to determine the effectiveness of PD intervention plus individualized coaching using an online calculator (Thalheimer & Cook, 2002). Cohen’s d was interpreted using the following scale: An effect size of .20–.44 is considered small, .50–.74 is medium, .80–.94 is large, 1.0–1.44 is very large, and 1.5 or above is huge (Cohen, 1988). Because of variations in the dependent measures between studies, the outcome measures were grouped into five categories: teacher knowledge and beliefs of emergent literacy skills; teacher practice of code-focused instruction (i.e., phonemic awareness, sound awareness, alphabet knowledge, print concept); teacher practice of oral language instruction (i.e., conversational strategies, storybook reading discussion, vocabulary teaching); general teacher practice (i.e., when multiple aspects of emergent literacy instruction were aggregated into one outcome measure); and changes in classroom environment.
Two studies (Girolametto et al., 2012; Milburn et al., 2014) used the PD program ABC and Beyond™: The Hanen Program® for Building Emergent Literacy in Early Childhood Settings (Hanen Centre, 2016). Both achieved very positive results and a high-quality appraisal score. In both studies, the experimental group participated in an 18-hour group training, which included four workshops that focused on promoting the following six building blocks of literacy: oral language, vocabulary, story comprehension, language of learning, print knowledge, and phonological awareness. According to Girolametto et al. (2012), the procedures used in all four workshops included a review of the previous week’s content, interactive lectures, small group discussions, role-play activities, and completing action plans for strategy implementation in the classroom. In addition, educators in the intervention group participated in three one-hour individual coaching sessions with an SLP. In these sessions, educators were videotaped during a small group literacy activity and coaches provided feedback as needed.
Girolametto et al. (2012) demonstrated very large effect sizes for both oral language and code-focused instruction in the experimental group compared to the control group. The authors believed that the encouraging results may be, in part, due to how the coaching sessions were conducted. Using videotapes during coaching sessions and immediately viewing the videos enabled educators to reflect on their practice and gave coaches the opportunity to provide specific feedback to improve future adult-child interactions (Girolametto et al., 2012). Another important feature of this study is that coaching was conducted by an SLP. Seasoned SLPs who had extensive knowledge in language and literacy facilitation strategies may have enabled the gains in teachers’ practice.
Milburn et al. (2014) demonstrated similar results to Girolametto et al. (2012) and observed higher quality oral language instruction with the experimental group rather than the control group. This study also utilized SLPs as coaches and videotaped interactions for the purpose of reflection and feedback. In addition, the PD program provided instructions on using shared book reading to ensure educators used strategies in an activity that occurs regularly in preschool classrooms, which may have also contributed to the successful outcomes (Milburn et al., 2014).
Two studies used the Exceptional Coaching for Early Language and Literacy (ExCELL) PD intervention model (Hindman & Wasik, 2011, 2012). The ExCELL program features five modules: oral language, phonemic awareness, alphabet knowledge, print awareness/writing, and book reading. Each month for an entire academic year, teachers attended a three-hour workshop, focusing on one aspect of a module. In addition, within a week of the workshop, the coaches visited each teacher’s classroom, provided individualized guidance, and modeled the target strategies. One to two weeks later, the coaches returned to observe teachers’ use of the target strategies and provide feedback (Hindman & Wasik, 2011). Hindman and Wasik (2012) found that the experimental group achieved significantly higher quality of general practice than the control group post-intervention. The PD’s long duration plus coaching intervention could be one explanation for the positive results. However, Hindman and Wasik (2011) demonstrated a much-reduced intervention effect on teachers’ knowledge of emergent literacy. One possible explanation for the lack of change in the 2011 study is that the instruments used to measure teacher knowledge (i.e., multiple choice and true/false exam questions) may not be sensitive to changes. This pattern is also noted in other studies in this review that measured teacher knowledge (Neuman & Cunningham, 2009; Neuman & Wright, 2010; Piasta et al., 2017).
McCollum, Hemmeter, and Hsieh (2011) examined year-long coaching on educators’ use of instructional skills in three clusters of emergent literacy skills. Cluster A included vocabulary, comprehension strategies, and story structure; cluster B focused on sound awareness and the alphabetic principle; and cluster C targeted print concepts and written language. Teachers initially participated in a 10-hour orientation and were introduced to important areas of emergent literacy skills. Coaching occurred biweekly for an entire academic year, with five visits on each of the three clusters for a total of 15 visits; the length of each visit was unspecified. McCollum et al. (2011) found that the experimental group outperformed the control group only moderately for oral language instruction but quite significantly for classroom environment and code-focused instruction. The authors noted that despite random assignment, there was a significant pre-intervention difference favoring the intervention group for the oral language component, with no significant group difference post-intervention. Therefore, the smaller effect size for oral language instruction may reflect a ceiling effect for teachers who received the intervention (McCollum et al., 2011). The huge effect sizes for environment and code-focused instruction may be attributed to the length of coaching provided for each cluster of skills. Also, during each visit, the coach provided written observations and feedback about what the educator could improve on in subsequent sessions. The authors stated that the increased classroom environment ratings may be attributed to the direct instructions from the coaches regarding how to improve the broader classroom environments. Also, as teachers gain more knowledge in instructional behavior, they develop a deeper understanding of how to modify the environment to provide additional opportunities for emergent literacy learning throughout the day.
Neuman and Cunningham (2009) implemented a 45-hour, 15-week course with year-long individual coaching sessions. The course focused on developing teachers’ knowledge in the following areas: oral language comprehension, phonological awareness, letter knowledge and the alphabetic principle, print convention, strategies for working with second-language learners, literacy assessments, parental role in early language and literacy development, and linkages between literacy and other aspects of the curriculum. Each class used a lecture format with video examples to present the week’s topic, followed by hands-on activities designed to link concept to practice. Coaching was held weekly for approximately 90 minutes each session for a total of 32 sessions. Neuman and Cunningham (2009) found that the experimental and control groups did not differ in knowledge and beliefs; however, teachers who participated in the intervention achieved moderately higher scores on general teaching practice quality and much higher scores on classroom environment quality. In other words, the PD program brought the most changes in the classroom environment and the least amount of change in teacher knowledge and beliefs.
Neuman and Wright (2010) targeted the same emergent language and literacy skills as did Neuman and Cunningham (2009) and evaluated the same outcome measures. The primary difference between the two studies was that Neuman and Wright (2010) did not have a separate coursework component. Instead, emergent literacy knowledge was infused throughout the extensive coaching sessions for a total of 30 hours. Neuman and Wright (2010) achieved smaller effect sizes than Neuman and Cunningham (2009). The two groups did not differ on the knowledge and beliefs measure or on teaching practice ratings; however, the intervention group did better than controls on classroom environment ratings. Neuman and Wright (2010) stated that it was harder to find changes in teaching practice than in the structural environment for literacy when using the specific measuring tool. Another potential reason for the reduced effect sizes in Neuman and Wright (2010) could be the much shorter intervention duration.
Piasta et al. (2017) had a similarly high intervention intensity like Neuman and Cunningham (2009). In this study, the experimental group received 30 hours of PD program delivered in a workshop format with individualized coaching for 4 to 6 hours per month throughout an 18-month period. The PD aimed to improve early childhood educators’ knowledge, beliefs, and practices, with content derived from five domains: environment, play, oral language, early reading, and early writing (Piasta et al., 2017). During each workshop, the content was presented by experienced educators followed by demonstrations and opportunities for the educators to simulate the ideas into classroom situations and activities. What differentiated this study from the other studies in the current review was that the PD was implemented in a real-world context (i.e., outside of researcher-controlled settings) on a larger scale (i.e., across the state of Ohio).
Piasta et al. (2017) demonstrated virtually no difference between intervention and control groups for knowledge and beliefs, oral language instruction, and environment. The duration and intensity of the intervention could not be the reason for its lack of impact. On the other hand, the intervention used in this study was developed and implemented by contractors of the state department of education and had no previous evidence of efficacy (Piasta et al., 2017), unlike the other studies in this review in which the intervention was created, implemented, and monitored by researchers. Piasta et al. (2017) stated that the intervention content may not be well aligned with the outcome measures or may not be nuanced enough to capture changes. The study used four measures to assess teacher knowledge; however, the intervention targeted specific classroom language and literacy practices, such as oral language and code-focused instruction (Piasta et al., 2017). The authors also stated that the lack of effects can be attributed to the extensive, but not sufficiently in-depth, content. Moreover, although coaching was expected to align with the PD content, the actual coaching interactions targeted a wide variety of topics beyond the practices emphasized in the PD (Piasta et al., 2017). Additionally, both the quality of coaching and the exposure to coaching was variable; not all educators experienced 4 to 6 hours of coaching per month (Piasta et al., 2017). Lastly, this study had a much larger sample size than the other studies reviewed here. Implementing the intervention would have required many more resources, which may also be the reason why this study did not report a systematic way of ensuring treatment fidelity.
Powell et al. (2010) completed a study using an intervention entitled Classroom Links to Early Literacy. The purpose of this program was to improve teachers’ literacy instruction on code-focused skills, including phonological awareness and letter knowledge and oral language skills, such as vocabulary and listening comprehension. The intervention was held in a two-day workshop for a total of 16 hours, followed by individualized 2-hour coaching sessions from a literacy coach, for a total of seven sessions across one academic semester. Powell et al. (2010) reported a huge difference between intervention and control groups in general environment quality. However, between-group differences in code-focused instruction and oral language instruction were quite small. According to the authors, the lack of change in oral language and code-focused instruction may be attributed to the broad content coverage of the intervention program. Each coaching session typically focused on a new topic within one of the outcome areas. Instead, teachers may benefit more from repeated exposure to a specific strategy in consecutive coaching sessions (Powell et al., 2010).
The Evidence-Based Decision
At the beginning of this brief, a clinical scenario was presented in which Amy, a public preschool teacher, asked Susan, the school SLP, to conduct an evidence-based search to determine the efficacy of participating in a professional development program with individualized coaching to enhance teacher knowledge and practice of emergent language and literacy skills. This review and analysis of nine articles indicated that professional development with individualized coaching enhances overall classroom environment and teacher practice but has not shown reliably significant effects for increasing teacher knowledge.
Because Amy already possessed foundational knowledge of emergent language and literacy skills, she would benefit the most from a program that focused primarily on enhancing teacher practice and environment. This aligns with the evidence compiled in this review suggesting that the knowledge component may be the hardest to change. However, it is unclear why studies measuring educators’ knowledge (Neuman & Cunningham, 2009; Neuman & Wright, 2010; Piasta et al., 2017) did not yield positive effects. Regardless, it is curious that a dissociation between educators’ knowledge and instructional practice postintervention can happen (e.g., Neuman & Cunningham, 2009; Neuman & Wright, 2010), a phenomenon that warrants future research. The review of evidence also shows that professional development programs that had a broad instructional focus (e.g., Neuman & Wright, 2010; Piasta et al., 2017; Powell et al., 2010) tend to not fare as well as those that had a narrower instructional focus (Girolametto et al., 2012; Hindman & Wasik, 2012; Milburn et al., 2014). Therefore, for Amy to receive the most benefit from participating in professional development plus coaching, the intervention should focus on improving one or two outcomes (e.g., teacher practice) as opposed to three or more (e.g., teacher practice, environment, knowledge, and beliefs).
The manner of feedback delivery may also contribute to post-intervention results. Studies that incorporated videotaped feedback of teacher-child interactions during each coaching session (Girolametto et al., 2012; Milburn et al., 2014) or written observations and feedback (McCollum et al., 2011) yielded better results than those in which coaches provided feedback verbally or through modeling (Neuman & Cunningham, 2009; Neuman & Wright, 2010).
Another factor to consider is the intervention’s duration. One might assume that the longer and the higher dosage the intervention is, the better the outcomes; however, this may not always be the case. The studies that utilized 40 or more hours of training (Hindman & Wasik, 2011; Neuman & Cunningham, 2009; Neuman & Wright, 2010; Piasta et al., 2017) did not yield greater changes in teacher outcome measures than studies that provided less than 25 hours of interventions (Girolametto et al., 2012; McCollum et al., 2011; Milburn et al., 2014).
After considering these various factors (i.e., content focus, feedback delivery method, dosage), Susan recommended that Amy participate in ABC and Beyond: The Hanen Program for Building Emergent Literacy in Early Childhood Settings (Hanen Centre, 2016). The studies conducted by Girolametto et al. (2012) and Milburn and colleagues (2014) provided strong evidence to support the efficacy of this program in improving teachers’ oral language and code-focused instruction. Both studies had a short duration of about 21 hours of workshop and coaching. An intervention that yields improved teacher practice, while also requiring a shorter duration, would best suit Amy’s needs. The two studies that used this intervention (Girolametto et al., 2012; Milburn et al., 2014) also demonstrated high qualities in the study design and reporting, bolstering the recommendation. Although Susan was enthusiastic about the evidence for effectiveness of the Hanen program, she also pointed out to Amy that the researchers who conducted the studies were also affiliated with the Hanen Centre; therefore, these positive results should be interpreted with caution. Susan clarified that the research in this area is still growing and further research is needed to determine the long-term effects of professional development with coaching on children’s literacy achievement as they enter grade school.