Working Paper Series
This series features working papers completed by TERA staff and research collaborators. Working papers are preliminary versions meant for discussion purposes only in order to contribute to ongoing conversations about research and practice. Research papers have not undergone external peer review.
School Turnaround
This report presents the findings of a multi-year research project that sought to understand the iZone's underlying strategy and the dynamics of implementation as teachers and leaders attempted to maintain growth, expand to more schools, and at the same time, move toward more ambitious practices. Our research focused predominantly on math instruction and included a combination of surveys as well as interviews, observations, and focus groups in five case study schools (three elementary and two high schools). The results are particularly germane because, despite the national attention given to experiments like the ASD, the likeliest path to reforming most of the country's most under-performing schools goes through traditional urban districts like Shelby County. For this reason, our analysis has implications not just for the iZone, but also for policymakers, educators, and reformers across the country seeking to tackle similar challenges.
Recruiting and retaining teachers can be challenging for many schools, especially in low-performing urban schools in which teachers turn over at higher rates. In this study, we examine three types of school-level attributes that may influence teachers' decisions to enter or transfer schools: malleable school processes, structural features of employment, and school characteristics. Using adaptive conjoint analysis survey design with a sample of teachers from low-performing urban, turnaround schools in Tennessee, we find that five of the seven most highly valued features of schools are malleable processes: consistent administrative support, consistent enforcement of discipline, school safety, small class sizes, and availability of high-quality professional development. In particular, teachers rated as effective are more likely to prefer performance-based pay than teachers rated ineffective. We validate our results using administrative data from Tennessee on teachers' actual mobility patterns.
Teachers and Leaders
Despite their critical role in the preparation of pre-service teachers, very little research has explored the impacts of offering mentors professional development around how to coach and support their candidates. We conduct an evaluation of a professional development opportunity offered by the state of Tennessee to randomly assigned mentors at six different programs over the course of three years, investigating its impacts on the perceptions and practices of both the mentors themselves and their candidates. We find that professional development increases mentors’ frequency of coaching and mentors’ own instructional effectiveness, particularly in emphasized areas, as well as candidates’ employment rates, highlighting the potential of mentor professional development to improve the clinical placement experience as a whole.
Many prior studies have examined whether there are average differences in levels of teaching effectiveness among graduates from different teacher preparation programs (TPPs); other studies have investigated which features of preparation predict graduates’ average levels of teaching effectiveness. This is the first study to examine whether there are average differences between TPPs in terms of graduates’ average growth, rather than levels, in teaching effectiveness, and to consider which features predict this growth. Examining all graduates from Tennessee TPPs from 2010 to 2018, we find meaningful differences between TPPs in terms of both levels and growth in teaching effectiveness. We also find that different TPP features, including areas of endorsement, program type, clinical placement type and length, program size, and faculty composition explain part of these differences. Yet, the features that predict initial teaching effectiveness are not the same features that predict growth.
Examining the Human Capital of Formal Teacher Leaders (Hunter & Redding, 2020)
A growing body of work finds that instructional coaches (ICs) and teacher peer observers (POs) improve student learning, implying that policy might leverage these formal teacher leaders for student benefit. However, little is known about the observable characteristics of ICs and POs. Using unique statewide panel data from Tennessee, we describe the human capital (i.e. teaching experience, education level, effectiveness, and observation scores) of ICs and POs and the district and school settings where these formal teacher leaders work. We find that ICs and POs possess more human capital than other classroom teachers, and that these positive differences grow in magnitude as the concentration of economically disadvantaged students in a school rises. The evidence also suggests that ICs are more likely to work in districts with lower-performing teachers while working in schools with higher-performing teachers. Where POs work is mostly a function of district characteristics.
Workforce Outcomes of Program Completers in High-needs Areas (Bardelli & Ronfeldt, 2020)
For decades, federal and state agencies have identified teacher shortages in high-needs areas (HNAs), including science, mathematics, and special education, as a critical problem. Many states implemented policies and practices to recruit HNA-endorsed teachers, but little is known about how their workforce outcomes compare with other teachers. Leveraging statewide longitudinal data in Tennessee, we observe that the number of teachers who receive HNA endorsements has increased over time as the overall number of teachers prepared in the state has declined. HNA teachers are employed at higher rates and retained at similar rates as other teachers. HNA teachers have similar student achievement gains as non-HNA teachers. Though HNA and non-HNA teachers also have similar first year observation ratings, STEM and special education teachers improve at slower rates subsequently. Overall, findings suggest that efforts in Tennessee to recruit, prepare, and retain teachers with HNA endorsements have mostly been successful.
Prior work suggests that recent graduates from teacher education programs feel better prepared to teach and are more instructionally effective when they learned to teach with more instructionally effective cooperating teachers. However, we do not know if these relationships are causal. Even if they are, we do not know if it is possible to recruit cooperating teachers who are, on average, significantly more effective than those currently serving. This paper describes an innovative strategy to use historical administrative data on teachers to recommend the most instructionally effective and experienced teachers in various districts and subject areas to serve as cooperating teachers. In collaboration with a large teacher education program, partnering districts were randomized to receive either recommendation lists or use business-as-usual approaches. Those districts that received recommendations recruited significantly and meaningfully more effective and experienced cooperating teachers. Additionally, preservice student teachers who learned to teach in these same districts felt significantly better prepared to teach. This study offers an innovative and low-cost strategy for recruiting effective and experienced cooperating teachers and presents some of the first evidence that learning to teach with instructionally effective cooperating teachers has a causal impact on feelings of preparedness to teach.
Growing evidence suggests that preservice candidates receive better coaching and are more instructionally effective when they are mentored by more instructionally effective cooperating teachers (CTs). Yet, teacher education program leaders indicate it is difficult to recruit instructionally effective teachers to serve as CTs, in part because they worry that serving may negatively impact district evaluation scores. Using a unique dataset on over 4,500 CTs, we compare evaluation scores during years these teachers served as CTs compared to years they did not. In years they served as CTs, teachers had significantly better observation ratings and somewhat better achievement gains, though not always at significant levels. These results suggest that concerns over lowered evaluations should not prevent teachers from serving as CTs.
Principal Sorting and the Distribution of Principal Quality (Grissom, Bartanen, & Mitani, 2019)
Numerous studies document the inequitable distribution of teacher quality across schools. We focus instead on the distribution of principal quality, examining how multiple proxies for quality, including experience, teachers' survey assessments of leaders, and rubric-based practice ratings assigned by principals' supervisors, vary by measures of school advantage, using administrative data from Tennessee. By virtually every quality measure, we find that schools serving larger fractions of low-income students, students of color, and low-achieving students are led by less qualified, less effective principals. These patterns persist across urban, suburban, and rural settings. Both differential hiring/placement and differential turnover patterns by principal quality across school characteristics contribute to these patterns. Simulation evidence suggests that hiring and turnover vary in relative importance to principal sorting patterns according to the measure of quality examined, and that differential principal improvement across contexts may matter as well. Complementary analyses of national survey data corroborate our main results.
Several state policies link high-stakes consequences to teacher evaluation scores, which tend to be heavily weighted by observation scores. However, research has only recently investigated the validity of these scores in modern teacher evaluation systems. I contribute to this body of work by examining the sensitivity of observation scores to a novel source of bias: the differentiated assignment of observation by state policy. Several states differentiate the number of observations assigned to teachers. Although the receipt of observations should influence observation scores, the differentiated assignment of observations to teachers should not. I apply a two-stage least squares regression discontinuity design to teacher panel data, exploiting discontinuities in Tennessee's differentiated assignment of observations, and find strong evidence of substantially negative bias. Multiple sensitivity tests strongly suggest that the assignment of observations by state policy is the source of assimilation bias, but this suggestion is not definitive. Implications are discussed.
In the early 2010s, Tennessee adopted a new teacher evaluation system. Recent research finds Tennessee teacher effectiveness substantially and rapidly improved after this reform. However, there is little empirical research exploring which components of the reformed system might have contributed to this growth. Using longitudinal data, I apply a local regression discontinuity design to identify the effects of more frequent classroom observations, a cornerstone of Tennessee evaluation reform, on average student achievement scores. Much of the identifying variation is associated with an increase from one to two policy-assigned observations per year, potentially limiting the generalizability of results. However, most Tennessee teachers are assigned one or two observations by state policy, making this a margin of primary interest in the Tennessee context. Among teachers included in the research design, there is no evidence the receipt of an additional observation per year improved teacher effectiveness. Descriptive analyses suggest weak implementation of observational processes may explain the absence of positive effects. Implications are discussed.